Chat Direct to your local model — each send is a curl on your Mac
Training reading heartbeat…
The training pipeline how customer reality becomes the next HutFin AI — every step gated
Live progress lands in the card above the moment a run starts (the GB10 wrapper POSTs its heartbeat to /api/admin/training). Full detail: Models ▾ → Plane / Architecture.
Machines who does what
Measured 08-08, identical weights: M5 82.9 tok/s single · MSI 60.3 single / 225.6 tok/s aggregate @ 8 users.
Products in the stack every tool that touches the model lifecycle
| Product | Role | Where |
|---|---|---|
| HutFin AI 1.0 | the model — Qwen3.6-35B-A3B MoE (Q6_K · MLX 6-bit · CUDA) | M5 · MSI |
| Unsloth | LoRA fine-tuning engine (Unsloth-first doctrine) | MSI GB10 |
| NGC PyTorch + CUDA 13 | training container, 96.3 TFLOPS verified | MSI GB10 |
| Ollama | serving engine — the fleet-wide insurance policy | M5 :11434 · MSI :11434 |
| mlx-vlm / MLX-LM | Apple-GPU serving + vision tower (v5: :11902 batching) | M5 :11700/:11902 |
| llama-server | slot-based concurrency (the 225 tok/s @8 result) | MSI (bench) |
| Qdrant + BGE-M3 | ONE vector engine — listings + knowledge, hybrid + rerank | M5 :11903 |
| Valkey | L1 events · session pub/sub | M5 :11905 |
| DuckDB | Tier-0 exact counts + deterministic CRE calc | M5 |
| XGBoost | price estimator — banded, R² gate, monthly retrain | M5 :11600 |
| DataGate | custody: passport + vault + journal, permanent | M5 :11951 |
| SearXNG | Tier-3 web fallback, cited | Studio :8787 |
| logging-router | every turn → S3 · failover · model affinity | Studio :8090 |
| S3 | query-log corpus · fleet registry · training heartbeat | AWS |
| Caddy + Cognito | edge TLS + team-only SSO (hutfin-portal group) | gateway |
Server checking…
These act on the currently active Mac (named above). To switch which Mac serves, or turn an individual Mac on/off, use the Server Fleet tab.
Pull a model
Downloads from the Ollama registry onto your Mac. Big models take a while.
Installed models —
MODEL SETTINGS
Defaults are Ollama's. Recommended is the model vendor's guidance for vera:1.0 (temperature 0.7, top_p 0.8, top_k 20) — what this panel uses. Leave a custom field blank to fall back to the Ollama default.
| SETTING | DEFAULT | RECOMMENDED | YOUR VALUE |
|---|
Save settings keeps these as your per-request options in this browser. Apply to server bakes temperature/top_p/top_k/context into the selected model on its Mac for everyone (persistent). The system prompt also ships as the built-in default, so every device starts with it.
AVAILABILITY SCHEDULE
Keeps this model loaded during the window via a cron job on your Mac. Runs even when this app is closed. Times use your Mac's local clock.
No schedule set for this model.
Connect from anywhere
All traffic flows: your browser → CloudFront (WAF + TLS) → Caddy on the AWS gateway → reverse SSH tunnel → Ollama on the Mac. Requests are rate-limited at the edge; backend origins never reach the browser bundle.
none — public endpointList models Chat (streaming)
Keys are stored on the gateway and listed here for the authenticated admin. Copy a key when you create it. (The public chat path is open; these keys are for the developer API surface.)
Server Fleet Macs serving models, reached via reverse tunnels
Add a server
Sessions Every user session — logged to S3 (location fills in once CloudFront fronts it)
Model Console Fixed commands only — list · ps · load · unload · pull · run
Type "help" and press Enter.