Local AI Advisor

Build my local AI stack

From hardware to coding agent, one layer at a time

Choose your machine, what you’ll do and which tool you use. We pick the runtime, model and API that make the whole chain comfortable — and explain why.

Five connected parts of a local AI stack: hardware, runtime, model, local API, and AI tool

Hardware

Which Mac?
Chip
Memory
Mac mini M4 Pro 64 GB· 64 GB unified memory · 273 GB/s

Workload & tool

Use case
AI tool
Project size, agents, other apps
Coding style
Repository size
Repository size is a signal for retrieval and prompt-processing load — the whole repo never goes into the context.
How does the agent behave?
What else do you normally run while coding?
This memory is subtracted before the model is considered. Very important on 16–32 GB machines.

Override layers (optional)

Leave on Auto to let the engine choose.

Model
Quantization
Runtime
ComfortableMedium confidence

Comfortable for medium-scale agentic coding

Yes — this should be comfortable for agentic coding.

  1. Hardware

    Mac mini M4 Pro 64 GB

    64 GB unified memory

  2. Runtime

    LM Studio

    METAL backend · GGUF weights

  3. Model

    Recursive Flint (40B-A2.5B) — MXFP4

    20 GB weights · 32K context

  4. Local API

    OpenAI-compatible endpoint

    http://localhost:1234

  5. AI tool

    OpenCode

    Configure a provider with @ai-sdk/openai-compatible and the local base URL.

Full analysis & what-if

Set it up

Recursive Flint (40B-A2.5B) — MXFP4 on LM Studio, used from OpenCode. Replace placeholders like <model-name> with the name your runtime shows.

  1. 1

    Install LM Studio

    Download from lmstudio.ai.

  2. 2

    Download Recursive Flint (40B-A2.5B) (MXFP4, GGUF)

    In the model browser pick the GGUF build at MXFP4.

  3. 3

    Start the local server

    Developer tab → Start server (port 1234). Set context length to 32768 when loading the model.

  4. 4

    Add the provider to opencode.json

    {
  "provider": {
    "local": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:1234/v1" },
      "models": { "<model-name>": {} }
    }
  }
}

Why this stack?

Why?

  • Model fits with approximately 30 GB of memory headroom.
  • LM Studio supports hardware acceleration on this machine (Metal (Apple GPU)).
  • OpenCode can connect to LM Studio (officially supported).
  • Generation (65–90 tok/s) should be appropriate for agentic coding.
  • Ingesting a prompt of 18K tokens takes ≈20 s; prompt processing keeps up with agentic coding.
  • 32K working context is reasonable.
  • Good coding capability for agentic coding. Tool calling is reliable.

Why “Comfortable” rather than “Excellent”?

  • Workload suitability: Good coding capability for agentic coding. Tool calling is reliable.
  • Prompt processing: Ingesting the first prompt (18K tokens) takes ≈20 s; each of the ~25 follow-up agent steps takes ≈10 s (≈4 min per task).

How sure are we?

Medium confidence

Estimated from bandwidth and compute, calibrated against 2 verified benchmarks of other models on this same chip (measured with a different engine, so calibration is partial).

  • Calibrated with verified benchmarks of other models on this chip.
  • MoE performance varies more between runtimes.