AI Tools 8 min read

🖥️ Running Qwen3-Coder-30B Locally with LM Studio and pi

Local LLM Qwen3-Coder LM Studio pi Coding Agents

Introduction

A 16 GB laptop GPU is enough to run a genuinely capable coding model entirely on your own machine — no API keys, no data leaving the box. The model I settled on is Qwen3-Coder-30B-A3B-Instruct [1]: a Mixture-of-Experts model with 30 billion total parameters but only ~3 billion active per token, so it reasons like a big model while running at the speed of a small one.

This post is the settings sheet I wish I'd had: the exact LM Studio load configuration, the one VRAM rule that decides whether the model flies or crawls, how to wire it into pi (a terminal coding agent) [2], and honest benchmarks from driving it against a real codebase. Everything here was measured on a laptop RTX 5080 (16 GB VRAM).

⚠️ The one rule: never fill VRAM to the top

This is the single most important thing, so it goes first. The Q4_K_M quantization of this model is about 17.4 GB of weights — which is larger than a 16 GB card. That means some layers must be offloaded to the CPU no matter what; the full model physically cannot sit on the GPU.

The trap is the "max GPU offload" setting. Turn it up and LM Studio dutifully tries to place everything on the GPU. On Windows, instead of erroring, the NVIDIA driver silently spills the overflow into shared GPU memory — system RAM accessed over the PCIe bus. Every token then shuttles data across PCIe, and throughput collapses:

Same model, same GPU: 0.6 tokens/sec at "max" offload (VRAM 97% full, spilling) versus ~24 tokens/sec with the offload dialed back to leave headroom. A ~40× difference from one setting.

The fix is counterintuitive: put less on the GPU, not more. Leave roughly 1.5–2 GB of VRAM free for the KV cache and driver overhead, and verify the real number with nvidia-smi rather than trusting the loader's estimate. For a MoE model that overflows VRAM, partial offload beats "max" every single time.

🎛️ LM Studio load settings

Set these in the model's load panel before loading it (in LM Studio: My Models → the model's load configuration). The values below are the tested 16 GB baseline.

Setting Value Why
GPU Offload 0.60 The critical dial. Offload ~60% of layers on 16 GB; target ~1.5–2 GB free VRAM. Verify with nvidia-smi.
Context Length 32768 Agent harnesses need room. An 8K window silently truncates the system prompt and file context, causing "forgotten instructions." 32K is the working floor.
Flash Attention ON Large KV-cache savings — this is the headroom that makes 32K context affordable without spilling.
KV Cache Quantization Q8_0 Roughly halves per-token context memory. Enables 48–64K context, or spend the freed VRAM on a higher offload ratio. Negligible quality loss at Q8.

For sampling, start deterministic for coding — temperature 0.2–0.3. For more exploratory generation, Qwen's recommended defaults are temp 0.7 · top_p 0.8 · top_k 20 · repeat_penalty 1.05 [1].

Scale the offload to your card

The principle is identical at every tier: leave headroom. Only the 16 GB row is measured; the others are sensible starting points to verify and adjust with nvidia-smi.

VRAM GPU offload Context ~Speed Feel
12 GB ~0.35 16–24K 10–15 tok/s Usable, slow
16 GB (tested) 0.60 32K ~24 tok/s Sweet spot
24 GB ~0.90 32–64K 50–70 tok/s Comfortable
32 GB+ max 64K+ Fast Fully on-GPU

Below 24 GB the full weights can't sit entirely on the GPU, so throughput is capped by the CPU-offloaded layers. That's the price of running a 30B model on a small card — and it's still worth paying.

🚀 Start the server

LM Studio exposes an OpenAI-compatible endpoint that pi (and other tools like Continue or Cline) talk to. Its lms CLI [3] makes this scriptable:

# start the local server → http://localhost:1234/v1
lms server start

# load the model with the tuned settings (16 GB baseline)
lms load qwen3-coder-30b-a3b-instruct --gpu 0.60 --context-length 32768 --identifier coder -y

# confirm it's resident and see the real context window
lms ps

Note that Flash Attention and KV-cache quantization are set in the GUI load panel, not on the CLI — enable them there once, and this lms load command reproduces the rest.

🔌 Wire up pi

pi is a terminal coding agent with read, bash, edit, and write tools [2]. The community pi-lmstudio extension [4] bridges it to LM Studio and auto-discovers loaded models.

1. Install the extension — it connects to http://127.0.0.1:1234 by default:

pi install npm:pi-lmstudio

2. Make it the default model by editing ~/.pi/agent/settings.json:

{
  "packages": ["npm:pi-lmstudio"],
  "defaultProvider": "lmstudio",
  "defaultModel": "qwen3-coder-30b-a3b-instruct"
}

3. Or pick it live inside pi with the /model command or Ctrl+P, searching for models prefixed lmstudio. Then just run pi in your project. For a remote or custom endpoint, add { "url": "…" } to ~/.pi/agent/lmstudio.json.

📊 What to expect

I measured this driving pi against a real 121-file C# codebase. The encouraging finding: quality held up — the model stayed grounded and did not hallucinate. Latency, not intelligence, is the ceiling, and it scales with how far a task fans out across files.

Task type Time Result
Search / locate (ripgrep + read a couple files) 27–55 sec Accurate; every cited file verified to exist
Single-file edit or explanation ~1 min The daily-driver sweet spot
Multi-file synthesis (trace a pipeline across 5 projects) ~5 min Correct — all 9 cited classes were real, zero hallucination

Rule of thumb: it's excellent for scoped work — find X, explain Y, edit this file — and tedious for broad autonomous runs that fan out across a whole repo. Keep tasks focused, and reach for a cloud model on the big fan-out jobs.

🧭 Conclusion

A 30B MoE coding model on a 16 GB laptop is a real, private, offline pair-programmer — provided you respect the one rule: leave VRAM headroom instead of maxing the GPU offload. Get that right, set a 32K context so agent harnesses don't choke, wire it into pi, and you have a capable local assistant for scoped coding tasks that never sends a byte to the cloud.

References

  1. Qwen Team. "Qwen3-Coder-30B-A3B-Instruct." Hugging Face, 2025. Available: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct
  2. Zechner, Mario. "pi — a coding agent for the terminal." GitHub Repository, 2025. Available: https://github.com/mario-zechner/pi-coding-agent
  3. LM Studio. "lms — LM Studio's CLI for models, server, and runtime." LM Studio Developer Docs, 2025. Available: https://lmstudio.ai/docs/developer
  4. pi-lmstudio. "LM Studio integration extension for the pi coding agent." npm Registry, 2025. Available: https://www.npmjs.com/package/pi-lmstudio
Back to Blog
Local LLM Qwen3-Coder LM Studio pi Coding Agents