Most coding agents are monsters. They ship with a dozen tools, balloon to gigabytes of dependencies, and quietly route every keystroke to a frontier API in the cloud. Pi, the terminal coding agent built by Mario Zechner under the earendil-works umbrella, goes the opposite direction: four tools, a tiny system prompt, and a design philosophy that treats the model as the intelligence and everything else as plumbing. That minimalism is exactly what makes Pi one of the best agents to run completely offline.
Pi crossed 100,000 stars on GitHub while staying MIT licensed. Its core gives the model four tools and nothing more: read, write, edit, and bash. Every capability beyond that arrives as an opt-in extension or skill, which means you control exactly what a local model has to juggle at inference time. For the "Deep-Dive Technical Analysis" crowd, that tradeoff is the whole ballgame: a lean agent gives a small model the best possible chance of staying coherent over a long session.
Why a Four-Tool Agent Shines on Local Hardware
The key insight is token efficiency. Pi keeps its system prompt small, leaving more of the context window for actual work: file contents, tool output, and conversation history. A frontier model can afford to waste tokens on boilerplate. A 26-billion-parameter local model cannot. When every token is expensive in terms of memory bandwidth and latency, reducing prompt noise is measurable engineering, not cosmetic trimming.
Pi also sandboxes nothing by default. There is no built-in permission layer restricting filesystem, process, or network access. It runs with whatever permissions the launching user has. That is a deliberate posture: enforcement is delegated to the containerization strategy you choose. The project documents three patterns, Gondolin (keep Pi and provider auth on the host while tools and commands run in a local Linux micro-VM), plain Docker for simple isolation, and OpenShell for policy-controlled sandboxing. If your local model is the whole point, this matters, because it means you can build a fully air-gapped setup where neither your code nor your API keys ever leave the machine.
Internally, the agent is split into focused packages: pi-ai provides a unified multi-provider LLM API with streaming and tool calling, pi-agent-core is the runtime with state management, pi-coding-agent is the interactive CLI, and pi-tui is the terminal UI. Because every one of these talks an OpenAI-compatible protocol, Pi does not care whether the model behind it is a remote flagship or a local llama-server process.
The Gemma 4 26B A4B Sweet Spot
For offline work, the smart choice is Google's Gemma 4 26B A4B, released under Apache 2.0. It is a Mixture-of-Experts model with 26 billion total parameters but only 4 billion active per token. In practice that yields the quality of a far larger dense model at inference speeds closer to a small one, and it adds the capabilities a coding agent actually needs: native function calling, system prompt support, and thinking modes.
There is one gotcha. Although only 4B parameters activate per token, all 26B must be loaded into memory for fast routing, so VRAM requirements track a dense 26B model. Quantization gives you the dial:
- Q4_K_M: roughly 18 GB download, the best general balance for a coding rig.
- Q6_K: about 24 GB, higher fidelity for long reasoning chains.
- Q8_0: near 28 GB, closest to the original weights.
Context size is the other lever. The model supports up to 256K tokens, but more context means more VRAM on top of the base weights. Small single-file edits fit in 16K with about 1 GB of extra headroom. Standard coding sessions want 64K or roughly 4 GB extra. Multi-file refactors land at 128K and about 8 GB. Full-repo context costs another 16 GB. For practical offline work, 128K is the sweet spot, because coding agents accumulate a lot of context over a session and running out mid-task is the single most annoying failure mode.
Pi's session management exists specifically for this. The /compact command summarizes older messages to free context, /new starts fresh, /tree navigates the whole session history, and /fork branches a new session from any past message. Together they let a local model survive long, expensive sessions without drifting into incoherence.
Hooking it together is genuinely simple. Any of LM Studio, Ollama, or llama-server can expose an OpenAI-compatible endpoint at localhost, and Pi does not care which one you use. A models.json file points a provider at that base URL with an api type of openai-completions, and a /model command switches Pi onto the local endpoint. Slots the model into the existing harness with no retraining and no network egress.
The result is a coding agent that is fully local, reasonably fast on modest consumer hardware, and MIT licensed end to end. For developers who care about latency, privacy, or just not paying per token, the four-tool minimalist with a MoE model is a stronger recipe than most of the heavyweight agents would like to admit.
Comments