Using Local Coding Agents
by Sebastian Raschka (Ahead of AI)
Run Qwen3.6 35B-A3B and other open-weight models inside Qwen Code, Codex CLI or Claude Code through Ollama, with speed numbers, a small agent benchmark and a security audit.
Overview
Raschka's June 2026 Ahead of AI tutorial splits a coding agent into the model, which reasons, and the harness, which reads files, edits code and runs commands, and then sets up each layer locally. His main pairing is Qwen3.6 35B-A3B, about 22 GB and needing roughly 30 to 40 GB of RAM, served by Ollama (the MLX build on a Mac) and driven by Qwen Code. Cohere's North Mini Code and Gemma 4 are the comparison models. A speed script, ollama_speed_memory_bench.py, measures prefill, generation speed and memory from 1k to 50k words of context: at 50k, the two larger models use up to about 30 GB and generate about 40 tokens per second on a Mac Mini M4 and 30 on a DGX Spark. A five-task reasoning and tool-use test and a five-task agent problem pack follow, where Qwen3.6 and North Mini Code each solve 4 of 5 in Qwen Code and Gemma 4 E2B performs poorly. The tutorial includes an audit of Qwen Code before installing it, which found that telemetry can still reach Alibaba endpoints with a local model, and a settings file that turns off telemetry, auto-update, hooks and MCP servers. It then configures Codex CLI with an Ollama profile, runs Claude Code through ollama launch claude (one run used about 578k input tokens over 25 turns), and serves a model from a DGX Spark to a Mac over an SSH tunnel. Raschka still uses Codex and Claude Code day to day and says GPT 5.5 and Opus 4.8 remain stronger.
At a Glance
- Topic
- Agentic
- Level
- Intermediate
- Format
- Tutorial
- Cost
- Free
- Duration
- ~25 min read (about 5,000 words), plus setup time for Ollama and a harness
- Provider
- Sebastian Raschka (Ahead of AI)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓How to serve Qwen3.6 35B-A3B locally with Ollama, including the MLX build on Apple Silicon
- ✓How to connect Qwen Code to a local OpenAI-compatible endpoint and switch models with /model
- ✓How to configure Codex CLI with an Ollama profile in ollama.config.toml for a local model
- ✓How to run Claude Code against a local model and why its token usage runs highest
- ✓How to benchmark prefill speed, generation speed and memory across 1k to 50k context lengths
- ✓How to build a small personal task set to compare models on reasoning and tool use
- ✓How to audit an agent harness for telemetry, default permissions and prompt-injection surfaces
- ✓How to serve a model from a DGX Spark to a Mac client through an SSH tunnel
Highlights
- •Reports measured numbers, such as about 40 tokens per second at 50k context on a Mac Mini M4, rather than vendor claims
- •Compares the same model across Qwen Code, Codex CLI and Claude Code, showing that token use depends more on the harness than the model
- •Includes a harness security audit with the concrete telemetry finding and a locked-down settings file
- •Written by an author who uses Codex and Claude Code daily and states where local models still fall short
Who It’s For
Best For
- ✓Developers who want private or offline coding agents for sensitive code or documents
- ✓Teams weighing local open-weight models against Claude Code or Codex subscription costs
- ✓Engineers who want a repeatable way to benchmark local models before trusting them in an agent
Prerequisites
- •Comfort with the terminal, npm or curl installs, and editing JSON or TOML config files
- •A machine with roughly 32 GB or more of memory, such as an Apple Silicon Mac or an NVIDIA DGX Spark, for the 35B models
- •Basic experience with at least one coding agent such as Claude Code or Codex CLI
FAQ
What is Using Local Coding Agents?
Using Local Coding Agents is a June 27, 2026 hands-on tutorial by Sebastian Raschka on running open-weight models in coding-agent harnesses as an alternative to paid Claude Code and Codex subscriptions. It is for developers with a 32 GB or larger Mac or a DGX Spark who want private, fixed-cost coding agents and want to measure how well they work before relying on them.
Is Using Local Coding Agents free?
Using Local Coding Agents is free to access.
What level is Using Local Coding Agents for?
Using Local Coding Agents is aimed at a intermediate audience. Recommended background: Comfort with the terminal, npm or curl installs, and editing JSON or TOML config files, A machine with roughly 32 GB or more of memory, such as an Apple Silicon Mac or an NVIDIA DGX Spark, for the 35B models, Basic experience with at least one coding agent such as Claude Code or Codex CLI.
How long does Using Local Coding Agents take?
Expect roughly ~25 min read (about 5,000 words), plus setup time for Ollama and a harness. Most learners work through it at their own pace.
What will I learn from Using Local Coding Agents?
You'll learn: How to serve Qwen3.6 35B-A3B locally with Ollama, including the MLX build on Apple Silicon; How to connect Qwen Code to a local OpenAI-compatible endpoint and switch models with /model; How to configure Codex CLI with an Ollama profile in ollama.config.toml for a local model; How to run Claude Code against a local model and why its token usage runs highest; How to benchmark prefill speed, generation speed and memory across 1k to 50k context lengths; How to build a small personal task set to compare models on reasoning and tool use; How to audit an agent harness for telemetry, default permissions and prompt-injection surfaces; How to serve a model from a DGX Spark to a Mac client through an SSH tunnel.
Topics
Sources
This page was written from 3 sources, 1 on domains other than magazine.sebastianraschka.com.