Running local agentic AI coding workflows without cloud subscriptions

Running local agentic AI coding workflows without cloud subscriptions

Cloud coding assistants fall over on agentic work in a very specific way: they run into quota walls halfway through a task, not halfway through a chat. A normal chat sends one prompt and waits. An agent sends a planning prompt, reads a file, writes a diff, runs a linter, reads the error, and tries again before you have touched the keyboard. Each step burns tokens, and the total is nothing like the neat little numbers quota calculators like to show.

Google’s Gemini free tier is a decent example. The headline rate looks generous until you count context re-injection. Agentic tools like Cline resend the full conversation and file context on every step. A 100,000-token context window, cycled ten times through one bug fix, eats a million input tokens before a line ships. Free tier weekly caps disappear fast. Paid tiers are not much better; metered pricing means a busy day of agentic coding produces a bill that follows the number of reasoning steps, not the number of features completed.

Running the model locally changes that. No rate resets, no quota emails, no API key revocation in the middle of a refactor.

Why agentic token usage breaks cloud quota assumptions

Standard quota estimates assume conversational use: a prompt, a response, maybe some context. Agentic tools do something else. Cline, for example, runs a plan-then-act loop. It sends the task, the file tree, relevant source files, and prior conversation state on every step. A task involving five files and twelve agent steps resends those five files twelve times. With an average file size of 200 lines of Python, that is a lot of input tokens that never show up in a rough estimate.

Codestral and GPT-4o pricing pages show per-million-token rates that look fine at chat scale. At agentic scale, with context windows filling and refilling on each loop, the cost looks more like a small batch job than a message.

The move from per-seat pricing to credit-based pricing reflects that. Credits disappear faster under agentic use, and you usually notice after the dashboard says they are gone.

Choosing a local model for coding tasks

Three models are worth looking at for local agentic AI coding: Qwen2.5-Coder, Codestral, and DeepSeek Coder. The right one comes down to available VRAM and the context window you need.

Qwen2.5-Coder 7B fits comfortably on 8 GB of VRAM at Q4 quantisation. The context window is 128K tokens, which covers most single-file agentic tasks without truncation. It works well on Python, TypeScript, and shell scripting. Use the 32B variant if you have a 24 GB card and want stronger multi-file reasoning.

Codestral 22B needs at least 16 GB of VRAM at Q4. Mistral’s fill-in-the-middle training makes it strong for inline completions, but it also handles instruction following well enough for Cline’s planning steps. The context window is 32K tokens, which is tight for large codebases but fine for focused tasks.

DeepSeek Coder V2 Lite runs on 8–10 GB of VRAM and handles multi-language tasks cleanly. The 16B instruct variant works with Cline’s system prompt format without modification.

Pull the model before building the stack:

ollama pull qwen2.5-coder:7b

Setting up the Docker Compose stack

The stack has two named services: ollama for model serving and open-webui for a browser interface. Cline talks to the Ollama API directly. Open WebUI is optional but useful for testing prompts before running the agent.

version: "3.8"

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    ports:
      - "3000:8080"
    volumes:
      - open-webui_data:/app/backend/data
    restart: unless-stopped

volumes:
  ollama_data:
  open-webui_data:

If you are running an AMD GPU, swap runtime: nvidia for devices with the ROCm device path. For CPU-only, remove the runtime and environment keys entirely. It will be slower, but it still works.

Bring the stack up:

docker compose up -d

Confirm Ollama is responding before pointing any agent at it:

curl http://localhost:11434/api/tags

A JSON list of pulled models means the service is ready.

Pointing Cline at the local Ollama endpoint

Open VS Code with the Cline extension installed. Go to Cline settings and set the provider to Ollama. Set the base URL to http://localhost:11434. Select the model name that matches what you pulled, for example qwen2.5-coder:7b.

Cline sends requests to /api/chat on that base URL. No API key field is needed; leave it blank or Cline will ignore it when the Ollama provider is selected.

For Aider, set the model and base URL from the command line:

aider --model ollama/qwen2.5-coder:7b 
      --openai-api-base http://localhost:11434/v1 
      --openai-api-key ollama

The --openai-api-key ollama flag is a placeholder. Ollama’s OpenAI-compatible endpoint at /v1 needs a non-empty key string, but the value does not matter.

Restricting filesystem access to the project directory

Running an agent with access to your full home directory is unnecessary. Use a bind mount scoped to the project when running the agent container, or when launching Aider from a container rather than directly on the host.

For a containerised Aider setup, add a service to the Compose file:

aider:
  image: paulgauthier/aider:latest
  container_name: aider
  volumes:
    - /home/youruser/projects/myproject:/app:rw
  working_dir: /app
  environment:
    - OPENAI_API_BASE=http://ollama:11434/v1
    - OPENAI_API_KEY=ollama
  network_mode: service:ollama
  stdin_open: true
  tty: true

The bind mount /home/youruser/projects/myproject:/app:rw gives the container read-write access to that directory only. Nothing outside /app is reachable from inside the container.

If you run Cline on the host rather than in a container, restrict its working directory in VS Code’s workspace settings to the project root. Cline respects the workspace boundary and will not traverse above it without explicit instruction.

Confirming the stack runs with no outbound connections

Pull all images and models before going offline. Once they are pulled, the stack does not need internet access.

Block outbound connections at the Compose level using an isolated network with internal: true:

networks:
  ai_local:
    driver: bridge
    internal: true

Attach both services to this network. Remove the ports mapping if you want to lock the stack entirely to local inter-container communication, or keep it if you need host access from VS Code or Aider running on the host.

Verify no outbound traffic is leaving once the stack is running:

sudo ss -tunp | grep -E 'ollama|webui'

You should see only loopback (127.0.0.1) and local network addresses. No connections to external IP ranges. If you want a harder check, run the stack with --network none on the Ollama container and confirm model inference still works. If it does, the model is fully loaded into memory and serving from local storage with no network dependency.

The stack costs nothing per token, resets nothing weekly, and keeps running as long as the machine does.