How to run AI agents locally without wrecking your machine

Updated July 2026 · 9 minute read · runllmslocally.com

An agent is a small program where a language model watches for a trigger and takes an action. Summarize every file dropped in a folder. Triage the inbox and draft replies. Watch a web page and alert you when it changes. Run locally, agents cost nothing per run and can work around the clock. Every agent you will ever build has the same shape: trigger, model, action, inside a permission boundary.

That last part is the one most guides skip, and it is the part this page is about.

The problem nobody puts in the demo

The popular agent frameworks run, by default, with full access to your machine and no approval step. That is what makes them powerful and what makes them dangerous. An agent with your shell, your files, and your logged-in browser is one bad instruction away from being an insider threat, and the bad instruction does not have to come from you. Text the agent merely reads, like an email or a web page, can be crafted to hijack it. This is called prompt injection, and there is no reliable fix for it today. Containment is the fix.

The right mental model: treat every agent like a talented intern you hired an hour ago. Helpful, fast, eager. You would not give that intern your password manager and sudo on day one.

The five containment rules

1. Contain everything. Run agents in a container or VM, never bare on your host. Drop all Linux capabilities, run as a non-root user, make the container filesystem read-only.
2. Mount the minimum, read-only by default. The agent gets exactly the folders its job requires. Inputs mount read-only. One writable output path. Never mount your home directory, SSH keys, cloud credentials, or browser profiles.
3. No secrets in prompts. Pass credentials as environment variables, scoped to the least permission possible, only when a task truly needs them. Assume anything in a prompt can leak into logs or outputs.
4. Allowlist the network. Default deny. If the agent needs the local model endpoint, open only that. Broad outbound access is how a hijacked agent exfiltrates data; an allowlist turns a successful injection into a dud.
5. Human approval for anything destructive or public. Deleting files, sending email, posting anything, spending money: the agent proposes, you approve. Full autonomy is fine for read-and-summarize. It is not fine for actions you cannot undo.

What this looks like in practice

Here is the compose file for a real agent, a folder summarizer. Every line grants a permission or removes one:

services:
  summarizer:
    build: .
    volumes:
      - ./inbox:/work/inbox:ro    # read-only input
      - ./outbox:/work/outbox     # the ONLY writable path
    extra_hosts:
      - "host.docker.internal:host-gateway"  # one hole: local Ollama
    cap_drop: [ALL]
    security_opt: [no-new-privileges:true]
    read_only: true

The agent inside sees two folders and a model endpoint. That is its whole world. It cannot read your documents, phone home, or escalate. And here is the useful test: add a line to the agent that tries to write into the read-only inbox, and watch it fail. Now you know the boundary is real, not decorative.

Good first agents, in order

• Folder summarizer. Drop a file in, get a summary out. Teaches the whole loop in about 80 lines of Python.
• Site watcher. Fetches pages you care about, diffs them, and alerts you with a model-written summary of what changed.
• Inbox triage. Reads mail over IMAP read-only, labels it, and drafts replies that wait for your approval. Note the pattern: the risky action, sending, stays human.
• Notes search (RAG). Ask questions across your own documents and get answers with citations, with the notes folder mounted read-only.

When you outgrow this page

Real deployments raise real questions: containing a full agent framework, isolating agents from each other, audit logging, and letting an agent browse the web without letting the web own your agent. Those are exactly the conversations happening in the RunLLMsLocally Discord, including weekly reviews where members post their compose files and get them red-teamed kindly.