Stories
AI·May 4, 2026·8 min read

Building a Local-First AI Agent

Notes from building PATROAM — why local-first changes every architectural decision, and how a knowledge graph plus vector memory work together.

AI AgentsLLMPythonRAG

Most AI assistants live in someone else's cloud. PATROAM doesn't. Building a local-first agent forces a different set of tradeoffs — and they turn out to be the interesting ones.

Why local-first

When the model, the memory, and the data all live on your machine, three things change at once: latency, privacy, and cost. You stop thinking about per-token billing and start thinking about RAM and disk.

Memory: graph + vectors

A knowledge graph gives you structure — entities and relationships you can traverse. A vector store gives you recall — fuzzy retrieval over everything else. Used together, the graph answers "how are these connected" and the vectors answer "what's relevant."

results = retriever.search(query, k=8)
context = graph.expand(results, hops=1)

What I'd do differently

Start smaller. The first version tried to remember everything; the version that works remembers selectively.

ShareXLinkedIn