Most coverage of local AI in 2026 still talks about hardware nobody owns. A $1,500 GPU box. A water-cooled workstation. A render farm. That framing is wrong. The interesting story is what is shipping on hardware most people already have in a drawer. A working example on a Raspberry Pi 5 is the headline of the week. It is also a useful excuse to take stock of where the rest of the local stack actually stands.
The honest version is that local AI right now is half-finished. On a good day, the tool does its job and you forget you are running it. On a bad day, you spend an evening chasing a Python version conflict for a one-line fix. The middle ground, where things work but barely, is where most of the audience lives. That is where this article lives too.
Here is the split I keep returning to. Cloud AI is the right answer when your question is generic and your data is fine to leak. Local AI is the right answer when your question is specific and your data is not. Most of the work I do these days falls into the second camp. The hardware and the open models are finally catching up to that.
A $60 board and a working agent
Abhishek at It’s FOSS published a write-up that genuinely did not seem possible a year ago. He ran the Hermes AI backend on a Raspberry Pi 5, then drove it from his desktop over the network using Hermes Desktop. The Pi does the work. The desktop is just the UI. That is not a benchmark screenshot. It is a working setup on a board that costs less than a decent keyboard.
Why this matters: most people still assume local AI means a gaming PC with a discrete GPU. The Pi story is the counter-example. You will not run a frontier model on it. You will run a focused model that handles a narrow set of tasks. The narrow set is wider than you think. Drafting replies, summarizing notes, scripting the boring repeatable work, all of those fit. Anything that needs raw reasoning hops to a bigger model running elsewhere. The Pi is a router with privacy guarantees, not a brain.
Setup cost and latency are the two practical questions. The It’s FOSS write-up describes a working setup on a Pi 5, with smaller focused models handling the kinds of tasks that show up in daily work. The narrow set is wider than you would expect. If your queue is more about summarizing notes and less about drafting legal briefs, the Pi is fast enough for the work that matters.
The lesson, and the one I keep returning to, is that the local AI stack is splitting the way Linux distros split twenty years ago. One tool will not rule them all. A dozen will, each tuned for a different job, each running on different hardware, each sharing MCP (Model Context Protocol, the standard that lets agents talk to local tools and data sources) as the common wire format. The Pi story is one example. By the end of the year there will be ten more.
Voice stops being a parlor trick
I called voice as the next interface layer for AI a couple of years back. Voice was already the natural input for anyone whose hands were full. The model quality crossed the everyday-use threshold in 2025. What changed recently is that the open source world has caught up to the proprietary vendors on the model side.
Two data points. Ubuntu 26.10 is reportedly baking voice support into the desktop. Vocalinux, an indie project, is going after the same target. Neither is shipping a slick product. Both are trying to make “talk to your computer” a real desktop feature, locally, with no cloud round-trip. In the meantime, Hermes Desktop has a working voice mode that ships today.
What voice actually unlocks is the boring stuff. Catch up on email while you make coffee. Ask your agent for a status update when your hands are covered in bike grease. Stop a long-running build with a sentence instead of breaking flow. None of those are demos. They all work today in early form, and they get noticeably better every release. The hard limit is compute. Voice models want more RAM and more cycles than text models. On a Pi, you are typing by hand. On a laptop with 32GB of RAM, local voice is fine. On anything less, you either spend on hardware or send the audio to a cloud. If privacy is the reason you are doing this locally, the cloud option defeats the point.
Two projects that deserve a closer look
I test a lot of AI tooling each week. Most of it is forgettable. Two from this round are worth your time, with the usual caveat that they are early stage and you should expect bugs.
OpenKnowledge is a local-first wiki for both humans and agents. Notes live in a git repo, the CLI is for humans, and a built-in MCP server lets agents read and write the same notes. The integration is the headline feature. Instead of pasting context into a chat window, your agent pulls what it needs from your wiki. If you already keep a personal knowledge base, or want a shared second brain your coding agents can use across sessions, OpenKnowledge fills a real gap. The git backend means every edit is a commit you can review, branch, or revert.
Cleat takes a different angle and addresses a problem I have complained about for years. It runs Claude Code inside an isolated Docker container with a single command. The agent gets its own filesystem and process tree. Your SSH keys, your other projects, and the rest of your machine stay untouched unless you explicitly allow access. The maintainer sums it up as “Give the agent a cage, not your keys.” That phrasing is exactly right. The trade-off is that the project is young. Its GitHub has been quiet for about three weeks, which is a yellow flag, not a red one. Watch the commit log before you commit a real workflow to it.
What you actually get from each, in concrete terms:
- OpenKnowledge: markdown files in version control, a command line tool for humans, and an MCP endpoint for agents. Every change is a commit, so your notes double as an audit trail.
- Cleat: one Docker command wraps Claude Code in an isolated shell. The container has its own filesystem and network namespace. Your host stays clean.
- OpenKnowledge weakness: no live collaboration, no rich editor, and a small MCP surface. You will write the missing pieces yourself.
- Cleat weakness: quiet repo for about three weeks, the install needs Docker 24 or newer, and there is no native Windows path yet.
Both are rough around the edges. Both solve problems I have personally hit. Both will look different in six months. That is fine. The point is that the local AI scene is shipping things that work today, not just demos.
The open model buffet
Moonshot AI released Kimi K3 on July 16. It is a 2.8 trillion parameter model, which puts it in the same league as the frontier labs. The interesting part is the release cadence, not the parameter count. The open model space used to drop a major release every quarter. Now it is monthly. The bar moves every few weeks.
What this means in practice is that one-model thinking is over. You can keep a small local model for anything that touches private data. You can hit a bigger model through an API when you need raw capability. You can route between them based on the task at hand. If your workflow assumes a single model, you are missing the point.
For most people, the realistic split is local-first for personal data and cloud for the frontier. That is not a compromise. It is what good engineering looks like when the tools allow it. The annoying part is keeping track of which model wins which task. The fun part is that the answer changes every few weeks.
Trade-offs
Local AI is not free. A usable setup costs somewhere between $60 (a Pi 5 plus a cheap SSD) and $1,500 (a real GPU box). The Pi tier is fine for routing and small models. The GPU tier is what unlocks real voice or any model over about 13B parameters.
Software setup is where most of the pain lives. Most local AI stacks assume Python 3.10 or newer, a working CUDA or Metal toolchain, and about an hour of patience for the first install. macOS is finally smooth on Metal. Linux still wants swearing for CUDA. On Windows, you are on your own unless you run WSL2 (Windows Subsystem for Linux, a Linux compatibility layer that runs inside Windows).
OpenKnowledge and Cleat are both early. Expect bugs. Expect missing features. Expect the occasional quiet week from the maintainer. If you build a production workflow on either today, you are betting on the project surviving its first rough year.
If you care about privacy and your tasks are small and repetitive, the Pi setup is a clear win. If you need raw capability and you are willing to spend on hardware, a real GPU box is the move. If you are starting out, begin with OpenKnowledge and a hosted model, get the workflow solid, and move pieces local as you learn what you need.
Bottom line
The story for the rest of the year is going to keep repeating: cheaper hardware, better open models, more tools, more fragmentation. The cloud is not going anywhere. The local stack is not going to replace it. What you get is a real choice for the first time. That choice is worth more than any single benchmark.
If I had to send one tool home with you this week, it would be OpenKnowledge. A git-backed wiki your agents can read is the kind of unglamorous infrastructure that pays off for years. Everything flows from having a single place where both you and your tools can find what they need.