Local-First Agents: Running Models, Voice and Research Without a Cloud Bill
Local inference has been technically impressive and economically unconvincing for most of its life. The calculus has started to change for a mundane reason: people are paying for subscriptions they only use a fraction of, and they now know exactly what fraction. That accounting honesty produced a wave of practical tooling.
Eleven projects from the last stretch of Hacker News make up the current shape of that wave.
Inference in the browser
WebLLM remains the reference point for this category. It is a high-performance in-browser LLM inference engine that runs models directly in a browser tab on WebGPU, with no server round trip at all. The privacy property is total, since no input ever leaves the device, and the operational property is equally appealing: no API key, no rate limit, no invoice.
The constraints are the ones you would expect. Model weights have to download on first load, and the ceiling is set by device memory. For a privacy-shaped feature on a website, though, it has no real competitor.
Fine-tuning on a laptop
The surprising project in this batch is Soup, which fine-tunes LLMs from one YAML file and uses layer streaming so an 8B model can be trained on a 4 GB laptop GPU. It covers DPO as well as supervised fine-tuning and exports GGUF so the result is usable by local runtimes.
The hardware floor is the story. Fine-tuning has been gated behind rented accelerators, and anything that genuinely lowers it changes who can participate, even if laptop-scale training is slow. The single-file configuration is the other reason it spread: most people who want to fine-tune are blocked by configuration sprawl before they are blocked by compute.
Transcription that stays home
Speech is the most convincingly local category, because hosted transcription is expensive and audio is the data people least want to upload.
Yap does voice dictation for macOS using Apple's Speech framework, entirely on device: no cloud, no API key, no account. It is the clearest example in the batch of the value proposition being simply the absence of a subscription.
Qwen Scribe takes the same need to the open-weight side, offering private local transcription and system-wide dictation for Apple Silicon with Qwen3-ASR running through MLX. The choice between the two is really a choice between Apple's speech model and an open-weight one, which is a speech-accuracy question rather than a software one.
Egma covers the other end of a voice pipeline. It is an open-source platform for simulation testing, monitoring and self-improving voice agents. Voice agents are conspicuously underserved by test tooling compared with text agents, and simulation before deployment is the only practical way to find the failures this category is full of.
Research agents with a budget
Mole is the formally interesting one here. It is a deep-research agent with an enforced budget, verified quotes and a privacy boundary for local data. Each of those three properties addresses a known failure: unbounded spend, uncited claims, and local documents being shipped to a provider by accident. Enforced budget in particular is unusual enough that its absence in other tools is worth noticing.
Jev Ultrafast attacks research cost from the other end. It is the fastest and cheapest web agent from the Browser Use team, with a dynamic, indexed action space that keeps steps per task down. For high-volume web tasks, step count is what determines cost, so the two are the same problem. As with any vendor benchmark, the comparative claims are the vendor's own.
Agentability is not a tool you install, but it belongs in this group for what it publishes. Every day an AI producer turns what the world is searching into ten real errands, a real agent attempts them with plain web requests and no logins, no JavaScript and no human help, and every transcript is published verbatim, wins and failures alike. Alongside it, the 1,219 most-visited sites are scored on how usable they actually are for an agent. It is the closest thing to public documentation of where the agentic web currently fails.
Running local as a financial decision
Sunk Cost is a calculator, and a useful one. It takes a machine, its power draw and your usage assumptions and returns the number of months until running models locally pays for itself against an API bill. All of the assumptions are visible and adjustable. It exists because the local-first argument is usually conducted in anecdotes, and it turns it into arithmetic.
Stale supplies the other half of the argument. It lists release age and training cutoff for 23 models across 8 labs and counts upward from each, so you can see how stale a model's knowledge actually is rather than reading a press release about it. It also publishes the data as JSON. The finding that only some labs even publish a cutoff is itself informative.
Nightcrawler is the odd one out and the most fun. It is a local AI-powered red teamer that runs on a phone. That is a genuine use of the local property: a portable testing rig that carries its own model and needs no laptop or cloud account. It is also limited in exactly the ways phone hardware implies.
Where this leaves the decision
The honest reading of this batch is that local inference stopped being a purity argument and became a price argument with a purity side effect. Sunk Cost exists because the first question anyone asks is a number, and Stale exists because the second is about whether the model knows what happened last month.
If you are choosing a starting point, the sequence that makes sense is to run Sunk Cost with your actual usage, then decide whether the result justifies Soup on hardware you already own or a hosted plan you already pay for. Only then does the choice of WebLLM, Yap or Qwen Scribe matter, because by then you know whether you are optimising for privacy or for cost.
