22 August 2026
Before AI coding assistants became normal, developers spent a lot of time hunting around.
You would search for a class, inspect a few files, run tests, read logs, check Git history and look through documentation until you found the bit that mattered.
AI tools now do much of that for us. The problem is that they often use the most expensive model to do all of it.
The model reads the files, logs and search results itself, even when most of that information is only needed to find one useful answer.
I started wondering whether some of that work could happen locally, before the paid model ever sees it.
My first attempt was a local MCP server.
MCP is simply a way of giving an AI assistant access to tools. Those tools can search a codebase, run commands, inspect Git changes and read documentation.
My server does that investigative work locally. It finds the useful information, removes the noise and sends the smaller result to the main model.
If a test produces ten thousand lines but only twenty explain the failure, it returns the twenty.
If a search finds forty possible files but only five matter, it returns the five.
The stronger model still makes the important decisions. It works out what caused the bug, whether the fix is safe and how the change fits into the wider system.
The local tools just prepare the information first.
AI coding sessions become cluttered quickly.
After an hour, the model may have read dozens of files, long logs and several documentation pages. Much of that information is no longer useful, but it still takes up space in the conversation.
Eventually the assistant becomes less focused, forgets earlier decisions or asks to compact the context.
A local tool avoids that problem earlier by keeping unnecessary information out of the conversation.
Most developers do not think much about token usage. We pay for a subscription and let the assistant read whatever it needs.
But every token still costs money to process.
AI companies are currently competing hard for users, which means access is probably cheaper and more generous than it will be forever.
The dot-com comparison is not that AI will disappear. The internet did not disappear when that bubble burst either. What changed was that weak business models failed and the market became more interested in who was actually paying for everything.
I think AI will go through something similar. The technology will remain useful, but cheap access to huge amounts of compute may not.
This is why developers should start getting comfortable with local models now.
They are not as capable as the strongest cloud models, but they do not need to be.
A small local model can search logs, rank results, summarise changes and remove duplicated information. Those are useful jobs, but they are not difficult enough to justify using the most expensive model every time.
Local tools can search, filter and organise. The stronger model can reason, decide and write the code. That is a better division of labour.
Moving some work locally also reduces dependence on one provider. If prices rise, limits tighten or products change, a workflow built around local tools and interchangeable models is easier to adapt.
There is a privacy benefit too. Source code, logs and internal documentation can be processed locally, with only the relevant parts sent to the cloud.
The aim is not to replace frontier models, It is to stop using them for work that cheaper local tools can already do well.
That makes AI coding sessions simpler and more focussed today, and when token usage becomes a more visible cost, it may make them much cheaper too.
← All posts