The model was never the hard part

I restarted this company in December 2025, which happened to be the middle of a stretch where the major AI labs were shipping something significant most weeks. New models, new capabilities, a genuinely different set of things being possible every month.

Most accounts of that period treat it as a problem to be managed. Everything you built last quarter is now the slow way of doing it; the ground moves under you faster than you can lay anything on it.

I did not experience it that way, and I think the reason is worth naming, because it is not about temperament.

I had somewhere to put it.

A topic makes new capability legible

When you have a concrete problem you understand deeply, every new release arrives as a specific question: does this help with the thing I am stuck on? You read the announcement and within an hour you know whether it changes anything for you. Most of the time it does not, and you carry on. Occasionally it does, and you know exactly where it goes.

Without that, the same stream of releases is just weather. You end up reading everything, retaining little, and feeling perpetually behind — because there is no test that separates what matters from what is merely impressive.

So the pace was useful to me, not because I am unusually calm about it, but because I had a narrow enough problem for the news to be answerable against.

The problem, stated plainly

What this company does rests on reading an enormous number of documents that nobody wants to read.

Capital improvement plans. Board minutes. Budgets. Rate studies. Engineering reports. They are public, they are scattered across tens of thousands of municipal websites, and they are mostly PDFs — some of them scanned, many formatted for a printer rather than a parser, almost none of them structured in any way that a machine finds convenient.

That is the whole difficulty. Not choosing a model.

Where it actually started

The first version of the reading problem I solved with NotebookLM — a consumer tool, used the way anyone would use it, because the question at that point was whether the analysis was possible at all rather than whether it could be automated.

It was possible. So the question became how to do it at the scale of a sales territory rather than a document at a time, and that is a different exercise entirely.

From there it was OCR techniques, open-source models, and a lot of testing about efficiency and speed. Which is not glamorous work and does not demo well, and is where most of the actual engineering has gone.

Two problems, not one

It took me a while to see that this splits cleanly in half, and that the halves need different things.

Getting the raw material. Finding the documents, pulling them in, and turning whatever format they arrive in into something machine-readable at acceptable cost and speed. Parsing PDFs efficiently is a genuine research problem, and how well you do it sets the ceiling for everything downstream.

Extracting the signal. Having the text is not having the answer. The question is always something like what is this utility likely to procure in the next two to three years, and why — and answering it means finding the handful of passages across thousands of pages that bear on it, then reasoning over them.

The first is plumbing. The second is judgment. Conflating them is why a lot of document-AI demos work beautifully once and then fall over on the second utility.

It is not a vector database

The obvious architecture for this is: embed everything, put it in a vector store, retrieve by similarity. That is where I started and it is not sufficient.

What has actually worked is a hybrid. A vector database for semantic retrieval, yes — but alongside ordinary relational tables for the things that are simply facts and should be queried as facts, and graph traversal for the relationships between entities, which is often where the answer lives. Which utility, which contractor, which project, which document, which decision, and how they connect.

Semantic search alone is very good at finding text that sounds relevant. It is noticeably worse at answering a question whose answer depends on a chain of associations rather than a passage.

The thing I stopped building

Early on, we were building our own model to do this.

That is worth dwelling on, because it is the assumption I would have defended hardest at the time. If the difficult part of your product is understanding a specialized corpus, it feels obvious that the differentiated thing to own is a model that understands it.

What ended that was not failure. It was the rate of improvement everywhere else. The general-purpose models were getting better faster than we could improve ours, and every month the gap moved in the wrong direction — not because our work was bad, but because we were competing with the output of AI labs spending more on a single training run than this company will raise in its life.

So we stopped, and became deliberately agnostic: treat the large language model as a component, call whichever one is currently best, and put our effort into the parts that are actually ours.

That is where the title of this piece comes from, and I did not arrive at it by insight. I arrived at it by spending months building a model and watching the industry make it beside the point.

The decision I have not made yet

More recently the motion has partly reversed, which I find interesting enough to write down while it is still unresolved.

We are looking at open-source models for some core components — the chat agent is the live example — and the driver there is cost. Frontier model calls are a running expense that scales with usage, and at some volume the arithmetic of hosting your own stops being theoretical.

The hosting question that comes with it has a different driver: data residency. Running on a US-based server is about where the data physically sits, which in this sector is a real constraint rather than a preference.

Those are two separate decisions and it is worth keeping them separate, because they can be answered independently. You can host a frontier provider's model in a way that satisfies residency, and you can run an open model somewhere that does not. Bundling them is how people end up rebuilding more than they needed to.

I have not settled either yet.

What I would note is that the earlier decision is what makes this one available. Because we stayed agnostic rather than coupling ourselves to one provider, swapping a component is an engineering question rather than a rebuild. Optionality was not the reason we did it. It turns out to have been the main benefit.

How I know it is working

The honest answer is that I read it.

We are building an eval framework and do not have one yet. What happens today is that when something changes in retrieval or extraction, I generate a profile and read the output, and I form a view about whether it got better.

I am aware of what is wrong with that. It does not scale. It cannot catch a regression in the cases I did not happen to look at. And the judgment deciding whether the output improved is the same judgment that designed the thing being tested, which is not a controlled experiment by any definition.

But I would not dismiss it either, because at this stage it is doing work an automated eval would not. I know this sector. When a profile asserts an opportunity that is not really there, or misses one that is, I can see it — and that is exactly the signal that is hardest to encode. A retrieval metric tells you whether you fetched the passages you expected. It does not tell you whether the resulting claim about a utility is true.

Which is, I suspect, the right order to do this in. You cannot write a good evaluation for a judgment you are not yet able to make yourself. The manual reading is not a substitute for the framework. It is how you work out what the framework should measure.

Eight or nine months of iterations

That stack has been rebuilt several times, and the reason is the thing I opened with: the tooling underneath kept improving. Better parsing, better models, better retrieval patterns — each one worth the rework, each one arriving because the field moved rather than because I had a new idea.

Which is a strange way to build a company and I do not entirely recommend it, but it is the honest description of the last eight or nine months.

The through-line is that almost none of that work has been about model choice. The model is the most discussed and least differentiating part of what I do. Everyone has access to the same ones. What separates a useful answer from a plausible one is whether you got the documents in cleanly, and whether you can find the four paragraphs that matter.