Twenty websites

I thought I had built a website with about twenty pages. I had built twenty separate websites, and I only found out when I tried to get them to show up in search results.

Each page described one thing the company could do for a customer: a way of screening the market, a way of reading a utility, a way of finding the right person to call. Almost all of them came out of a real conversation. Someone would ask on a call whether we could do a particular thing, and if the answer was yes, or was about to be yes, that became a page.

They were good pages, and that is why I did not see the problem.

How I got there

The site started on Framer. I found it rigid enough that I stopped wanting to touch it, and a tool you avoid opening sets your publishing rate to zero. For a company that says it can read a market faster than anyone else, having nothing to show was a real cost.

So I moved to Lovable and started describing pages instead of building them. I would explain a use case and get back something that looked considered, with proper typography and structure. I have no website development experience. Two years ago the gap between what I could imagine and what I could put on the internet was most of a career. Now it was an afternoon.

So every time a call surfaced a use case I had not described yet, I described it and got another page. Each page was correct, and each was better than what I would have briefed an agency to make. Nothing failed along the way. There was no error message, no broken build, and nothing in the tool telling me I was doing something unwise.

What I had done

On Lovable, each of those pages was its own project. I knew that, but I had never had to think about it. I was not building a site with twenty pages. I was building twenty sites with one page each, and the only thing connecting them was that I had made them all.

That only costs you something when you try to be found. A search engine does not rank a page on its own. It ranks it in the context of a site: what links to what, which page is the authority on a subject, how a visitor moves from one page to the next. Twenty single-page sites have none of that.

I spent a weekend consolidating them into one website, with a new structure and everything migrated across, and left Framer behind. I did it with Claude, which is the only reason a person with no web development experience could do it at all. It still took the whole weekend.

The part I did not see coming

Consolidating them was not enough. The pages rendered through JavaScript. A person visiting saw everything. A crawler (a search engine, or an AI system fetching the page to answer someone's question) got an empty shell. All that writing about what this company does and for whom, and the machines that decide whether anyone finds it could not see any of it.

The funny thing is that this is my own product's problem. AquaIntel reads the documents nobody reads. The information is already public, and the difficulty is that it sits in formats no machine will touch. I had spent months on that problem for water utilities and had shipped exactly that problem about my own company.

The fix was prerendering, which means generating the finished HTML ahead of time so the content exists before any JavaScript runs. I used a tool called LovableHTML, which has since been renamed Encited. It serves finished HTML to search engines and Markdown to AI agents. I would not have thought to draw that distinction, but it is right: the two kinds of reader want the same content in different shapes, and neither wants the shape a browser wants.

It worked. I checked it again while writing this. Fetch any page with JavaScript switched off and the text is all there, six hundred to fifteen hundred words a page, with headings and descriptions.

And then the part I really did not see coming

While checking, I read the site's robots.txt, the file that tells crawlers what they may and may not fetch. I had never read it, because I had never written it. It is generated.

It bars GPTBot, ClaudeBot, Google-Extended, CCBot and several others outright. Ordinary search is unaffected: Googlebot is not on the list, and the file explicitly permits indexing. What it turns away are the AI systems, which are the same readers I had just spent a weekend making the site visible to.

I did not choose that. I inherited a sensible default, written by people protecting site owners from having their work used for model training. That is a real concern and I have some sympathy for it. But nobody asked me and I never looked, so a position I have never held has been my company's public policy for months.

That default costs me more than most. The second of the two conversations that restarted this company, in December, came from a man who had written up an idea of his own and asked an AI whether anyone was already building it. It pointed him at me. He booked a call to tell me I had built the thing he had been sketching. That channel produced a customer before I had a product, and for some months now a file I never wrote has been closing it.

I have not changed the file yet, and this is why. I hold the objection the default protects against. I would rather what I write not be swallowed as training data, and the file says so: it carries a signal declaring search allowed, training refused, use by reference only. That is close to my position.

Then, a few lines further down, it names the individual crawlers and refuses them outright, and that is the instruction they follow. So the file states the distinction I care about and then drops it for the systems where it would matter. What I want is to be readable and citable without being absorbed. The file can express that and does not, and I do not yet know whether I can fix that in practice or only argue about it in principle. Until I know, it stands as I found it.

I know every week I spend deciding is a week the door stays shut. I would still rather publish this unresolved than wait for a tidy ending.

What this was about

Some of this is inexperience. A person who had built websites before would have noticed on day two. But the failure was not in any single piece of work. The pages were good, the migration was competent and the prerendering fix was correct.

What I believe happened is this: the cost of producing something collapsed, and the cost of owning it did not move at all. Making a page went from a week and an agency to an afternoon and a description. But a page still has to sit inside a structure and still has to be found. Somebody still has to keep an inventory, so they know there are twenty of them and notice when one starts to contradict another six months later. And it still arrives with defaults attached that somebody has to read.

None of that got cheaper. Because the making got so much cheaper, structure, inventory and defaults became a bigger share of the total work, not a smaller one. I was producing faster than I was keeping track, and I do not think I am unusual in that. A lot of people can now produce things they could not produce two years ago, and the rest of the job did not get automated alongside it.

The site is in decent shape now. We change it in pieces rather than rebuilding whole sections, which is what having a structure buys you. If your company's site was built the same way, two quick checks are worth doing: fetch a page with JavaScript switched off and see what is left, and read your robots.txt to see who it turns away.