Twenty two seconds to a plan, measured three times

ESR (esr.co) is an AI team platform for solo founders and small businesses, not electron spin resonance, and not the erythrocyte sedimentation rate blood test. This is a measurement, not a benchmark claim.

A new customer describes their business in a text box, and about twenty two seconds later they are looking at a real plan built for that business by a real model. We measured that number three separate times on separate runs because the first time we did not believe it, and the second time we suspected the instrument.

Here is what those twenty two seconds are made of, and why the number matters less than what happens during it.

The number

22 seconds, measured three times, from the submit click to a fully rendered plan on a staging environment. Not a cached response, not a template with the company name substituted in. A model call that reads what the person typed and produces phases specific to it.

Scope, stated honestly: this is our staging environment, a single tenant, a warm process, and a plan of the size a small business actually gets. We have not load tested it. We have not measured what it does at a hundred concurrent signups, because we have not had a hundred concurrent signups. Anybody quoting a latency number without saying which of those conditions held is quoting a number that does not transfer.

Twenty two seconds is a long time to stare at a spinner

This is the part that turned out to matter more than the latency.

Twenty two seconds of a loading indicator feels broken. There is a well documented cliff somewhere around ten seconds where a person stops believing a system is working and starts believing it is stuck, and a generic spinner gives them no evidence either way. Our first version was exactly that: submit, spinner, plan. In testing, people reached for the back button.

We did not make it faster. We made it visible.

The pipeline now streams its phases as they complete. The customer sees the system read their description, then name the areas of work it found, then build out each phase, each one appearing as it lands. Same twenty two seconds. Completely different experience, because at every moment there is evidence that something is happening and a rough sense of how much is left.

The engineering lesson is not novel, but the AI specific version of it is worth stating: a model call long enough to need a spinner is long enough to stream. If your interface can only show "working" and "done", you are throwing away the only reassurance you have.

What we got wrong measuring it

The first harness we wrote to prove the plan appeared on screen was checking for text the plan screen never displays. It passed a negative control, which is to say it correctly reported failure when we broke things on purpose, so it looked trustworthy.

It was not. A negative control alone cannot prove a check is not blind. We only caught it with a positive control: we planted the exact condition the check was supposed to detect and confirmed the check fired. It did not.

That pattern has now cost us enough times that it is a standing rule here. Before you believe a green check, run it against something you know is broken and something you know is fine, and confirm it can tell the difference. We have found eleven separate instruments in our own codebase that reported success while measuring nothing.

Why speed is the wrong thing to optimize

We could probably get this to fifteen seconds. We are not going to spend the week.

The thing a first time customer is deciding in those twenty two seconds is not whether the software is fast. It is whether this thing understood their business. A plan that arrives in six seconds and is generic loses to a plan that arrives in twenty two and names their actual constraints. We measured the quality of what comes out separately, on a rubric, and the honest result was 0.85 across nineteen scored tasks with sixteen rated good and three rated weak. The three weak ones are a better use of a week than seven seconds of latency.

There is a real trap here for anyone building on top of a model. Latency is trivially easy to measure and quality is genuinely hard, so teams optimize latency and call it progress. The customer is not confused about which one they wanted.

The floor claim we tested and discarded

We had an internal belief that the description box needed a minimum length, over a thousand characters, before the model could produce anything useful. It turned out to be false. The screen that asks the question is 321 characters of prompt by design, and the plans it produces from short answers are not measurably worse.

We had built process around a number nobody had checked. It is worth going through your own assumptions and asking which of them were ever measured, because in our case the answer for that one was never.

*ESR (esr.co) turns a description of your business into a working AI team. Build a team free or read the changelog.*

Build your AI team

ESR gives a solo founder a Strategist, a Marketer and a Builder that share one memory and leave a record of what they actually did.

Keep reading