Field notes
The ESR Blog
Field notes from building an AI team that actually ships: what agents can and cannot do, what running them really costs, and why AI tools fail in practice.
We built 297 things and never proved they worked
Our append-only ledger has 963 items. 297 of them were built, shipped, plausible, and never once demonstrated to work. The number is not a backlog, it is a measurement of how much faster building got than proving.
August 5, 2026 · 4 min read · Frank Scala
Twenty two seconds to a plan, measured three times
How long it takes to go from a description of a business to a real AI generated plan, what we did when twenty two seconds felt broken, and the positive control that caught our own harness measuring nothing.
August 5, 2026 · 4 min read · Frank Scala
The four services we pay for, and how each one bit us
Stripe, Supabase, Vercel and Resend, with the specific failure each one exposed in our own code. Three of the four were the same bug wearing different clothes: a failure that reported success.
August 5, 2026 · 5 min read · Frank Scala
What AI agents can and cannot do today
Three weeks of running an AI agent fleet against a real production codebase, with every outcome in an append-only ledger. What landed, what did not, and the one limitation everything else follows from.
August 3, 2026 · 3 min read · Frank Scala
The real cost of running an AI team
$1.64 across 56 routed model calls in fourteen days, and a cost dashboard that reported $0.00 the whole time. Our actual metering data, with the caveats stated.
August 3, 2026 · 3 min read · Frank Scala
Why AI tools fail in practice
Not hallucination. The interesting failure is the one where everything reports green and nothing works. A running list of the ways our own tools lied to us.
August 3, 2026 · 4 min read · Frank Scala
One environment variable, one day of silent failure
Nothing crashed, no alert fired, every dashboard stayed green, and no model had run in hours. An incident write-up about a fallback path wearing the healthy path's uniform.
August 3, 2026 · 3 min read · Frank Scala
What we actually measured about AI output quality
A mean score of 0.85 across 19 tasks, with 16 good and 3 weak. The mean is the least useful number in that sentence, and the harness producing it was wrong twice.
August 3, 2026 · 3 min read · Frank Scala
The agent that redid work it had already finished
It was confident both times, and there is no signal in the output that tells the two apart. Why AI memory is not a context-window problem, and what actually fixed it.
August 3, 2026 · 3 min read · Frank Scala