Fifth Postulate

Back
11 min
ai /
notes

Nobody Writes Down How They Failed

A note on why the most valuable thing an engineer produces is the part nobody keeps — and what that has to do with the last generation of human engineers.

Production note. This started as two separate conversations — one with Claude about where my long-term technical direction should point, one with Gemini about a narrower question of information theory in AI training. I didn’t want to leave either sitting in a chat window. An LLM summarized and drafted it; I edited it by hand.

Last spring, at two in the morning, I was staring at a Kalman filter that refused to converge. The drone I was building was supposed to track a falling tennis ball, predict where it would land, and get there first. For three days the predicted intercept point kept drifting somewhere off-screen. I eventually found it — a sign error in how I’d rotated the camera frame into world coordinates, buried under two other bugs I’d already half-fixed on the way there. The moment I found it, I felt something close to relief, wrote the fix, committed it, and moved on to the next problem.

Here is what nobody will ever know about those three days: not the two other bugs I fixed along the way, not the four hypotheses I tested and discarded, not the specific reasoning that made me suspect the coordinate transform before I suspected the filter tuning. All of that — the actual process of debugging — evaporated the moment the commit went through. What survives, in my GitHub history, is a diff. A correct answer. The three days of being wrong in increasingly specific and informative ways left no trace at all.

I want to argue that this small, ordinary loss — repeated by every engineer, every night, for as long as there have been engineers — is the actual thing being wasted. Not “time spent on STEM,” which is a lazy way to say it. Something more specific: the record of how a mind moves from broken to working. And I think the fact that this record is finally starting to get built, deliberately, at industrial scale, is the most interesting thing happening in AI right now.

The information-theory instinct, and where it breaks#

My first pass at this idea was cruder, and worth admitting to. I reasoned: a “problem” has one correct answer and an enormous number of wrong ones. Telling you the correct answer hands you everything — it collapses the whole space in one move. Telling you one wrong answer only rules out that single point. By that logic, correct answers should be the dense, valuable signal, and errors should be almost worthless — a rounding error in the information you actually wanted.

This is true for a fixed, static problem. It is backwards for training a system that has to operate in the world.

The flaw is treating “wrong” as a single point when it’s really a path. A correct answer tells you where the destination is. An error — specifically, the trajectory from making the mistake to recognizing it to correcting it — tells you where the walls are, which direction leads to a dead end, and what backtracking from that dead end actually looks like. A model trained only on finished, correct solutions has memorized destinations without ever having walked the terrain. The moment it takes one wrong step in a context slightly different from its training data, it has no idea what a “wrong step” even looks like, let alone how to recover from one. It doesn’t get stuck the way a person gets stuck — thoughtfully, one hypothesis at a time. It just breaks.

The childhood I already half-described#

I’ve written before, on this site, about why a three-year-old picks up an iPad faster than her grandmother does — and about the machine-learning research that borrows the same shape: curiosity-driven exploration, safe environments where a wrong tap costs nothing, task-free pretraining that looks, from the outside, exactly like play. At the time I framed that as a story about generalization — how a system builds a flexible internal model instead of memorizing one narrow skill.

I think I was one layer away from the real point. Childhood generalizes precisely because it is the one period in a life where making an error is cheap. A toddler can fail at stacking blocks fifty times in an afternoon and nothing is lost — no client is angry, no deadline slips, no reputation is dented. That’s not incidental to how children learn; it’s the whole mechanism. Cheap failure is what makes a system willing to generate the failure data in the first place.

Adult engineers don’t get that deal. Every failure I have is attached to a cost — three sleepless nights, a missed milestone, the quiet embarrassment of showing a broken prototype to someone I was trying to impress. So I do exactly what the incentive structure asks of me: I hide the failure as fast as I can, replace it with a clean commit, and never speak of it again. Multiply that by every engineer who has ever lived, and you get an entire civilization that has been running the highest-fidelity trial-and-error experiment imaginable, for centuries, and throwing away almost all of the data on purpose — because keeping it made no one look good.

The industry finally agrees, and it happened faster than I expected#

I asked around — not people, search engines — about whether anyone was actually building the thing I was describing: a system for capturing the error-to-resolution process rather than the finished answer. The honest answer is yes, and it’s more mature than I assumed.

Microsoft Research’s own writeup of debug-gym ↗, an environment they built specifically to teach coding agents to debug like people do — setting breakpoints, inspecting variables, running experiments — states the problem in almost exactly these terms: today’s AI coding tools are weak at real debugging in large part because sequential, decision-by-decision debugging traces are simply scarce in the data these models were trained on. Not because debugging is conceptually hard for a language model, but because almost nobody wrote it down.

On the robotics side, a team including researchers at NVIDIA built something called FailGen ↗, which does exactly what its name suggests: it takes successful robot demonstrations and deliberately perturbs them — a bad grasp, a slipped object, a rotation that’s slightly wrong — to manufacture, at scale, the failure trajectories that real-world data collection almost never captures on its own. They used it to train a model called AHA, whose entire job is to look at a robot’s manipulation attempt and reason in language about why it failed. That framing — failure as something you reason over rather than discard — is precisely the shift from “record the answer” to “record the process.”

And underneath the current wave of coding-agent research, the pattern repeats at the infrastructure level: benchmarks and training pipelines with names like SWE-Gym ↗, SWE-smith ↗, and R2E-Gym ↗ exist specifically to manufacture large numbers of executable, self-verifying tasks rather than scrape more static code, because static code was never the bottleneck. Some of the most recent test-time-scaling work ↗ goes further still — instead of just generating more attempts, it explicitly compresses a rollout into a summary that preserves the failure modes and half-formed hypotheses, on the theory that the useful signal lives in those discarded pieces, not in the final answer.

None of this existed as a coherent field five years ago. It exists now because enough people independently ran into the same wall I ran into at two in the morning: a system that has only ever seen correct answers is a system that has never learned how to be lost.

Where I think the story gets too neat#

I want to push back on my own framing a little, because I think it’s currently more flattering than accurate.

Not all errors carry the same information, and I was sloppy about that distinction earlier. A typo is an error. A syntax mistake is an error. Neither tells you anything interesting about the boundary of a hard problem — they just tell you where the syntax rules are, which any parser already knows. The errors I actually care about are the ones near a genuine local optimum: the almost-right idea that fails for a subtle reason, the design that works until one specific edge case, the kind of mistake a competent person makes, not a careless one. FailGen’s approach — take a correct demonstration and perturb it procedurally — is honest about generating the first category. It is not obviously generating the second. A robot that fails because a script deliberately mis-rotated its gripper by ten degrees has not failed the way a robot fails when it encounters a genuinely novel object in a genuinely novel lighting condition. Those are different kinds of “wrong,” and I don’t think the field has cleanly separated them yet. Some of what’s being sold right now as “failure data” is closer to noise injection than to captured human struggle — useful, but not the same claim.

I also want to be honest that “an error one step away from the current state is a small act of innovation” is a clean analogy — it maps neatly onto exploration noise in reinforcement learning and onto simulated annealing ↗ accepting a worse state to escape a local optimum — but analogies that clean are usually hiding a step. Simulated annealing finds a better point in an already-defined search space. It does not explain how anyone arrives at a genuinely new search space — a new abstraction, a new architecture, a category of solution nobody had framed as a candidate. I don’t think anyone currently has a good mechanistic story for that kind of jump, and I’d rather say so than pretend the local-search analogy quietly covers it.

What I actually think is being wasted#

So here is the more careful version of the claim I started with, because “engineering is a waste of human life” was never quite what I meant.

What’s wasted is not the engineering. It’s that every engineer, in every generation, has had to personally regenerate the same trial-and-error signal that the engineer before them already generated and then threw away, because nothing existed to keep it. My three nights with the Kalman filter weren’t wasted because debugging is beneath me. They were wasted because someone, somewhere, has almost certainly hit the exact same coordinate-frame sign error before, learned the exact same lesson, and then let it die with them the moment their code compiled. Civilization has been running this experiment forever and discarding the results on purpose, one clean commit at a time.

If a system can finally be built that keeps that signal — not just the destination, but the specific, embarrassing, informative shape of getting there — then the argument for being “the last human engineer” isn’t a claim that engineering doesn’t matter. It’s closer to: I don’t want to keep being an unpaid, involuntary generator of a dataset that gets deleted every time I fix a bug. I’d rather help build the thing that finally keeps it, so that the next mind that hits my exact wall — human or otherwise — doesn’t have to rediscover, from zero, at two in the morning, something I already paid for in full.

I’m eighteen, I have one working drone pipeline and a handful of small projects to my name, and I’m aware that everything above is a compass, not a result. But it’s the first time I’ve been able to say precisely what I think is being thrown away, instead of just gesturing at “wasted time” and hoping the feeling was self-evident. That specificity is the actual output of this piece of writing, more than any conclusion in it.

Sources#

Nobody Writes Down How They Failed
https://fifth-postulate.pages.dev/blog/nobody-writes-down-how-they-failed
Author Haoran Yang
Published at September 2, 2026
Copyright CC BY-NC-SA 4.0

Sixteen-Year-Old Hands, Sixty-Year-Old Brain