There is an argument going around, and it deserves to be stated at its strongest before anyone takes it apart.
It goes like this. The market has stopped experimenting with AI and started installing it. The economics are no longer interesting to argue about: an internal tool that used to cost a year and a team now costs a competent person and a subscription. Everybody can see what their competitors ship. The developer who still says the models are toys is not being careful, he is being expensive, and he will be replaced by someone who is not. And then the step that makes the argument sharp: the process itself does not change. Specification, implementation, review, testing, release. Every one of those steps is a step, and a step can be executed by an agent. Validation included. If you are still assigning review to a human, you have understood the tooling but not the economics.
Most of that is right. I have been running a spec-driven factory for six months and building agentic pipelines for about a year, and I would not argue with the first half of it. I would not even argue with replacing people who refuse to learn the tool. That has happened in every trade that got a new machine.
The last step is where it breaks. Not because models are weak. Because of who answers for the result.
A week of green pipelines
This week my factory made music videos. It is a good testbed precisely because it is not mission critical: nobody dies if a caption lands late, so nothing artificial props up the quality. What happens is what the process actually produces.
Here is what the process produced, all of it from one working day, all of it with every stage reporting success.
The cover. Six candidates, generated locally, all six technically clean: correct aspect, correct mood, no artifacts a checker would catch. One of them put lettering on the object in the frame, and the lettering was gibberish, the way image models still write. The instruction not to render text was in the prompt. The generator ignored it, reported success, and produced a file that would have gone out with nonsense printed on it. It took a human half a second to see and reject it.
The captions. I align known lyrics to the vocal stem: the text is truth, the machine only supplies timing. The aligner returned a complete, monotonic, internally consistent mapping. It was wrong. A repeated hook had stolen two words from the line before it, so one line ended early and another started too soon. Nothing in the output looked broken. Both readings were self-consistent, the confidence was high, and the only way to catch it was to listen to the song with the file in hand.
The silence that was not silence. For twelve seconds in the middle of the track, the transcriber returned nothing at all. In the data that is indistinguishable from a pause. In the song it is the hook, shouted and chopped. I only found it because I measured the energy of the vocal stem across that window and saw it was as loud as the chorus. A pipeline that trusted its own transcript would have placed two lines of the hook wherever the interpolation felt like putting them.
The archive. For another track we needed wartime footage, and the brief had a constraint that no metric encodes: no shots of the successful attack, because the song mourns a boy who did not want to become a symbol, and footage of a victorious raid argues against the lyric. Title filtering gets you part of the way. It does not get you a reel that is titled flight deck activities and contains burning ships in its second half. Somebody had to sample forty eight frames and look at them.
And the joke of the day. My voice bridge transcribed a long dictation and inserted a line the speaker never said: a subtitle credit, the kind that runs at the end of a pirated video. Whisper models learned from YouTube captions, so when they meet silence they complete it with the most common ending in their training data. Confident, fluent, fabricated, in the middle of a real sentence.
Five failures. Zero red lights. Every one of them was caught by a person looking at the artifact and asking whether it matched the world.
Why another agent does not close the gap
The obvious reply is that I am describing missing automation. Put a verifier agent after the generator. Have the model check the model.
I run cross-vendor review in the factory and I recommend it: a second model, from a different vendor, reading the diff against the spec, catches a great deal, and the numbers in my own logs are the reason I keep paying for it. But it does not close this gap, for two reasons that survive better models.
The first is correlated blindness. The verifier shares the generator’s priors. It is unbothered by gibberish lettering for the same reason the generator produced it. It reads the caption file as consistent for the same reason the aligner produced a consistent file. When the failure is a plausible, well formed artifact that happens not to match reality, the second model is exactly as likely to accept it as the first, and it will accept it in the same fluent tone.
The second is that the verification a model performs is a claim about a document, and the thing I need is a claim about the world. Did the file actually play. Did the port actually open. Does that frame actually contain what the brief forbids. Agents can execute checks, and mine do, thousands of times a day. What they cannot do is be the one who says I looked, and it is true.
Nobody fires a model
Here is the part that I think is missing from the whole argument, and it is not technical.
When a person tells me a feature works, that sentence carries a stake. If it turns out they never checked, something happens to them. They lose trust, they lose the next project, they lose the job. Because that consequence exists, their yes contains an implicit I actually went and looked, and the whole discipline of engineering rides on that small implication. The organisation is not a pile of artifacts, it is a chain of people who have staked something on their answers.
A model has no stake. It has no fear of being switched off and no memory of yesterday’s disgrace. You can praise it, scold it, threaten it, thank it. Tomorrow, on the same input, it returns the same output, and it has no attitude toward how you felt about the last one. The remedy that works on people does not exist here: catch a person in a lie and their behaviour changes, catch a model in a hallucination and nothing changes, because there is nobody in there for the catching to land on.
And the replacement move fails too. Fire a careless engineer, hire another, state the expectation clearly, and you have improved your odds. Swap one model for another and you have the same failure mode with a different accent. This is not a complaint about the current generation. It is a structural fact about delegating to something that has nothing to lose.
You cannot fire a model. That sentence is the whole reason the last step stays human, and it will still be true when the code is perfect.
So the gate is not there because the machine is stupid. It is there because accountability does not transfer. Somebody has to be able to say I checked, and I am answerable for it, and that somebody has to have something at risk. Every regulated industry figured this out long before software: the signature at the bottom of the drawing is not a technical control.
The gate is not only a veto
There is a misunderstanding worth clearing up, because it makes the gate sound like drag. The human at the end is not merely the one who says no.
Half of what I do at that gate is say yes to something I did not ask for. In one of this week’s clips the agent proposed an idea for the visual treatment that I had not specified and would not have thought of, and it was better than my plan. I took it. That is also the gate working. A pipeline with no owner at the end has no way to promote a good accident, because it has no taste and nothing to compare against except the spec it was given.
And the practical shape of the gate is smaller than people expect. Three questions, every time. Does this artifact say what we agreed it would say. Is there a check that connects it to the world, and did I watch it run. Am I prepared to answer for it. That is minutes, not days.
I have started writing those checks down as documents that execute: each step in the setup carries its own probe, the probe reports green or red against the actual machine, the critical ones refuse to let you continue, and a person presses the button. It removes the part of verification that is clerical and keeps the part that is judgment. The machine proves the port is open. I decide whether we ship.
If the code becomes perfect
I will not pretend to know where model capability lands. It is entirely possible that in some number of years the generated code is better than what a good engineer writes, with fewer defects, and that the failures I listed above read as quaint.
That would not remove the gate. It would only change what the gate is for. The question at the end is not is this correct, which is a property of the artifact. The question is are we shipping this, and who says so, which is a property of a person. As long as consequences land on humans, humans hold the last step. The day a model can be sued, sacked, or ashamed, we can talk again.
Until then the honest version of the argument is narrower than the one going around, and I would sign the narrow version. Put agents on every step of the process. Replace the people who refuse to use them. Automate the review until it is continuous and cheap. And keep one person at the end who looked at the thing with their own eyes and is willing to put their name on it.
The alternative is not a faster factory. It is a very convincing report about a factory, and no way to find out whether it was ever there.