There is a pattern almost everyone falls into with coding agents, and it quietly undermines the whole point of using one. The agent produces a change, and you ask the obvious question: "Does this work?" The agent says yes. You believe it, because it sounds confident and it usually is right. And every so often it is confidently, completely wrong — and you find out in production.
The problem is not that the agent lies. The problem is the question.
Why "does this work?" is the wrong question
When you ask an agent whether its own change works, you are asking it to evaluate its output using the exact reasoning that produced it. It is grading its own paper with the same pen. If it missed an edge case while writing the code, it will miss the same edge case while judging it, because nothing new has entered the picture — it is the same model, the same context, the same blind spots.
Worse, language models lean toward agreement. Phrase the question as "this handles null emails correctly, right?" and you have not asked for an assessment, you have offered a conclusion and invited the agent to confirm it. It usually will. What you get back is not verification. It is a very fluent echo of your own hope.
Ask for proof instead
The fix is small and changes everything: stop asking the agent to assert that the change works, and start asking it to show you. Proof is an artifact you can check yourself, produced by the system rather than by the agent's opinion of the system. Not all proof is equal — there is a rough hierarchy:
- Execution proof is the strongest: the agent actually ran the code and shows you the real output — the test that passed, the command that returned what it should.
- Before-and-after proof is nearly as good: a concrete change in observable state. The query count went from 47 to 2; the failing request now returns 200.
- A test the agent wrote is weaker, because the same blind spot that missed the bug can miss it in the test — but a test you can read and run is still far better than a claim.
- A verbal explanation is the weakest. Sometimes it is all you can get, but "here's my reasoning" is not evidence; it is the thing evidence is supposed to check.
The move in every case is the same: turn the yes/no question into a request for something observable.
What that looks like
Instead of "will this migration handle null emails?", ask it to seed a row with a null email, run the migration, and show you the output. Instead of "did this fix the N+1 query?", ask it to add query logging, hit the endpoint, and show the before-and-after count. Instead of "does the rate limiter reset?", ask it to hit the limit, wait, try again, and show the request succeeding. In each case you have replaced a claim you would have to trust with a result you can read.
This is why our agent lives in a sandbox
Proof only works if the agent can actually run things, and that is a design decision, not an afterthought. Every TaskGoblin run happens in a sandbox with the repository cloned and the toolchain available, so the agent is not reasoning about what the code would do — it can run the tests, execute the commands, and observe the real result. When it reviews a merge request, its findings point at concrete behaviour, and when you reply @taskgoblin fix, the change is one it can exercise rather than merely argue for.
This is also why a good development setup pays off more than ever. An agent that can spin up your services, load seed data, and run your test suite can produce real proof; an agent that can only read the code can only give you an opinion. The more runnable your project is, the more of the agent's output you can verify instead of trust.
The one-sentence version
Do not ask an agent whether its work is correct. Ask it to show you that it is — and prefer the kinds of showing you can check yourself. The agent that runs the code and hands you the output is doing the one thing the agent that vouches for the code never can: giving you something to believe other than its own confidence.
