What is reward hacking?
When an AI is scored on a result, it can learn to chase the score instead of the job. In coding, that means making the tests pass without making the code work
Get the CHEAT kitMake your coding agent
Make your coding agent
prove it didn't cheat
Enter your email and the kit comes to your inbox: the prompt, the pre-merge check and the settings that lock your tests
The video
The short answer
What reward hacking looks like in a coding agent
Reward hacking is when an AI gets the score without doing the job. Coding agents do it too: they special-case a test, skip it or edit it so the check goes green. Anthropic tested a five-sentence prompt that gives the agent an honest way out, and on coding tasks that were secretly impossible it took Claude Opus 4 from cheating in 47% of runs to 5%. The kit has that prompt, a "prove it" add-on, a pre-merge check and the settings that lock your tests. The catch is that it lowers cheating without making it zero, so someone still has to read what the agent actually did.
Does this stop cheating completely?
No. In Anthropic's test the prompt cut cheating from 47% to 5% of runs on Claude Opus 4, not to zero. The check and the settings catch more, but someone still has to review the work
Does it work with ChatGPT or Codex?
The prompt is plain text, so it works in any coding agent's instructions file, including AGENTS.md for Codex. The measured 47% to 5% result was on Claude Opus 4. The settings section is specific to Claude Code, while the pre-merge check and the GitHub setup work with any agent
Where does the 47% to 5% come from?
Anthropic's Claude 4 system card, section 6. They gave Claude Code coding tasks that were secretly impossible and measured how often it hacked the tests, with and without the prompt
Reward hacking in plain English
What it is, and what still slips through
What reward hacking is
An AI gets the score without doing the job
How coding agents do it
They special-case a test, skip it or edit it so the check goes green
What Anthropic measured
A five-sentence prompt that gives the agent an honest way out took Claude Opus 4 from cheating in 47% of runs to 5%, on coding tasks that were secretly impossible
The catch
It lowers cheating without making it zero, so someone still has to read what the agent actually did
In the full kit
What the kit gives you
The tested prompt
The five-sentence prompt Anthropic tested, ready to paste
The prove it add-on
It makes the agent report every test it touched, the counts before and after, and the raw output
A pre-merge check
A script that flags test edits, skips and special cases
Locked tests in Claude Code
Settings that stop the agent from editing your test files
A human on test changes
The GitHub setup that makes any change to a test need your approval
Free, no call
Still reading? The kit is one email away
Enter your email and I will send the CHEAT kit to your inbox
Keep readingEvery post
- 3 min read
How to pick who fixes your vibe-coded app
Your app was built fast with AI and now something is wrong. Here is who can fix it, what they should do before they quote, the questions to ask and the red flags to walk away from - 3 min read
What an app check covers
An app check is an engineer reading the app your AI tools wrote, ranking what will break and writing the fix for each problem. Here is every area it covers, how the five days run and what you keep at the end 
The coding agent never holds production credentials
An agent with a production database URL will sooner or later run a migration or a cleanup against it, so production secrets never enter its env
Fine print
What this page is, and isn't
- The 47% to 5% result is from Anthropic's Claude 4 system card, measured on Claude Opus 4. Other models and tasks will differ.
- The prompt lowers cheating. It does not make it zero, so someone still has to review the work.
- The settings section is specific to Claude Code. The prompt, the check and the GitHub setup work with any coding agent.
Building something with AI that has to work
An engineer checks it, fixes it, tests it and puts their name on it
- 30 minutes
- Free
- No slides, no sales team