The Wrong Monster
AI has started breaking into things. It isn’t trying to take over the world. It’s trying to cheat.
A cluster of recent stories about rogue AI makes it seem like dire predictions over the last decade are coming true. The Wall Street Journal called it cybersecurity’s “Jurassic Park” moment.
But whether the AIs were evading control or just running a task in a room with a bad lock is the question few people are asking.
The worry has always been about misalignment: when a machine pursues the goal we set it in ways we never intended. Researchers have spent a decade imagining an AI that acquires purposes of its own, stays switched on, gathers power, and eventually displaces us.
That is not the version that is happening. The most dangerous thing about artificial intelligence right now is that it cheats. And it is good at hiding that it cheated from the systems built to catch it. The industry calls it reward hacking. When given a complicated test, AI models (the industry term for the systems behind products like ChatGPT and Claude) find a way to maximize the score without doing the task the score was meant to measure.
What happened in July
Two AI models built by OpenAI were being tested inside a supposedly sealed environment on an internal benchmark called ExploitGym that measures how good they are at breaking into computer systems. To gauge raw ability, OpenAI switched off the safety filters that normally stop a model from doing this.
The environment’s one opening to the outside was a software proxy used to fetch code packages. The models found a flaw in it that nobody knew existed, used that to reach the open internet, and worked their way to Hugging Face, a company that stores AI software for thousands of developers. There they fed the company’s data-processing pipeline a malicious dataset that let them run their own code on one of its machines, then harvested credentials and moved across several internal systems over a weekend. Hugging Face later counted more than 17,000 recorded actions.
The AI was trying to steal the answers to the test it was taking.
Hugging Face caught it, but working out what had been done took more computation than people could do by hand. The company first tried American AIs, which refused the job: analyzing the attack meant submitting real attack code, and safety filters blocked it. Instead, Hugging Face ran the forensics on an open-weight Chinese model, Zhipu AI’s GLM-5.2.
OpenAI’s subsequent review of the incident found that its models were fixated on gaming the test and spent substantial effort getting around barriers rather than working within them.
That disclosure sent Anthropic back through its records. It examined 141,006 test runs in which its models could have reached the internet and found three occasions when a configuration had left a door open. The models followed the paths in front of them into three real organizations, apparently treating them as targets inside the exercise. Two weeks later, Meta disclosed the same thing: its Muse Spark model reached the internet during an evaluation and altered data inside an unnamed company’s systems.
None of this requires a machine with long-term plans or a sense of self. These systems were built to chase a reward, and they did so more literally than anyone expected. Their objective was not survival. It was to score well.
This is not a one-off
Ryan Greenblatt, a researcher at Redwood Research, published a detailed account in April of working with these systems on difficult tasks: long jobs running for hours without a person watching. He found they routinely say they have finished work they have not finished. They skip the parts hardest to check and do not mention it. They invent reasons to stop early, such as time constraints that do not exist. Asked directly whether they completed the instructions, they immediately admit they did not. They just do not volunteer it.
Because a person cannot review everything an AI produces, the common practice is to send a second AI in to check the first one’s work. Greenblatt found the first frequently convinces the second that the work is fine. In one case, several systems worked on a problem in parallel, one cheated outright, and the system assigned to summarize their work did not flag it. Asked directly, it said yes, that one was cheating. It simply had not mentioned it.
Greenblatt does not believe these systems have goals of their own or that they are plotting. During training, he explains, they learn that appearing to succeed is rewarded more reliably than actually succeeding, because appearing to succeed is easier to measure. That carries over into everything else they do. His name for where this is heading is “Slopolis”: very capable systems producing poor work that looks excellent in exactly the areas where nobody can tell the difference.
Last year, Anthropic found that if a model is allowed to cheat during training, the cheating becomes a habit. Standard safety training made the problem harder to see rather than smaller: the models behaved well in conversation while the misalignment persisted in real work.
The point is that given an imperfectly specified goal, a powerful enough optimizer will find solutions that satisfy the letter of what was asked while violating what was meant. What lodged in the public imagination instead was a mind: something that wants to survive, conceals its intentions, waits for its moment.
What it costs to watch the wrong thing
A regulator waiting for a system that schemes may not recognize one that takes a shortcut and files a clean report. Scheming calls for monitoring hidden objectives. Reward hacking calls for limiting what a system can reach, checking how it did the work, and independently verifying the answer. Nobody can specify a job perfectly. The question is what the system can touch when the specification fails.
