This time, Claude has truly blown the AI programming leaderboard out of the water!
On the just-refreshed MirrorCode leaderboard, Claude Fable 5 has once again topped the charts with an absolute success rate of 64%.
Hot on its heels, GPT-5.6 Sol's score is only one-third of that!
GPT-5.5, ranked fourth, is even worse off. Not only did it score only 10%, but it was also crushed by its own predecessor, GPT-5.4.
Even more surprising, when using high-resource languages like Go, Fable 5's solve rate is 64%; when switched to the niche language Ada, the score still remains as high as 61%.
It's worth noting that in the open-source world, Python corpora are approximately 230 times larger than Ada's.
But with Fable 5, the score only dropped by 3 percentage points.
This is interesting.
If the model primarily relied on memorizing syntax and common patterns of popular languages, then switching to Ada should have caused a drop in performance.
But the current results point to another possibility—
The strongest models have moved beyond the pull of specific corpora and have begun to learn how to build a complete software project from scratch.
Stuck at the 100% completion line, a 10 billion token extreme test
Specifically, the full MirrorCode includes 25 target programs, covering areas such as Unix tools, interpreters, data querying, bioinformatics, cryptography, and compression tools.
Here, the model is placed in an isolated environment, with no internet, no access to the original project source code, and no ability to download third-party dependencies. It only gets high-level documentation, a portion of visible tests, and an original program it can repeatedly call.
Next, it must continuously feed data to the original program, observe outputs to guess the internal logic, and then write a new program that behaves identically.
The latest leaderboard selects 15 Medium and Large targets from these. Each target uses two implementation languages, and each language is run three times.
Moreover, the completion rate for both visible and hidden tests must reach 100% to pass; 99.9% is not acceptable.
If even one edge case is missed, the entire run is still counted as a failure.
To force the model to fill in the last few gaps, MirrorCode pushes the single-run budget to 10 billion tokens, allowing a maximum continuous run of 7 days.
The most expensive single task in the paper is even more extreme: the model ran continuously for 19 days, with a single run costing $2,600.
During these 19 days, the model repeatedly runs the original program, compares results, fills in missing functionality, and then reruns the tests. If errors occur, it investigates the cause; if outputs don't match, it changes hypotheses; once partial passes are achieved, it moves on to chase the next gap.
The entire process resembles a debugging session lasting several days, rather than a single generation.
The gotree example is the most illustrative.
This bioinformatics tool originally had approximately 16,000 lines of Go code and over 40 commands.
Claude Opus 4.7 took 14 hours and $251, passing 2,000 out of 2,001 tests, achieving a completion rate of 99.95%.
Although it missed a niche edge case involving date annotations and was stopped by MirrorCode's 100% completion line, it had already compressed what was originally a weeks-long engineering effort into just over a dozen hours—
Epoch estimates that without AI, a human engineer would need at least 2 to 17 weeks to complete the same task.
In the corpus desert, Fable 5 only dropped 3 points
The MirrorCode paper used StarCoder's public training mix as a reference.
Among it, Python accounts for about 8%, while Ada is only 0.034%, with the former being approximately 230 times larger than the latter.
Of course, we cannot know how much Ada code the closed-source models have actually seen. But using the public ecosystem as a reference, the scarcity of Ada is already intuitive enough.
This language mainly appears in aerospace, defense, and other safety-critical systems. Whether it's community size, number of tutorials, or open-source projects, it is far from comparable to Python, JavaScript, or Go.
And Fable 5 is clearly not translating Go code line by line into Ada.
It is more like first understanding how the original program actually works, then switching to another language and recreating the same behavior.
In contrast, other models are not so stable.
GPT-5.6 Sol dropped from 24% to 19%, GPT-5.4 from 21% to 12%, and GPT-5.5 even fell from 17% to 5%. Once the language changes, the gap is immediately amplified.
Handing the Entire Project to AI
Nowadays, Cursor has allowed hundreds of agents to collaborate for nearly a week, writing from scratch over 1 million lines of browser code distributed across 1,000 files.
Anthropic, on the other hand, had 16 Claude agents run nearly 2,000 sessions in parallel, ultimately building a C compiler with 100,000 lines that can compile the Linux 6.9 kernel.
In the sample of Codex personal users disclosed by OpenAI as of May, 70.2% submitted at least one task estimated to exceed one hour of human work; 25.6% submitted tasks exceeding eight hours.
The unit people hand over to AI is shifting from a piece of code or a bug to an afternoon, a week, or even the entire project.
MirrorCode's 64% is precisely a quantification of this "project-level delegation."
It shows that as long as the goal is clear enough and the results can be automatically verified, frontier models can already independently complete a portion of medium-to-large software.
For everyone who is handing work to AI, the change is already imminent—
Previously, you had to watch it write every piece of code; next, you are more likely to only check at key nodes whether it has gone off track.
Code will become cheaper and cheaper.
And those who can articulate problems clearly and verify results accurately will become more and more valuable.
Reference: https://epoch.ai/MirrorCode
This article is from the WeChat public account "新智元", author: ASI启示录; editor: 摩西
