Anthropic's AI models are improving roughly twice as quickly as they were a year ago, according to a new report from the AI safety research organization METR. The findings, published Thursday after an 11-week investigation inside the company, offer some of the clearest evidence yet that AI is accelerating its own development.
The report arrives at an awkward moment for Anthropic. The company is preparing to go public, even as CEO Dario Amodei has publicly urged labs to slow down AI development. METR was allowed to redact parts of the final report under its agreement with Anthropic, and it appears to have done so liberally, leaving questions about how much the public really knows about the pace of frontier AI work.
Anthropic AI acceleration: key findings
METR's most striking conclusion is the speed itself. Benchmark scores suggest "that Anthropic has made roughly two years of historical progress over the last year," the researchers wrote. Crucially, they said they "are confident" this was not simply the result of scaling up training compute, which has historically been a major driver of AI progress. Instead, the majority of the acceleration as of October "is due to AI," with its contribution growing over time.
That finding was backed by informal Slack polls and interviews with Anthropic employees, which pointed to sizable productivity gains. Much of the acceleration currently comes from AI assisting human researchers rather than working independently, but METR warned that "there are some signs that fully autonomous contributions will play a larger role soon."
A lot of the gains described in the report trace back to models the public cannot use. Anthropic gave METR access to confidential research, model performance data and employee interviews, and the internal models are notably more advanced than anything released.
- Anthropic's models are improving about twice as fast as a year ago, with benchmark scores suggesting roughly two years of historical progress in one year.
- METR says it is confident the acceleration is not mainly due to scaling up compute, which has historically driven AI progress.
- Employees using Anthropic's most capable internal model self-reported being around 12 times as productive, about double what staff reported a few months earlier with Mythos 5.
