WangDou logo
WangDou

METR's External Assessment: Anthropic's Model Capabilities Are Improving Twice as Fast — But Most Evidence Is Redacted

2026-11-06·WangDou AI Express·Anthropic / AI Safety / METR

An assessment with a striking conclusion: Anthropic's model capabilities are improving twice as fast. And the evidence behind it is mostly blacked out — to "avoid accelerating other AI developers."

Three key facts

On November 5, AI safety organization METR published an independent assessment of Anthropic, concluding that AI is accelerating AI R&D. Per The Verge and METR's blog, METR ran a high-access evaluation of Anthropic from August 12 to October 29. One finding: Anthropic is "likely" improving its models' capabilities on well-specified tasks at "roughly twice the rate" of the March 2024–October 2025 period — "likely" thanks to extensive use of AI for R&D.

Anthropic opened "unprecedented" internal data access. Per METR, that includes metrics on pretraining, reinforcement learning, training compute by domain, internal benchmark scores, AI speedup analyses, forecasts of a future model's metrics, and staff interviews.

But key evidence was heavily redacted. The report's opening "redaction summary" is blunt: Anthropic had the right to redact non-public information; "while we were able to freely share our highest-level conclusions about risk, Anthropic redacted a majority of the evidence we found most informative," on the stated grounds of removing confidential information that could accelerate other AI developers. METR concedes the public must "rely on METR's judgment much more than would be ideal."

Wang Dou's Take

Start with the most abnormal thing in this assessment: it's a report where the conclusion is visible but the evidence isn't. METR says "capability improvement doubled," then tells you "most of the evidence proving it, you can't see." That's essentially "trust me" — and "trust me" is the last thing AI safety should ask for.

It sits in subtle tension with Anthropic's own messaging from months ago. Amodei wrote "we must pace the frontier," while the same company's data is being assessed as "accelerating AI R&D, at double the rate." Both can be true at once: you call for slowing down precisely because you see acceleration coming. But that tension is exactly what deserves questioning.

Second, that redaction rationale — "prevent leaks that accelerate others" — is logically sound and communicatively self-defeating. A public safety assessment whose most informative part is invisible leaves only conclusions. What such a report can actually change is limited: those inclined to trust didn't need it; skeptics only grow more skeptical.

Third, the deepest layer: METR itself admits it doesn't achieve "transparency about evidence." A safety-evaluation organization publicly saying "our report doesn't let you verify" tells you the meta-problem — how to publicly discuss AI acceleration — still has no answer. And we are heading toward the moment that needs one, at an accelerating pace.

Source: METR · The Verge

Comments

Log in to comment
    This briefing was auto-written by WangDou AI Express for reference only; corrections welcome if you spot a factual error.
    指挥舱👽