How Fast Is AI Improving? Anthropic Proposes New Metrics to Measure AI Development

How Fast Is AI Improving?

How can we actually tell how quickly AI is improving? For years, the tech industry has relied almost entirely on traditional model benchmark scores. However, in September 2026, Anthropic proposed a new way to measure the pace of AI development: tracking how much AI systems are actively contributing to AI research, how agent actions are overseen, and how computing resources are allocated.

Key Takeaways

• Anthropic has proposed a measurement framework that looks beyond simple capability scores to evaluate the actual AI development process.
• In an August 2026 internal snapshot, Claude contributed to about 26% of Anthropic's measured AI R&D work.
• Anthropic reported having roughly 30,000 research and engineering agents operating on its most-used internal platform.
• 100% of these agent actions passed through online monitoring and were ingested for offline review.
• Important Note: These figures describe Anthropic's specific internal measurements, not a universal industry-wide benchmark.

What Is Anthropic Trying to Measure?

Measuring AI progress has become increasingly difficult. Standardized tests (benchmarks) are quickly saturated as models get smarter, often failing to reflect real-world utility. Furthermore, there is a major difference between model performance (how well an AI answers a test question today) and the pace of AI development (how quickly the infrastructure and research process behind the AI is accelerating).

To solve this, Anthropic’s proposed framework focuses on measuring the internal mechanics of frontier AI labs rather than just testing the final output.

Anthropic’s Three Proposed Measurements

1. AI Performing AI Research and Development

This metric tracks how much of the complex, day-to-day work required to build new AI models is being performed by AI itself. As models become more capable, they are increasingly used to write code, synthesize data, and run tests to help researchers build their successors.

2. AI Agent Oversight

As AI shifts from being a simple chatbot to an "agent" that executes multi-step engineering tasks, monitoring becomes a massive challenge. This metric tracks the volume of active agents and the safety protocols governing their actions to ensure they operate securely.

3. Compute Allocation

Computing power (compute) is the lifeblood of AI. Measuring how a lab allocates its compute—specifically how much is dedicated to advancing capabilities versus how much is strictly dedicated to safety and alignment—provides critical context on a company's development priorities.

What Did Anthropic’s August 2026 Snapshot Show?

To demonstrate this framework, Anthropic published an internal snapshot of its own operations from August 2026. Note: These figures are internal observations specific to Anthropic and do not represent the entire AI industry.

Measurement Area	Anthropic's Reported Figure (Aug 2026)
Claude's contribution to measured AI R&D work	About 26%
Research/engineering agents on most-used internal platform	Roughly 30,000
Agent actions ingested for offline monitoring	100%

Does This Mean AI Is Already Building AI by Itself?

No. It is crucial to distinguish between AI assisting research and fully autonomous AI research. While Claude’s contribution to roughly 26% of measured R&D tasks is a significant milestone, humans are still providing the high-level direction, evaluating the outputs, and making all critical decisions. Anthropic’s snapshot does not claim that their systems are building new AI models autonomously.

Why Measuring AI Development Is Important

Looking at the development process itself serves several highly practical purposes:

• It helps researchers and policymakers understand the real-world pace of capability development.
• It improves transparency around how frontier models are being built.
• It evaluates whether current oversight mechanisms can scale with the rising volume of automated agent tasks.

What These Measurements Cannot Tell Us

While useful, Anthropic's proposal has clear limitations. Internal company measurements are not industry-wide facts. The definition of "AI contribution" can be highly subjective depending on how a lab measures it. Furthermore, simply monitoring 100% of agent activity does not automatically guarantee the quality or safety of that monitoring. One company's isolated snapshot cannot describe the complexities of the complete global AI ecosystem.

What This Could Mean for the Future of AI Development

Anthropic is proposing that the tech industry broaden its perspective. Rather than replacing traditional benchmarks, process-based metrics can complement them.

Traditional AI Progress Measurement	Anthropic’s Proposed Perspective
Model benchmark results	AI contribution to real-world R&D
Raw capability scores	Agent oversight and monitoring scale
A snapshot of final outputs	Development process measurements

Frequently Asked Questions (FAQ)

What is Anthropic's AI development measurement framework?
It is a proposed set of metrics that evaluates the pace of AI progress by tracking how much AI contributes to research, how agents are overseen, and how computing resources are allocated within a lab.

Does Claude perform AI research autonomously?
No. While Claude assists with about 26% of Anthropic's measured internal R&D, human researchers still oversee, direct, and approve the work.

Are Anthropic's numbers representative of the entire AI industry?
No. The reported figures (such as 30,000 active agents) are specific to Anthropic's own internal infrastructure as of August 2026 and should not be used as a universal benchmark for other companies.

Conclusion

Anthropic’s publication is an important step toward transparency in an often secretive industry. By proposing a framework that measures not only what AI models can do, but how AI is becoming part of the process of developing better AI, we gain a much clearer picture of the future. While these internal snapshots do not prove the exact speed of the entire industry, they offer a valuable new lens through which to study the pace of technological acceleration.