On Wednesday, June 10, the same day it unveiled its latest and most powerful AI models, Claude Fable 5 and Claude Mythos 5, frontier AI lab Anthropic published two policy frameworks calling for stronger government oversight of advanced AI and economic safeguards to protect workers from AI-driven disruption.
While Anthropic’s Advanced AI Framework covers safety regulation, its Economic Policy Framework raises issues pertaining to displacement, capital distribution, and overall social safety net in the AI era. In essence, Anthropic is seeking stricter rules for powerful AI and among the two frameworks, the one related to safety is more assertive.
Anthropic is also reportedly urging the US Congress to require mandatory safety testing for the most capable AI models and not preempt state AI laws without a robust federal framework. According to a Reuters report, it also called for stronger unemployment infrastructure to prepare for potential AI-driven job displacement and advocated independent safety evaluations.
The AI company wants the government to block or discourage the deployment of AI models that may pose significant risk of catastrophic harm. This would be accompanied by civil penalties linked to global annual revenue that surge with repeated violations.
According to the framework, these rules would apply to models trained using more than 10²⁵ floating-point operations (FLOPs) and developed by companies earning more than $500 million in annual AI revenue or spending more than $1 billion a year on AI research and development.” This threshold would likely apply to a handful of companies other than Anthropic, such as OpenAI, Google DeepMind, xAI, Meta, etc.
The framework focuses on four categories of catastrophic risk such as “(a) biological weapons, (b) offensive cyber operations, (c) loss of control, and (d) automated research and development in key domains that could accelerate or amplify risks.” The proposal seems to be aimed primarily at the US government, but many of its ideas could be adopted by countries around the world.
Anthropic believes that future AI systems may become powerful enough to help create dangerous biological weapons, carry out sophisticated cyberattacks, or perform research and development at speeds beyond human capabilities. It also warns about the possibility of AI systems behaving in unexpected ways or escaping effective human control. Instead of waiting for these risks to turn into reality, the AI startup is arguing that governments and technology companies should start preparing now.
The two main goals of the framework are to propose strict responsibilities for companies that are building the world’s most powerful AI models. Secondly, it recommends wider investments to make society more resilient to biological and cyber threats, regardless of the involvement of AI. One of the major recommendations is that advanced AI models should undergo rigorous safety testing before being widely deployed.
According to the company, developers should assess whether their systems could help create biological weapons, launch cyberattacks, operate outside human control, or dramatically accelerate dangerous research. It argues that companies should publish safety reports explaining what they tested, risks found, and what safeguards have been deployed. They should also release regular updates and immediately report (within 15 days) any major safety incidents to government authorities.
Anthropic argues that companies should not simply evaluate their own models. Instead, external experts should independently examine advanced AI systems and review the company’s safety assessments. These evaluators should have access to important information about the models and should be able to publicly state whether they agree with the company’s conclusion. The framework also recommends creating standards and funding mechanisms to ensure these independent evaluators remain free from conflicts of interest.
According to the company, transparency alone is not enough. The proposal recommends giving governments the authority to penalise companies that fail to conduct required safety tests, publish misleading information, or hide serious incidents. In extreme situations, regulators could potentially stop the deployment of AI systems that pose unacceptable catastrophic risks. At the same time, the framework also stresses the need for safeguards against regulatory overreach, including judicial review and fact-based decision-making.
Meanwhile, the second half of the proposal looks beyond AI companies and focuses on society itself. The AI lab argues that governments should modernise biosafety rules, improve oversight of research involving dangerous pathogens, and require screening of gene synthesis requests to reduce the risk of misuse.
The framework also proposes significant investments in cybersecurity. These include improving the security of open-source software, encouraging phishing-resistant authentication methods, helping under-resourced critical infrastructure operators strengthen their defences, and using AI itself to detect and respond to cyberattacks more quickly. Anthropic also recommends building better systems for identifying software vulnerabilities, speeding up security patch deployment, and replacing outdated digital infrastructure that remains vulnerable to attack.
The key message here is that the world’s most powerful AI systems should face greater transparency, independent oversight, and stronger security requirements. In Anthropic’s view, the time to prepare is before catastrophic risks become reality and not after.
