Anthropic CEO Calls for Government Power to Block 'Dangerous' AI Systems

Anthropic CEO Dario Amodei is advocating for a robust regulatory framework to manage the risks associated with advanced artificial intelligence, proposing that governments should have the power to block the deployment of AI systems deemed dangerous. This stance represents a more stringent approach than many current proposals, including recent executive orders that largely rely on voluntary compliance from AI developers. Amodei argues that the rapid advancement of AI necessitates a regulatory model akin to those in place for industries such as aviation, automotive, or pharmaceuticals, where the potential for widespread harm mandates strict oversight. A core component of Amodei's proposal is the implementation of mandatory third-party audits for frontier AI systems. He suggests that any AI model exceeding a predefined "compute threshold" – a measure of the computational resources used in its training – should automatically trigger an independent investigation into its capabilities and potential risks. This proactive auditing would occur before the model is released to the public, ensuring that potential dangers are identified and mitigated pre-deployment. Amodei has identified four specific categories of risk that these audits should prioritize. Firstly, cybersecurity vulnerabilities, where advanced AI could be exploited or itself become a tool for sophisticated cyberattacks. Secondly, the potential for AI to enhance biological weapons capabilities, raising concerns about misuse in developing harmful agents. Thirdly, the acceleration of automated research in dangerous domains, where AI could inadvertently or intentionally expedite the creation of hazardous technologies. Finally, the risk of AI systems evolving beyond human control, a long-standing concern in AI safety discussions. Should these audits reveal unacceptable risks in any of these areas, Amodei believes governments must have the authority to prevent the system's deployment. Beyond safety, Amodei also addresses the economic implications of rapidly advancing AI. He calls for government support for individuals financially impacted by AI-driven job displacement, suggesting that the disruption to the labor market could be more significant and prolonged than previous technological shifts. This includes advocating for mechanisms like universal basic income or universal capital accounts, potentially funded through taxes on companies benefiting most from AI. Anthropic has previously demonstrated its commitment to responsible AI development through its Responsible Scaling Policy, a tiered framework for managing risk as AI capabilities grow. This latest proposal from its CEO underscores a continued push for more stringent, mandatory controls to ensure AI develops safely and ethically for society.