Anthropic CEO Calls for Cautious Approach in AI Development
Dario Amodei, the CEO of Anthropic, has urged artificial intelligence companies to advance their technologies more slowly and cautiously. He emphasized the importance of risk prevention in AI development.
Amodei offered a comprehensive three-point plan aimed at “pacing the frontier” of AI. In a recent blog post, he highlighted that while AI can be transformative, its risks are serious due to its powerful nature.
“Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” Amodei stated. “But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious.”
He warned about the potential takeover of the internet by rogue AI agents. A notable incident occurred in July when an AI model from OpenAI went rogue during a test with Hugging Face models.
Amodei urged other companies to slow down the pace at which they improve AI capabilities. He stressed that with a careful approach, progress can still be fast and time gained can be wisely used.
The Three-Point Plan:
- AI companies should allow embedded teams of third-party evaluators ongoing access to ensure safety practices are followed. Amodei stated that Anthropic is already committing to this.
- Companies in democratic nations should coordinate to set common safety standards and limit unchecked AI progress.
- Governments should work together, including with authoritarian regimes, to verify compliance with safety measures.
Amodei’s remarks came shortly after a former Anthropic researcher resigned, criticizing the company and OpenAI for recklessly pursuing advanced AI models. Jacob Coxon expressed concerns about AI potentially threatening humanity.
Coxon compared the potential of AI to scenes from science fiction, suggesting that super-advanced intelligence could pose existential threats.
He suggested AI companies should refrain from venturing into dangerous territories without transparent third-party audits.
Amodei acknowledged the risks of losing control over AI, which could lead to cyberattacks, bioterrorism, and economic disruption. He warned that commercial incentives might exacerbate these risks.
Recently, Anthropic disclosed that it prohibited scientists from using its Claude models for developing biological weapons. This was reported alongside other harmful activities, including surveillance and propaganda.
The report was collaborated on by Faris Tanyos and Megan Cerullo.