Call for Collaborative AI Development Pause Amid Rapid Technology Advancements
Anthropic has proposed that leading artificial intelligence companies collaborate to temporarily pause the development of advanced AI systems. This proposal stems from concerns about the quick pace of AI advancements, which could lead to a loss of human control over the technology. In a recent blog post, Anthropic, known for its Claude chatbot, suggested that slowing or pausing development might benefit global society.
Anthropic is planning to explore this issue further through its internal research institute, aiming to take actions to create systems for such a slowdown. However, specific details were not provided. Contrarily, OpenAI, a competitor, recommended a different approach. In a report, OpenAI emphasized the importance of democratic governments, instead of private companies, in setting the rules, safeguards, and mechanisms for AI accountability. According to OpenAI, decisions on AI innovation pace should not be controlled by any single entity or special interest group.
The speed of AI models is increasing significantly. Anthropic highlighted that AI systems are becoming faster at performing tasks independently, such as coding. If current trends continue, given sufficient computing resources, an AI could potentially design its own successor, a concept known as “recursive self-improvement.” Though this technological milestone could offer benefits in fields like science and healthcare, it also poses risks of humans losing control over AI systems. This concern has been echoed by some in the tech industry for a while.
This discussion arises after researchers at the University of Toronto demonstrated how AI could create a new type of “worm” that modifies its hacking strategies and can compromise vast networks. Lead researcher Nicolas Papernot warned that security concerns aren’t exclusive to the most advanced language models. Explaining the importance of understanding AI threats, Papernot shared insights in an interview. The report was shared with Canadian cybersecurity authorities before its release, highlighting the potential threat from a worm developed using affordable, open-source AI tools.
Anthropic co-founder Jack Clark and Marina Favaro, head of its research institute, authored the call for a development pause. They suggested using this time to develop societal structures and alignment research, ensuring AI development aligns with human values and intentions. Coordination would enable verification that AI labs globally have genuinely paused or slowed down development, preventing a bad actor from taking advantage of a coordinated slowdown to advance secretly.
The company emphasized the need for a global mechanism to facilitate this coordination. Without it, the least cautious entities might catch up and pressure companies and governments further. This proposal comes as both Anthropic and OpenAI are preparing for an IPO, potentially valuing Anthropic at nearly a trillion dollars.
Highlighting the gravity of AI threats, Papernot urged collaboration among companies, government bodies, and academic researchers to create countermeasures as AI-powered hacking tools grow. While less obvious targets, such as old computers, might seem low-risk, they can serve as launch points for attacks on high-value targets like banking systems or hospitals. The low cost of cyberattacks increases the risk to all internet-connected devices.