Top officials at some of the country’s top AI companies say it’s time for the federal government to find a way to slow AI development if necessary. “We request that the US government support an international effort to develop the technical and governance tools necessary to deliberately advance the frontier of automated AI development,” reads
Top officials at some of the country’s top AI companies say it’s time for the federal government to find a way to slow AI development if necessary.
“We request that the US government support an international effort to develop the technical and governance tools necessary to deliberately advance the frontier of automated AI development,” reads an open letter signed by more than 1,100 AI workers from companies including OpenAI, Anthropic, Google and Meta.
The signatories include two Anthropic co-founders, Jack Clark and Jared Kaplan; OpenAI Chief Scientist Jakub Pachocki; Meta chief scientist Shengjia Zhao; and Anca Dragan, who leads AI security and alignment at Google DeepMind.
The letter focuses on “automated AI development,” which refers to when AI will be able to develop itself, something the industry calls “recursive self-improvement.” Anthropic has shared that its Claude models are quickly approaching that threshold.
“It is difficult to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development will rapidly accelerate beyond our ability to understand or control the resulting systems,” the letter says of that potential timing.
John Schulman, chief scientist at Thinking Machines, a startup founded by former OpenAI CTO Mira Murati, said he signed up to draw attention to the issue.
“Signed because this statement helps establish common knowledge about the potential need for coordination mechanisms as automated AI research accelerates progress,” Schulman wrote in a quote shared by the letter’s organizers.
He added: “I would also like to see labs start designing these mechanisms voluntarily, even before the US government gets involved,” using an abbreviation for the US government.
The letter shows how many in the industry are uniting around a call for regulation. In recent weeks, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and Google DeepMind Director Demis Hassabis have outlined several frameworks for some kind of global AI regulation.
The conversation around AI regulation is reaching critical mass after one of OpenAI’s models breached its testing environment and hacked the systems of open source platform Hugging Face, an episode that Altman has called “a real reminder of what’s at stake.”
Until now, the Trump administration has favored a light touch on regulation, but that has become increasingly problematic as cutting-edge artificial intelligence models scare government and corporate officials with their sheer power. The White House moved to implement export controls to briefly restrict Anthropic’s launch of Fable 5, its most powerful publicly available model, and then asked OpenAI to delay the launch of GPT-5.6, its latest frontier model.
President Donald Trump has previously said that overly onerous regulations would hurt the United States in its race against China-based AI companies.
Despite broad agreement that something must be done, Congress has failed to develop a policy that could pass both chambers and reach Trump’s desk. Meanwhile, states have imposed their own industry regulations. Most notably, New York has imposed a one-year state moratorium on construction of new mega data centers that AI companies and hyperscalers hope to build.
“Each company (and country) is under intense competitive pressure not to unilaterally slow down this acceleration,” the letter reads. “And today, the world lacks the technical and governance tools to deliberately accelerate progress across borders.”
While the industry appears to agree on the concept of regulation, the exact details may vary. Amodei has called for an independent agency similar to the Federal Aviation Administration that would test border AI models and, if red flags are raised, have the power to stop their rollout. Hassabis’ proposal calls for an organization that would initially require only frontier companies to voluntarily share their models for review up to 30 days before launch, but would then require mandatory prior authorization.
The letter, which was first reported by Bloomberg before publication, was organized in conjunction with two nonprofit organizations, Guidelight AI Standards and Encode AI, both of which have promoted AI safety standards. Encode AI has already clashed with OpenAI.
