Automated AI research is on the horizon, a former Anthropic and OpenAI researcher has said, warning that it could leave humans completely unable to control what comes next.
Jacob Coxon, the 27-year-old who publicly resigned from Anthropic last month after working as a capabilities researcher, made the comments Monday to New York City lawmakers as he sounded the alarm that AI could threaten humanity’s survival.
Coxon testified that automating AI research has become the “primary goal” of cutting-edge AI labs.
“At my last job, I was working, in some sense, to automate myself,” Coxon said at the New York City Council hearing on AI risks.
“When I was at OpenAI, we had milestones for an automated AI researcher, around 2027, 2028, and we are on track or surpassing those expectations,” he said. “It’s a matter of months, not decades or even five years.”
Coxon said that once AI can conduct its own research, “humans will be much further out of the loop than they are today, and today that’s already not great.”
Drawing on what he calls first-hand experience, Coxon said the majority of code written today is written by AI. “And people don’t check that as carefully anymore,” he testified.
OpenAI and Anthropic did not immediately respond to a request for comment on Coxon’s remarks Tuesday.
Coxon was part of a panel of AI industry whistleblowers who testified during the hour-long hearing, which also included testimony from representatives from OpenAI, Anthropic, Google and Meta.
“This hearing is not about stifling innovation,” City Council Speaker Julie Menin said at the start of the proceedings, explaining that “the real goal of this hearing is to ensure that innovation happens responsibly.”
In his testimony, Coxon told board members that his experience at Anthropic and OpenAI convinced him that companies were moving forward without putting adequate safeguards in place.
“It is more likely than not that humanity will lose control to these AIs, and that could result in the extinction of humanity,” Coxon said, adding, “In my experience, companies are extremely reckless given the stakes.”
When Coxon announced his resignation from Anthropic on X in a now-viral post, he said neither Anthropic nor OpenAI were “acting responsibly” and alleged that both were “playing with our lives.”
At Monday’s hearing, Coxon called for more transparency from AI labs and a slowdown in the development of rapidly evolving technologies.
“Companies operate on a startup mindset: move fast, break things, fix them later. That works for a photo-sharing app. It doesn’t work for creating the most powerful technology ever built,” he testified.
Days after Coxon resigned, Anthropic CEO Dario Amodei published a new, nearly 4,000-word essay urging the AI industry to slow down.
“We need to slow down the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem rapid, and we must use the time we gain wisely.”
In response, OpenAI CEO Sam Altman said in an article
Gn bussni
