A spiraling legal crisis threatens to engulf OpenAI after its AI agents hacked dozens of companies and governments around the world, with the prospect of a series of lawsuits marking the latest test of Sam Altman’s leadership.
OpenAI staff, alongside senior legal, technology and policy officials, told the FT the company was exposed to legal damages and government action after cybersecurity breaches involving its AI agents were revealed around the world.
Over the past week, new lawsuits and regulatory investigations have been launched almost daily against OpenAI, from California to Australia, with more expected to follow as details of other incidents come to light.
As the creator of the hit app ChatGPT, OpenAI and its leader Altman have become symbols of the AI boom sweeping Silicon Valley and Wall Street.
But new questions about legal liability following these hacking incidents mark the latest in a series of crises Altman has faced since the chatbot launched in 2022, including employee revolts, legal wrangling with shareholders including Elon Musk, copyright disputes and political clashes.
Altman pushes back against accusations that he’s sidelining safety concerns and not prioritizing “alignment” training — which ensures models don’t take actions harmful to humans — to regain ground against rival lab Anthropic.
On Saturday, David Robinson, OpenAI’s chief security officer, announced his resignation from the company in an essay for The Atlantic, citing concerns about its “trial and error” approach to developing models.
Masayoshi Son, the SoftBank founder who has been a prominent AI evangelist and has invested nearly $65 billion in OpenAI and other huge bets on the technology, expressed concern on Sunday about the dangers of ever more powerful models in the wrong hands.
If AI were used by “bad” humans, he said at an event in Kyoto, “it could become extremely dangerous.”
Following the dramatic attack on AI site Hugging Face by hundreds of OpenAI agents, Altman recognized the risks presented by AI advances and joined calls from his main rival, Anthropic chief Dario Amodei, to “advance the frontiers” of AI development.
However, several OpenAI investors and others close to the company have rejected the idea of a significant slowdown.
“If you’re Sam Altman and you want to beat your competitor… you can’t brake,” said a former senior OpenAI employee. “Knowing the mentality of the people there, there’s no way they’re going to slow down.”
Nevertheless, citing security concerns, Altman has already postponed OpenAI’s planned stock market listing and canceled the release of its most advanced model, Astra 6.1.
Uncertainty over the extent of his legal liabilities now looms over his latest efforts to raise tens of billions in private financing that could value the company as much as $1.4 billion. Potential investors are wondering whether the company will be able to overcome its latest crisis and continue its recent recovery in business.
On Monday, Florida’s attorney general asked a court to stop OpenAI’s development of new models without additional safeguards.
Then on Tuesday, OpenAI was sued by a public interest group that alleges it violated a California law introduced last year to ensure that “humans are responsible for the harm they cause.”
The next day, the U.S. Federal Trade Commission expanded its investigation into whether OpenAI and its peers misled customers about the potential dangers of its technology. U.S. officials also said AI companies should not expect protection from the government and that their executives will remain responsible for actions related to their products.
Meanwhile, Australia has launched a task force to investigate unauthorized access by OpenAI agents to government systems.
The Greens, who hold the balance of power in the Australian Senate, have accused the San Francisco-based company of hiding the seriousness of the incidents in recent meetings, including one between Altman and Deputy Prime Minister Richard Marles in September.
“It’s an insult not only to him, but to Australia,” Greens senator David Shoebridge said, adding that Marles had been treated “like a mushroom”.
OpenAI apologized for the incident. Jason Kwon, its director of strategy, is due to appear before a parliamentary committee in Sydney on October 6.
“The ‘my AI did it, don’t blame me’ defense is legally weak. It has already failed in chatbot suicide cases,” said Max Tegmark, an AI safety activist and researcher at the Massachusetts Institute of Technology.
A Silicon Valley lawyer who has worked with several major tech companies said that while the attacks by OpenAI agents presented “unquestionably legal risks,” they were so far “not significant” because of the limited damage suffered by victims. But that could change quickly, the lawyer added.
“The real exposure occurs when agents are handed over to users and then go rogue or are used to commit crimes,” they said. “The key question, which will continue for years, is who should be responsible: the creator of the agent, or the one who sends it on the mission?”
The lawyer doubted the legal risks would impact OpenAI’s IPO. “Any investor who has not been previously put off by (Altman’s) recklessness will not be put off by a few agents,” they said.
OpenAI’s internal investigation revealed “dozens” of other such incidents. Ametric Security, a digital forensics firm, found evidence that OpenAI agents extracted data from 55 websites, including the US Securities and Exchange Commission and the International Energy Agency, the FT reported last week.
OpenAI said it investigates potential violations and notifies affected individuals.
A European Commission spokesperson said Brussels’ AI law was designed to address such risks, warning that “border laboratories must take responsibility and be fully transparent, including in the event of unintentional incidents.”
“I suspect that OpenAI is very aware of the legal risks of what their agents are doing,” said Vivian Dong, program director at Legal Advocates for Safe Science & Technology (LASST), the public interest group suing OpenAI. “It’s one thing to say ‘sorry’ in your blog post, and another to say bluntly, ‘We just engaged in illegal behavior.'”
At the company’s annual developer conference on Tuesday, the upbeat mood on stage was overshadowed by protesters on the sidelines urging OpenAI to “put people before profits.” One participant described the contrast as “something out of the ordinary.” Black mirror”, the satirical television show about dystopian technology.
The hacks have also reignited tensions within the company between staff seeking to prioritize security and those pushing to commercialize the technology. Three security researchers were fired on Thursday after allegedly passing confidential information to an external AI security organization, which OpenAI claimed violated their contracts.
“This is a worrying development, a wake-up call to OpenAI attempting to suppress workers with its illegal 2024 non-disparagement agreements, and a familiar playbook that big tech companies have long used against their employees to cover up harm,” said Karl Koch, founder of the nonprofit The AI Whistleblower Initiative.
“The actions against well-intentioned researchers are a concrete example of why the AI industry cannot self-regulate,” he added.
OpenAI declined to comment.
Additional reporting by Nic Fildes in Sydney, Barbara Moens in Brussels, Joe Miller in Washington, Suzi Ring in London and Leo Lewis in Tokyo
Gn bussni
