NEW YORK (AP) — Artificial intelligence experts warned New York City leaders Monday about the dangers of the emerging technology, calling for greater oversight as local governments work to address growing concerns that systems could put humanity at risk.
At a New York City Council hearing, recently departed employees of Anthropic, OpenAI and Google DeepMind described a reckless culture within those companies, saying the biggest tech companies are rushing to create technology they don’t understand how to control.
“Companies operate on a startup mindset: move fast, break things, fix them later. That works for a photo-sharing app. It doesn’t work for creating the most powerful technology ever built,” said Jacob Coxon, former engineer at Anthropic who has become a leading voice on regulation.
State and city governments are considering ways to regulate AI as the Trump administration has taken a hands-off approach and Congress has been inactive. President Donald Trump called AI fears a “hoax” and signed an agreement late last month that laid out a series of voluntary steps companies must take and, in large part, allows them to self-monitor.
Industry call for slowdown followed Coxon’s warnings
Coxon was a little-known engineer at Anthropic until he announced his departure through a series of social media posts that gained worldwide attention. He told the Council that AI could have huge benefits, but without transparency and safeguards it is more likely that things will spiral out of control.
“On the current path, I think it’s more likely than not that humanity will lose control of these AIs and that could result in human extinction,” Coxon said.
Days after Coxon resigned and expressed fears of worst-case scenarios, senior executives including Anthropic’s Dario AmodeiSam Altman of OpenAI and Elon Musk of SpaceXAI said they agree that AI development needs to slow down. AI companies have detailed the steps they’ve taken to prioritize security, but a coordinated slowdown the sector faces several obstacles.
A panel of AI company staffers — representing tech giants OpenAI, Anthropic, Google and Meta — also testified at the Manhattan hearing Monday, telling officials their systems are already improving people’s daily lives and emphasizing their focus on security.
Anthropic and OpenAI said their advanced AI products it is necessary to regulateand business representatives present at the hearing said they were open to some of the Council’s suggestions.
“I share your commitment to ensuring that technology is developed safely and its benefits are widely shared,” said OpenAI’s Morgan Dwyer.
New York council members urge AI companies to quantify disaster risk
At one point during the hearing, City Council President Julie Menin asked each of the company’s representatives to “quantify the risk AI poses in a worst-case catastrophic scenario.”
Dwyer responded, “I don’t know. I also don’t think it matters whether there’s a 1%, 10%, or 20% chance of something catastrophic happening. Neither level is acceptable. We shouldn’t train models that we can’t make extremely strong arguments that we can keep under human control.”
Menin called the response “flippant at best.”
“I would say the chance percentage doesn’t matter. What matters is the commitment to security, and that’s what OpenAI is committed to doing,” Dwyer said.
The City Council is considering several AI regulations, including proposals to require companies to disclose information about AI tools, require third-party approval for AI systems in the city, and allow whistleblowers to receive a portion of fines from AI companies that violate laws.
In addition, New York Governor Kathy Hochul, a Democrat, has decided to impose reporting and transparency. standards around AI, as governors have done elsewherealthough their power is somewhat limited since AI technology can cross state borders as well as the entire world.
Anthropic’s Logan Graham told the Council that the company supports “smart regulation” and that state and local governments have a role to play in developing frameworks.
“AI is powerful enough that a mistake has real consequences, on individuals, institutions and public trust,” he said. “We must work tirelessly to mitigate risks. »
Gn bussni
