Microsoft Chairman and CEO Satya Nadella on Saturday called for building advanced artificial intelligence systems with containment, independent controls and an “emergency brake” that allows authorized people to pause or stop a model mid-task.
“We need to surround non-deterministic models with strong, deterministic system design, human controls and reliable operational procedures, and establish industry standards where existing ones fall short,” Nadella wrote in a post on the social media platform X.
“Treating closed and open border weighting models as internal risks is one way to build such a system,” he said.
Nadella’s comments come amid warnings from top tech executives and researchers, including Microsoft co-founder Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and EspaceX CEO Elon Musk — on insufficient AI security protocols and says the technology is evolving too quickly.
An AI researcher left Anthropic last month and accused the company and its main rival, OpenAI, of “gaming with our lives.” Later in the day, an Anthropic official focused on AI safety said there was more than a 10% chance the technology could “kill all humans” in the next decade.
Conversely, President Donald Trump has repeatedly dismissed AI’s risks of extinction and instead emphasized that the industry must stay ahead of China. Trump also recently introduced a new “AI Force,” led by Director of National Intelligence Jay Clayton, to help facilitate the industry and weed out bad actors.
Microsoft’s Nadella wrote Saturday that AI systems should be designed around “observability principles.”
They include model diversity; a human-readable imprint of the model’s actions; continuous system testing; independent controls and auditability; containment; and disclosure of the incident, Nadella wrote.
“The most reliable Super Intelligence system will not be the one with the model we trust the most,” he wrote. “It will be the one that allows us to trust the model the least.”
Gn bussni
