It’s hard to believe I’m writing this.
I absentmindedly read the same things as you on The acceleration of AI and the very significant risks that result from it. But this week in Toronto, two of the world’s leading AI experts made the issue more real and more urgent.
The speaker of the second edition Hinton Lecturesorganized by Geoffrey Hinton of the University of Toronto, was Paul Christiano from San Francisco, who has just joined the board of directors of the OpenAI Foundation.
Might as well start like Christiano. He presented his argument over three consecutive evenings at the Metro Toronto Convention Centre. I’ll extract the most important statements from each evening into indented quotes. There will be many.
Here’s how the first conference, Monday evening, began:
“I’ll start with the most important sentence. I think there is a high risk of AI takeover in the next 3 years.
“What I mean by AI takeover is a situation where AI systems are pursuing goals, and those systems are taking the lead – in the same sense that humans today are giving orders and what’s happening on Earth – and where, say, you or I or Donald Trump wanted to stop them, they would be as ineffective as if you or I were trying to stop the US military.
“I think it’s a terrible situation. I think it’s reasonably likely that a lot of people would die during this transition. Or that a lot of people would die soon after if we were unable to maintain a habitable environment. I think even if we don’t die, I think a takeover is sort of the end of human sovereignty and self-determination. And that’s something we should work incredibly hard to avoid.”
What is high risk? Hard to say, says Christiano. But if AI development continues as quickly as possible, the takeover he described will be “something of a coin toss” over the next three years. He hopes the figure will be lower, as he hopes governments will slow down AI development. But even then, “the risk remains exceptionally high.”
“I think for most of the people in this room, the most likely reason they will die is because of an AI accident.”
And it will take three years? Maybe. Maybe less.
“I think there is significant risk in the next six to 12 months. The trajectory on the ground has been extremely alarming over the last three to six months for people who have been following closely. I think we’ve seen pretty much the things we might expect shortly before something really bad happens.
“So I say ‘within the next three years,’ and indeed a lot of the risk is more distant. But I think there is also a significant risk in the immediate future. It’s not something we should treat as an abstract discussion. It’s something we should try to respond to immediately.”
Gn bussni
