AI expert warns of extinction risk within a decade

0:00Well, this week the world was hit by one of the most shocking warnings about the dangers of AI that we've seen so far. Jacob Coxon, a researcher at Anthropic and OpenAI, the company's behind Claude and ChatGPT, sensational resigned, posting, quote, "'Neither company is acting responsibly. They are racing straight to superintelligence and gambling with our lives.' He goes on, "'The people of building AI earnestly believe that it could kill us by the end of the decade.' He then appeared on American TV with more. If you extrapolate into the future, the level of capabilities of these AI's, with the same independent volition they could cause extreme havoc, for example, hacking critical infrastructure, building extension level bio-weapons. Well, for more on these dire warnings, we're joined now by Jeffrey Hinton, no-but-what prize winner, computer scientist, and the Godfather of AI. Good morning to you, Jeffrey.

1:03Good morning. We've all seen the headlines this week in your estimation. How scared should we be? I think we should be very scared. We should take this very seriously. Nearly all AI researchers believe that if we don't kill ourselves first, we will develop superintelligent AI, and nobody knows how to keep it under control. And with the attack on Hagenface recently, we saw that these agents can go rogue very easily. Okay, let's look at what you think the probability is, because also noticed this week, another staffer at Anthropics said that there was a greater than 10% chance of AI wiping out all humans within a decade. And I've seen it as high as some experts saying 25%. Is that possible? Well, it's very hard to estimate probabilities for things like this, because we've never been here before. We don't have any data to base it on. So what you're really going on is the gut instincts of the researchers. And because nobody can figure out a sure way of keeping these agents under control,

2:09I think it's not at all unreasonable to say there's at least a 10% chance they might wipe us out. So, I mean, how do we respond to that? How do governments respond to that? Because Jeffrey, it would seem from the outside that these companies open AI and other companies that are working on this type of technology. They are entirely ungovernable at the moment. They don't listen to the state run governments of the nations that they're really working in. What do we do to protect ourselves? Well, one thing we can do is get the public aware. So on climate change, not much happened until the public was aware that the big-end gerry companies were polluting the environment. So that's one thing we can do. But there is some good news, which is on the issue where these things wipe people out. All the different countries who will be able to collaborate. No country wants to be people to be wiped out by AI. So if the Chinese figured out how to make them so they didn't want to wipe us out, they would immediately tell the Americans. Just as in the height of the Cold War, the Russians and

3:12the Soviet Union and the Americans could collaborate to prevent a global nuclear war. What are the fears about how AI would do this? Well, my view is if it's much smarter than us, there's so many different ways it can do it. It's not really well speculating. If we ever put it in the controlled loop of nuclear weapons, it's obvious how it could do it. But it can design biological viruses. It could cause devastating cyber attacks. So there's many, many ways it could do it. And the more information that we give it, the more power it has. Is that how it works? Well, as we make it larger and train it on more data, it gets more and more intelligent. Already, if you use a chatbot, you've seen they're very smart. They can answer more or less any question you give them and they can do quite good reasoning. They're just going to get better and they're going to get better quite fast. So once they do acquire that knowledge, there's no backtracking from that position then. Is there Jeffrey? It's very hard, particularly if you've

4:17released agents. It's very hard to go back and say no, we shouldn't release any AI agents. We shouldn't allow them to do things by themselves on the web. I think that ship has sailed. So we've now got to figure out, given that people aren't going to release our agents, how can we ensure that they won't conspire with each other to do bad things? With the hugging face attack, we saw them. They spontaneously figured out how to communicate with each other by using a message board. And then they conspired with each other to try and cheat on the test they'd been given. It seems there is a huge amount of money and effort being put in by these companies for how to develop them and be the first, rather than them actually being for the greater good. Interesting that you're saying, there, Jeffrey, that we really need to, it's people power that need to pressure our governments. What are we pressuring the governments to do? What sort of guardrails can governments put on these companies? So there's many different risks from AI. There's risks to do with people misusing AI, like in fake videos to corrupt elections or in making nice

5:23biological weapons with them. And then there's risks of AI itself getting out of control. For all of those risks, there's different solutions. So for example, it's very clear now that before you release a large sample, you should have to do a lot of tests, then you'd have to tell the government what the results of those tests were. Then you should probably have to tell the government in every country whether AI is used. Just very lastly, very quickly as well, Jeffrey, before we do leave you and thanks for your time this morning. Are you positive about the future of AI or is this all pretty dumb and gloom now? No, I think we're to point where there's two possible futures. One possible future is going to be great. It's going to help us cure cancer. It's going to deal with climate change. It's going to make all interests more productive. It's going to be great in education, all aspects of healthcare. It's going to design new drugs. And because of all those wonderful things it's going to be able to do, it's going to be very hard to stop people developing it. But then on the other side, there's all these risks. Now, if we can put enough effort

6:24in and figure out how to contain all these risks, then it's going to be great. If we can't, we've got the other future, which is very bleak. And perhaps ironically, it's up to us as humans to try and stop it and put those guardrails around it. Jeffrey, into Nobel Prize winner, computer scientist and known as the Godfather of AI, we really appreciate your time this morning to speak to Australia. Thank you so much. Thank you for inviting me.