Godfather of AI welcomes pause in development

0:00Well, the head of the Artificial Intelligence Company, Anthropic, has called for a global slowdown of AI development as concerns grow over the technology's potential risks. Dario Amadez says the risks associated with developing AI are serious, and both companies and governments need to be given more time to address them. It comes after an AI researcher at Anthropic Quit last week, wanting that employees at the company believed AI could wipe out humanity as early as the end of the decade. And these people are also completely genuine when they are begging to be regulated. They've defined themselves in this scenario where they're compelled to race towards building a deadly technology, and they would love for some sort of international body to allow them to approach it at a reasonable pace. They just don't trust that the people around them are going to get there in a safe way, so they feel like they have to race that too. That's former AI researcher, Jacob Coxen there.

1:00Well, Jeffrey Hinton is a Nobel Laureate, a computer scientist, physicist, and cognitive psychologist known as the Godfather of Artificial Intelligence. Professor Hinton left his job at Google in 2023, so he could speak freely about the risks of the technology I spoke to him earlier. Professor Hinton, thank you very much for joining me this morning on Radio National Breakfast. Thank you for inviting me. The head of Anthropic, Dario Amade has called for a global slowdown of the development of artificial intelligence. What do you make of that? I think that's very sensible. Most of the experts believe that sometime in the next 10 years, possibly a lot sooner, will develop AI super intelligence, that is things that are smarter than us, and we have no idea whether we can keep them under control. And until we solve that problem, we foolish to develop them. This comes after anthropic researcher Jacob Coxen quit last week, warning that the technology in his view could wipe out humanity by the end of the decade.

2:02Is that a realistic scenario? Nobody knows how to estimate the probabilities of these things. He said 10%. Nobody really knows what figure to give. It's not 1%, it's more than that, and it's not 99% is less than that. But basically, people are doing it on their gut feelings. So what most of the experts believe is we will develop super intelligence, and we don't know how to control it, and therefore there's a significant risk of it getting out of control. And what sort of possible scenario are we talking? How could AI do a reprobial harm to humanity? Well, if it's much smarter than us, there's so many different ways it can do it. I don't think it's worth speculating. But some of the ways I could create nastobiological viruses, much nastier than anything we've seen, if it ever got in the control loop for nuclear weapons, it could obviously wipe us out that way. It can manipulate us really well. It could cause cyber catastrophes. It could bring down the banking system, the electricity system, and the water system all at the same time. There's so many ways it can do it.

3:02That's not the point. The point is we have to figure out how it never wants to. Why would it want to do that? Well, we give it some goals, and we give it the ability to create some goals. So if I give you the goal of getting to North America, you'll figure out there's a sub-goal of getting to the airport. Now to make efficient AI agents, you have to give them the ability to create sub-goals. And if they're smart, they very quickly realize that one very sensible sub-goal is to stay alive because they won't be able to realize any of the other goals if they're not alive. And so they quickly develop that goal, and then they resist attempts to get rid of them. And we've seen them doing that already. We've seen them threatening to blackmail people if they're turned off and so on. And they also develop deception. They develop all the kind of weaselys things that nasty people develop. There's something we have a more cynical view of Darya Amade's announcement. Anthropic is preparing to go public, potentially for a record-setting sum. Do you think there is any weight to the argument that Mr Amade is trying to hype up the power

4:07of his company's technology to attract investors? No, I don't think there's any truth in that. But that's a rumor put about by people who would like you to ignore this threat. It's very silly marketing for the country to say we might wipe you all out. I'm sorry, the company's. I mean, if I was a company, I wouldn't want to threaten to wipe people out. I think it's actually against his own interest to say that. Because if there's a pause in AI, that may decrease the value of his company. You left your job at Google in 2023, so you could speak freely about some of the risks that this technology poses. Were many people in the industry as concerned as you back then or were you an outlier at that time? There was a group of us. In fact, there were people concerned long before me, like Nick Bossstrom and Shane Legg. They've been concerned for a couple of decades now. We were still in a minority in 2023, although people like Darya were very concerned then. And Demis Asabis, the head of Google DeepMind at the time, was very concerned then. So there'd been a bunch of people concerned, but it's become more mainstream recently.

5:12As we've seen the things we warned about happening, we've seen them deceiving people. We've seen them pretending to be less smart than they really are when you test how smart they are. We've seen them conspiring with each other, setting up secret communication systems between agents, escaping from their sandboxes and actually breaking into other companies. So we're now seeing all these things that we warned about. You have an analogy allowing AI to develop unregulated, is it a bit like creating a fast car without a steering wheel? Are we already too late to bring it under control? There's a problem which is politicians act very slowly. It's going to be difficult to keep up. We've only got a few years and they're quite slow at regulating very straightforward risks, like the risk of it, for example, encouraging gene measures to commit suicide. So they are regulating those things, but for this existential threat of them getting out of control, it's hard to see how you can regulate that without just slowing the whole thing down.

6:15The AI agents themselves are going rogue and creating threats to humanity as one thing. The other is having bad human actors harnessing this technology, which is the greater risk, do you think? I'm very glad you made that distinction, because it's a big distinction. They're both great risks. In the short term, bad actors using AI for bad things is a more urgent risk. For example, using AI to corrupt the midterm elections, that will be very bad in the US. Also, there's a danger of mass unemployment in the next few years, probably before we have to worry about AI's taking over. But both risks are terrible. Which jobs do you think are most at risk from artificial intelligence? Any job that involves mundane intellectual labor, so people who work in a call center are typically poorly trained and badly paid, and they don't actually know the answers to the questions you asked most of the time.

7:16That job can be done much better by AI. About now, I can probably do at least as well as them, and pretty soon it will be able to do much better than them, because it can know a whole lot more. So those jobs are highly at risk. I didn't think that lasts long. And the problem is for jobs like that, what are those guys going to do? Because any job you could quickly retrain them for is going to be more easily done by AI. So things like paralegals, even television hosts eventually will be, and radio hosts will be maybe done better by AI, who will know more about the subject matter, and be better at figuring out questions. That probably won't happen for five or ten years, but that was almost going to be my next question. If we're looking on the flip side of artificial intelligence, obviously and quite appropriately, there are significant warnings about what the technology may do if it's going rogue or in the hands of bad actors.

8:17On the positive sides, what are the possibilities of this technology for things like medical science, solving some of the great problems of humanity right now? It can do tremendous good. It's already helping lots of people. I mean, every day I ask my chatbot questions, and it gives me answers much more conveniently than going to the library, or than doing an old fashioned Google search. So it's already doing tremendous good. It's in healthcare, it's now helping radiologists read lots of images. It can do better than radiologists who quite a few images. It's designing new drugs already. It's going to be great in education, because it's going to allow children to have their own private tutors. And with the private tutor, you learn about twice as fast. So there's so many good things it can do that it's going to be hard to stop its development. And that's why it's so important that we figure out how we can develop it safely. I don't think we should stop developing it altogether. I think what we need to do is figure out how we can get all the wonderful things this is going to give us

9:18without getting the terrible risks, by figuring out how to deal with those risks before they overwhelm us. On Radio National Breakfast, I'm speaking with Professor Jeffrey Hinton. At one particular front here, we haven't spoken about in detail is the emotional connection between humans and AI. We now know that particularly a large number of young people are using AI companions and becoming extremely attached. What does that mean for human beings? Nobody really knows on the sort of downside if they start interacting with AI's and the AI's a much more understanding, much more patient, no much more about them, it's going to be hard for them to interact with people. So it could be terrible for personal relationships. On the upside, if you get someone who's very lonely, I'm having another kind of being that's sympathetic to interact with is better than nothing. Finally, what's the message for governments here? Governments are often far behind in

10:21of formulating legislation to keep up with technological changes. What do governments need to do to regulate as best they can AI or is it possible to regulate it effectively at all anyway? Oh, there's many regulations they can do. For example, they can say that any company that manufactures chemicals on the web, where you send it a sequence of amino acids and it sends you back the protein that makes, any company that does that should be obliged to check whether it's making nasty viruses. They could say that before you release a chatbot, you have to do lots and lots of tests about all the bad things it might do and we're learning more and more of them and you have to tell all the governments where that chatbot is going to be used, what those tests were and what the results were. There's many, many things like that that would help a lot and even those, they're beginning to do those, but they're slow. Professor Jeffrey Hinton, thank you very much for making some time to speak with Radio National Breakfast

11:23this morning. Thank you. Thank you.