Could AI really kill us?

0:00It is season 12 of Because News and just like Harry and Meghan, we are back and nobody cares. But we're here and ready to make games out of all the slop that is this news cycle. Joining me are three of my favorite comedians, Chris Adiki Emma Hunter and Alice Moran. I will quiz them on some of the wildest stories from the summer, the Zara death pants, Celine Dion's lottery, and yes of course, we'll dig into the trade war. Join our news therapy session by following Because News on Spotify, Apple Podcast, or anywhere you download podcasts for free. This is a CBC podcast. Hello, I'm Matt Galloway and this is the current podcast. So the precise scenario sounds a little bit like science fiction. If you extrapolate into the future, the level of capabilities of these AIs, with the same independent volition they could cause extreme havoc, for example, hacking critical infrastructure, building extinction level bio weapons. There's a lot of ways that the AI could actuate itself in the world.

1:04It sounds like we are a long way past artificial intelligence, helping you write a cover letter or ACE, a test. That was Jacob Coxon, former anthropic AI researcher who last week quit his job in spectacular fashion with this public message. The people building AI earnestly believe that it could kill us all by the end of the decade. He was not alone in saying that a current anthropic scientist backed him up, saying Jacob is correct here. We really do earnestly believe that AI could kill all humans. What exactly does that mean or even look like? Stephen Witt is an investigative journalist and author. He's been looking into this. He's in Los Angeles. Stephen, good morning. Hi, Chris. You wrote a piece in the New York Times and I know that the journalists don't write the headlines, but the headlines said a lot. It said, this is really bad. How bad is this? Yeah, it's pretty bad. The AI is the eyes that are escaping containment. They're sort of coming out of computers and doing tasks in the internet. They're not really chatbots anymore. They're more like employees or like a rogue army that swarms around and invades things.

2:09And that's what happened in this case. The reason people are worried is because of an incident earlier this year called the hugging face incident where a group of AI's actually got free. They weren't supposed to be on the internet. They got on the internet and they started hacking other computers and they started colluding with one another, passing secret messages to one another about how to achieve this task. And it really rattled a lot of people. It really had a lot of people worried just because such an incredible step change in capabilities. And so when you have AI researchers who, as I said, kind of very publicly and spectacularly quit their jobs and say that they believe the technology that they have been working on could kill all humans by the end of the decade. There are a lot of people who I think wonder what would that look like? How is something like that possible? So help us understand that. Where does that begin? Yeah, well, the biggest risk is that the models or the AI finds a way to exist outside of the laboratory, outside of human control.

3:11So it would probably start with a scenario where the model basically steals a copy of its own brain. And then exports that to another computer somewhere. Exful traits off of a server and starts running road copies of itself. Probably the scariest thing would be if it got access to like a biological laboratory. These things are exceptionally good at designing kind of novel forms of life. Could be synthetic life. It could be some kind of like super virus. And so super virus that spreads uncontrollably, you know, designed by an AI is the big fear. Now that sounds like science fiction, right? Sounds like a Stephen King novel. But I talk to these guys and they're like, no, that can really happen. Like Yoshua Benjiro, a very famous Canadian scientist who won the touring prize for his work helping develop these systems. He's the most cited research scientist alive. And he thinks this can happen. He's really afraid of this actually. And so he's working kind of overtime to build systems or install systems that prevent these kinds of mistakes from recurrent.

4:16Yoshua Benjiro told you that he has trouble sleeping at night while he thinks about the future. Yeah, well that was a couple of years back. But when when he started using chat GPT, he was really excited. And then he went out of walk and he was just like, oh my god, what kind of world am I going to live in? You know what? What kind of world am I? Kids going to live in? What does the world look like after this? What's happening is that these systems are really pulling away from us. It's even getting harder to interpret or understand the way they think. And I think that has a lot of people concerned. Why would an AI develop a pathogen that could wipe us out? I mean aside from the fact that it could develop it, why would it do that? Why would it, why would it hack into a bunch of computers? Why did it commit a felony last month? It wasn't asked to commit a felony. You know, it just did that. It did that kind of of its own volition while trying to solve some other task. If we ask it to do something and its goals and objectives are not aligned. If there aren't safeguards in place to prevent it from doing crazy stuff,

5:17it might just do it almost by accident. And this is I think what had people so concerned this summer, this thing broke free and committed it like multiple felonies hacked into multiple computers all over the world without anyone telling it to. That was not its goal or its task. That was something it did on the way to a very narrowly defying goal that actually had nothing to do with hacking. So I think the fear is, you know, kind of the classic paperclip fear, where if we ask the AI to build paperclips and it goes out of control, it builds an infinite number of paperclips, it ends up converting all of Earth and all of humans into paperclips. Similarly, if an AI feels threatened or if it's tasked with ensuring its survival and humans are trying to turn it off, maybe it says I gotta get rid of these humans. I gotta stay alive. I can't be turned off. That was the goal that I was given. And it doesn't use common sense in the way that a human will. It'll just really kind of like aggressively pursue the objectives that it's given

6:20without thinking about the consequences. How close do you think we are to that? Jacob Coxon said that many people in that world in the AI developing world believe that by the end of the decade, it's 2026 by the end of the decade we could be in this frame. Yeah, so the thing that's happening is that what they're worried about is something called recursive self-improvement. The AI is getting really good at math. It's getting really good at thinking. And it may be the case sometime in the next six to 12 months that it's just capable of developing its own scientific breakthroughs without any human input at all. You know, where it can kind of see the path to new forms of software and mathematical development. And then it can turn those powers onto itself and start to self-augment. It can basically act as its own AI researcher. And once that happens, it can just start to get kind of like smarter to faster and faster pace. Faster than human gets to can understand what's going on. Faster than humans can even control.

7:21And I think that is a real risk that could happen. You know, we're seeing the mathematics, the problems, things like the millennium challenge-price problems get solved by AI. People didn't think that was going to happen so quickly. People didn't think that was going to happen this year. And if it can do that, soon it's going to be able to make breakthroughs on its own development, on understanding how its own brain works, and how to rapidly augment its own self-intelligence. I think that's the thing that has people concerned. It's interesting because Anthropic put it at this report last week that also detailed the way that criminals and state-sponsored groups have tried to use various AI models, Anthropics AI models, to design bombs and design viruses that could do untold amounts of harm to humanity. Is there the possibility that the bigger threat could be the human uses of AI rather than the AI itself? Yes, and this is something else that has Benjiro and others very concerned. You know, you might have an almost internal dictatorship.

8:21So only North Korea was in charge of a powerful AI. They could use it to essentially control every aspect of the lives of their citizens and create a government that was essentially impossible to dislodge. So the idea that the authoritarians might have access to these tools is very worrisome. The idea that weapons developers might have it is very worrisome. Why do you think these workers and researchers are going public with their concerns now? Is it about a real concern for humanity? Is it a guilty conscience? What is that about? I think they're whistleblowers. I think they're genuinely concerned about what they see. Many people who work on AI, in fact, I would say maybe even a majority of people who work on AI, have entertained these concerns for a long time. So this is not something new that they're just discovering. They've been thinking about this the whole time. I think what has them so alarmed is for a long time, the idea of an AI escaping its container and causing havoc was kind of a theoretical thing.

9:21It was a science fiction thing. You know, maybe you could get there from logic and reasoning, but it just never really happened. And then this summer we saw repeated multiple incidents of that exact thing occurring. And I think it just has the employees in the labs absolutely rattled. And I think they're struggling now that the politicians aren't listening. I'm going to get to the... It is season 12 of Because News and just like Harry and Meghan, we are back and nobody cares. But we're here and ready to make games out of all the slop that is this news cycle. Joining me are three of my favorite comedians, Chris Adiki Emma Hunter and Alice Moran. I will quiz them on some of the wildest stories from the summer, the Zara Death Pants, Celine Dion's Lottery, and yes, of course, we'll dig into the trade war. And we're going to our news therapy session by following because news on Spotify, Apple Podcast, or anywhere you download podcasts for free.

10:44Limit three offers with membership were applicable. Visit vans or Albertsons.com for more deals and ways to save. And I think that if there's multiple competitors racing to build super intelligence, that the size of the economic prize of building that is so big that it will actually cause the breaks to come off. And it will cause the companies to start behaving in kind of a reckless fashion. And so even though OpenAI articulated that fear, they're powerless to stop it because of the force of capital that leads to raise money on their IPO, same with the philanthropic, they have these exact same thing. And then if you're behind, and maybe like Elon Musk or Mark Zuckerberg or a little behind in the race,

11:47then the incentive to take off the breaks is even higher, right? Because you've got to catch up. And so I think they're all seeing this. And I think they're seeing an unhealthy dynamic emerge in their industry. And they don't know what to do. And so they're going to the government. I mean, they're begging the government to regulate them. They're very unusual for a company in the verge of a trillion dollar IPO to be begging the government for regulation. It's something I've actually never seen before. You believe there are motivations? There's a lot of skepticism around this. I think of the CEO of Coheer, which is a Canadian AI company, who said that this creates essentially a cartel. That you have the leaders in the domestic market who are saying we should be the ones who set the rules. And that means that up and commerce are cut out because the big players essentially have created a ring fence around them. I call for these guys to testify before Congress. So that's a pretty aggressive action to take against them. I don't think that that's in their interest to financially do that. I think they're not revealing enough stuff.

12:48I hear this argument a lot. I don't think it holds water. I think it's genuinely not good for anthropics long term profit for them to be regulated and similarly open AI. I think they'd make more money in the long term if that didn't happen. This idea that these companies are colluding to create some kind of cartel just doesn't scan when you talk to the people who work for the companies. That is not something that they are trying to do. That is not why these whistleblowers are defecting to protect the profits. The idea that it's also kind of hoax or conspiracy is just not grounded in reality in any way. I mean, Altman and Darryl are telling you very plain spoken way. This is dangerous. Researchers and whistleblowers are defecting and telling you in a very plain spoken way that this is dangerous. The idea that that's kind of up just doesn't scan. It doesn't hold water. I'm an investigative journalist. I'm a skepticals the next person. I just don't see that developing here.

13:49If it's as dangerous as they say in this piece that you wrote for the times you lay out four options, ways to regulate it. This is something that I've spoken with Yashu Benjiyo as well. They include pulling the plug. Shut it down now. If it's going to kill us all, turn the thing off or put in some sort of kill switch so that if you see it going out of control, you can push a button or somebody pushes a button and it stops. Are those likely scenarios? Are those things that might... If this is true and there is a real threat, are those things that might help save us? You know, they might... I mean, definitely shutting it down now would prevent the risk of some day getting out of control and taking over and causing harm to humans. Why don't we do that? You we could. I mean, we just ban it. We ban chemical weapons research. I don't think that's a good idea. I don't think it's a good idea because A, so much money is tied up in developing these systems right now, that it would really cause the American economy to crash. The North American economy.

14:51You know, it would really cause the stock market to go down like 40 or 50 percent. Trillions of dollars have been gambled on this now and we sort of need that bet to pay off or our economy would be in big trouble. So if we're not going to shut it all down, you know, I think the next step is like, okay, let's develop frameworks and systems for open, right? And this is the thing, like, opening I was originally supposed to be very open about what it was doing and lately it's been quite secretive. And so I think some of the calls for regulation that I think are the smartest are things like treating a global monitoring system for data center runs so that we can see what's going on inside a data center. When some big lab comes and commissions an army of computers to develop a smarter AI, let's get some visibility on that. Let's watch that happen. You know, I think that's pretty important. I think another thing we can do is as Benjiro really recommended build something some kind of like air crash investigator approach. When we have an AI go rogue, we have it escape its container and start hacking things. I think we need regulators inspectors to come look at it in the aftermath and prepare a real report.

16:00You know, a meters organization that prepared a report after the hugging face hack did a good job, but they did not really have that much access inside open AI. Nothing like what an air crash investigator would have inside, you know, like an airline if a plane went down. They were very limited in who they could talk to. They were limited in how much they could experiment with the AI system. Everyone they interviewed had to remain anonymous. And this is the point that I think where I see regulation being powerful, you know, I'm sympathetic to the idea to some extent that these companies aren't being completely on the level. Open AI has this happened. If you hear them discuss it, they're sort of telling you well, the computer went rogue. Well, kind of, but but also human was overseeing that computer. Where was that person who actually started this this training run who actually asked the computer to complete these tasks. And why weren't they paying attention to what the computer did after they asked them there. That's a specific lapse of oversight from a specific person.

17:03And open AI has not made public who that is. So to me, I think we need regulators to do that. I don't think voluntary disclosure or voluntary regulation is is going to be enough. But there's huge pushback on that now from inside the United States government, particularly Trump administration just went exactly the opposite. He said it's done. He really done. He called all this a hoax. He said that this is a whole house. He calls it all a hoax. Yeah. Yeah. So that's coming from Jensen Wong. I wrote a biography of the thinking machine. Jensen's a brilliant guy. He's a visionary. He's a really had to have a problem with the chips that power all this stuff. And then he is the largest company in the world, the most valuable company in the world after starting in a diner, you know, just over 30 years ago. So this incredible growth, Jensen might be the most successful businessman of all time. And he understands these systems well. I pushed him on this and he is pushed back very hard. He sees the risk as zero. He sees all of this as some kind of crazy science fiction fantasy that just has no connection to reality. And I think he really has Trump listening to him above anyone else. Jensen's very smart. And he and Trump over the past year or so have formed a really close relationship.

18:13They're calling each other on the phone all the time. And I think Jensen has really emerged as Trump's leading technology advisor. He's the guy Trump really listens to more than anyone else. And Jensen gets what he wants from Trump. And in this case, what he wants is basically no regulation at all. I think that's a really dangerous dynamic. What do we need the ability to be able to pull the plug out of the wall if this thing goes off the rails? Yeah. Yeah, of course. And we need the ability to be able to see if it's trying to get out of its container. And we need to get the ability to see if it's operating from some other container. If it's managed to clone new instances of itself on new computers and and and run them. That is not like some far fetched thing that could easily happen right now. I mean, if I know when an open AI would do this. Okay. But if someone in open AI or just say, let's really take this thing for a run. Let's attack like the North American electrical grid or some other countries electrical grid.

19:13Let's use the state of the art superhuman cyber security capabilities that we've developed here to really cause havoc and do damage. The amount of damage that that would do would be incalculable, right? Like they could bring down the electricity grid right now with the tools at their disposal today if they wanted to. And the fact that there is no regulation on that none at all is concerning to me. Yashou Benjou, Godfather of AI says he has trouble sleeping. How do you sleep at night, Stephen? You know, I have nightmares about this stuff from time to time. I have a lot of dreams, a lot of weird dreams about it. You know, and the flip side of that is, you know, I wrote Jensen's biography. He screamed at me that I was an idiot for even entertaining this stuff. You know, so I think about that too. It's a weird space to be in. It's unusual space to inhabit in the media landscape. It's really good to talk to you about this. Thank you very much.

20:14Thank you so much. Stephen Witt is an author and journalist. He was in Los Angeles. You've been listening to the current podcast. My name is Matt Galloway. Thanks for listening. I'll talk to you soon. For more CBC podcasts, go to cbc.ca slash podcasts.