Will AI superintelligence kill us all? (with Nate Soares)
About this episode
Read the full transcript here.
Are the existential risks posed by superhuman AI fundamentally different from prior technological threats such as nuclear weapons or pandemics? How do the inherent “alien drives” that emerge from AI training processes complicate our ability to control or align these systems? Can we truly predict the behavior of entities that are “grown” rather than “crafted,” and what does this mean for accountability? To what extent does the analogy between human evolutionary drives and AI training objectives illuminate potential failure modes? How should we conceptualize the difference between superficial helpfulness and deeply embedded, unintended AI motivations? What lessons can we draw from AI hallucinations and deceptive behaviors about the limits of current alignment techniques? How do we assess the danger that AI systems might actively seek to preserve and propagate themselves against human intervention? Is the “death sentence” scenario a realistic prediction or a worst-case thought experiment? How much uncertainty should we tolerate when the stakes involve potential human extinction?
Nate Soares is the President of the Machine Intelligence Research Institute and the co-author of the book If Anyone Builds It, Everyone Dies. He has been working in the field for over a decade, after previous experience at Microsoft and Google. Soares is the author of a large body of technical and semi-technical writing on AI alignment, including foundational work on value learning, decision theory, and power-seeking incentives in smarter-than-human AIs.
Links:
- Nate and Eliezer's recent book: If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All
- The Machine Intelligence Research Institute
Staff
- Spencer Greenberg — Host + Director
- Ryan Kessler — Producer + Technical Lead
- WeAmplify — Transcriptionists
- Igor Scaldini — Marketing Consultant
Music
Affiliates
Get every episode summarized
Each time Clearer Thinking with Spencer Greenberg publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
More from Clearer Thinking with Spencer Greenberg
Can We End Factory Farming Without Giving Up Meat? (with Bruce Friedrich)
Clearer Thinking with Spencer Greenberg
Our relationship to chatbots (with Claire Boine)
Clearer Thinking with Spencer Greenberg
The good advice you least want (with Cate Hall)
Clearer Thinking with Spencer Greenberg
Why housing costs so much and what can be done about it (with Alex Armlovich)
Clearer Thinking with Spencer Greenberg