Emmett Shear (OpenAI Ex-Interim-CEO)'s New “Softmax” AI Alignment Plan — Is It Legit?

Emmett Shear is the cofounder and ex-CEO of Twitch, ex-interim-CEO of OpenAI, and a former Y Combinator partner. He recently announced Softmax, a new company researching a novel solution to AI alignment.

In his recent interview, Emmett explained “organic alignment”, drawing comparisons to biological systems and advocating for AI to be raised in a community-like setting with humans.

Let’s go through his talk, point by point, to see if Emmett’s alignment plan makes sense…

00:00 Episode Highlights

00:36 Introducing Softmax and its Founders

01:33 Research Collaborators and Ken Stanley's Influence

02:16 Softmax's Mission and Organic Alignment

03:13 Critique of Organic Alignment

05:29 Emmett’s Perspective on AI Alignment

14:36 Human Morality and Cognitive Submodules

38:25 Top-Down vs. Emergent Morality in AI

44:56 Raising AI to Grow Up with Humanity

48:43 Softmax's Incremental Approach to AI Alignment

52:22 Convergence vs. Divergence in AI Learning

55:49 Multi-Agent Reinforcement Learning

01:12:28 The Importance of Storytelling in AI Development

01:16:34 Living With AI As It Grows

01:20:19 Species Giving Birth to Species

01:23:23 The Plan for AI's Adolescence

01:26:53 Emmett's Views on Superintelligence

01:31:00 The Future of AI Alignment

01:35:10 Final Thoughts and Criticisms

01:44:07 Conclusion and Call to Action

Show Notes

Emmett Shear’s interview on BuzzRobot with Sophia Aryan (source material) — https://www.youtube.com/watch?v=_3m2cpZqvdw

BuzzRobot’s YouTube channel — https://www.youtube.com/@BuzzRobot

BuzzRobot’s Twitter — https://x.com/buZZrobot/

SoftMax’s website — https://softmax.com

My Doom Debate with Ken Stanley (Softmax advisor) — https://www.youtube.com/watch?v=GdthPZwU1Co

My Doom Debate with Gil Mark on whether aligning AIs in groups is a more solvable problem — https://www.youtube.com/watch?v=72LnKW_jae8

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Get full access to Doom Debates at lironshapira.substack.com/subscribe