You Don't Need More Freedom. You Need Better Constraints | David Epstein - EP 818

0:00What if the thing holding you back isn't a lack of opportunity, but too much freedom? We tend to believe that more options give us more freedom. More possibilities, more paths to explore, more chances to get it right. But when I sat down with New York Times bestselling author David Epstein, he challenged that idea. He explained why some of the most important breakthroughs don't happen because we have unlimited freedom, they happen because we have constraints. Take a listen. I actually think the gap between the mentally of myth and reality is symbolic of something important, which is that we overvalue the complete freedom of something like the dreaming brain and undervalue the power of constraints to channel us into productive experimentation. I think most breakthroughs do with a much smaller problem that opened into something that changed the world. Hey, it's your friend John and welcome to PassionStrike.

1:00I'm so glad you're here. Whether you're a longtime listener or someone who just discovered the show, thank you for pressing play today. Maybe you've built the life you were supposed to want, but you've quietly wondered why it still doesn't feel like enough. If that's you, you're in the right place. Every week I sit down with the world's top scientists, researchers and thinkers to uncover what it actually takes to build an intentional life that matters. Thanks for joining me. Let's get into it. Welcome back to PassionStrike. I am so glad you're here. This is Episode 818 and today we're continuing our series, The Adaptive Human, How We Thrive in a Changing World. For the past few episodes, we've been exploring some big questions about what it means to shape our lives in a world that's constantly changing.

2:01Angela Duckworth challenged us to look at the situation shaping our behavior. Pulitzer Prize winning author Jared Diamond asked when individual leaders can actually change outcomes. And in our last episode earlier this week, Marty Slygman explored something even more fundamental. Do we believe that our actions can actually make a difference? But once we decide to act, another question emerges. Where do we begin? Because most of us have been taught to think that more freedom is always better. More options. More possibilities. More paths to explore. But what if too many possibilities can sometimes keep us from moving forward? That's what makes today's conversation with David Epstein so fascinating. David is the New York Times bestselling author of Range and the Sports Gene. And his work has challenged some of our most deeply held assumptions about expertise,

3:03success, and how people adapt to an increasingly complex world. His new book Inside the Box takes that conversation in a surprising direction. It's about why constraints can sometimes help us see possibilities we would otherwise miss. Why breakthroughs often begin with smaller problems than we expect, and why adaptation isn't always about having more freedom. Sometimes it's learning how to work productively with the reality that's already in front of us. In this conversation, David and I explore why the story we've been told about some of history's greatest breakthroughs isn't always true. While constraints can channel us into productive experimentation, why broad experience still matters, and what happens when we're constantly searching for the next horizon instead of learning how to engage with where we are. But here's what stayed with me long after we finished recording. David reminded me that we often imagine transformation as something dramatic, a new beginning, a bold

4:06leap, a completely different future. But many of the most meaningful changes in life don't begin that way. They begin with something much smaller. Our problem right in front of us, a constraint we didn't choose, a question we can't ignore, and sometimes that small opening becomes the doorway to something much larger. I think that's an idea all of us need to hear. So let's dive in. Here's my conversation with David Epstein. Thank you for choosing PassionStruck and choosing me to be your hosting guide on your journey to creating an intentional life that matters. Now let that journey begin. I am so honored and ecstatic today to have David Epstein on PassionStruck. David, so great to see you. Thank you for having me. Glad to be here. David, I loved your book range as did millions of others. Thank you. You argued that broad experiences prepare us for an unpredictable world.

5:11Your new book inside the box feels like the next chapter of that conversation. Is this really a book about how we move from possibility to progress? That is a great way to put it. And I'm glad you saw it that way because that's exactly how I was thinking about it. In fact, inside the box was in some ways responsive to the most common reader questions I would get after range, which were along the lines of, okay, I've got these broad experiences. I'm curious about so many things, but I have trouble channeling that energy into something, into a project, deciding what to actually do. And I counted myself in that boat absolutely. There's a reason that there's been so many years between my books, which is that I would really have trouble drawing boundaries around a project and diving in. So it was partly motivated by, they say, me search motivated by things I wanted to get better at, but absolutely that moving from building the tool set and the adaptability to how do you focus and channel that into it into achievement and fulfillment.

6:14I have to ask you this question. I was interviewing Angela Duckworth a couple of weeks ago. Angela is a personal friend. I asked her because it's been a number of years since grit came out. How do you approach a book when you've set the bar so high with the first one that you wrote? And she said it was really difficult that she questioned herself for a very long time about what she should explore in this new book. And it became a much harder book to write as a result of that. Did you find anything similar for you? Yeah, Angela and I actually talked about that. Well, we were both in the process, some. And yes, and I think part of my strategy for overcoming that. And by the way, when I used to be running buddies with Malcolm Gladwell, I once asked him this question where I said, are you ever concerned about competing with your past self? And he said, oh no, I don't have any of your Jewish anxiety.

7:18What did he say? He said, I have a, because he's part British and part Jamaican. He said, I have a Brit stiff upper lip and a Jamaican jua de vivre. So he just was like, that's not an issue for me. But for me, and Angela, obviously it is. And I think after my first book, which was a surprise bestseller, I was a little paralyzed after that. And what ultimately I think helped was one just getting what also just trying to do something a little different. So after the sports gene, which was my first bestseller, there was a lot of pressure to do the sports gene too. And I didn't even know what that meant. And to do it fast, right? This made me realize how bands can break out and then have a terrible sophomore album, because it's like, somehow people think they just figured out how to do what they do 10 times faster. And so I think what was helpful for me was to go in another direction, basically. So my first book was very grounded in physical science. And then range was in the social sciences. And so it really felt like something different to me. And there was enough of a tweak that I didn't feel like I was just competing with exactly what I'd done before.

8:19And then within the box, I think what really helped me get over that mental hump was an experiment where I said, look, I need to be more efficient in my writing projects. So I forced myself to outline the book on one page and one page only from a 100,000 word, note sheet. Actually, this I took 100,000 words down to this one page outline. You can see I wrote as small as humanly possible, but it's like bubbles and arrows all over the place. But it forced me to ruthlessly prioritize and it forced me to come up with this structure where I have a repetitive story that keeps coming up in the text and I have linked chapters. And that structural experiment, I found so engaging that it took away some of the anxiety of, can I do the same thing again, but better, and got me so focused on just this engaging experiment where I was trying something new with my writing. And I just lost that self-consciousness, as well as engaging the experiment. And I love that whole explanation and especially your one page. As I was writing my upcoming book,

9:19I was fortunate enough to get an hour-long-ish mentoring session from Dan Heath, because I was really struggling. I was banging my head against the wall about how to approach the book and he gave me the same advice that you just went through. He said, you want to keep coming back to the central hypothesis that you're making in the book. And you want to keep asking questions that allow you to do the research that allows you to keep coming back to the answer of that for the audience. And it was really a profound way to approach the book because some people may say it's repetitive when they read it, but I was trying to really come at it from every different angle that I could possibly think of to answer it for the reader. And as I was reading your book, it seemed like you were trying to do the same thing through your research, as well. Absolutely. And by the way, my publisher told me never to use that word, me search. So I'm glad you're okay with it. They said, it'll make it sound like the book is just

10:23for you and not for other people. And I'm like, well, I think I'm going to have to hope that other people struggle with the same things that I do. But coming back to it, the main idea from different angles, I think is really powerful. One is a way for us writers, we writers to structure a book, but also because of the way the human brain works, right? We know that what makes ideas sticky is when you embed them in what's called your semantic network, which is like if you picture a spider web of ideas in your head, the more connections there are between one idea, the more threads that are attached to it. The stickier it is because when one idea vibrates a thread, it's attached to more other things that then get activated. And so I think when you have a central idea and you commit it from all these different angles, you're basically building the semantic network that makes the idea memorable and makes it adaptable to all these different contexts. So I have to say, this is maybe a tangent here. I'm a digressive person if you haven't noticed already. When I first started playing with large language models, like chat GPT, when it first came out, I'm like,

11:27this is incredible. I'm opening up my old college physics textbook. I'm like, I'm going to relearn all this stuff. And I got the feeling of learning in this fast back and forth, but then I realized I really wasn't retaining well. And this is now what's showing up in some of this early research on how AI effects cognition, which is it's like too fast that you move through things. And so it's not sticky, which you really want to stick to well and approach the same idea from these multiple different angles. And so this you can get the feeling of learning. But if something is too fast and the idea is flying by, it's not actually going to be both sticky and embedded in a sort of flexible network in your brain. Interesting what you just said. And I'm going to take my own side path here that I wasn't expecting to, but I interviewed Marty Selegman a couple weeks ago. And David, you're going to love this. He is spending five to eight hours a day loading thousands and thousands of research papers across millennia into Claude. And he told me that in the past six

12:33months, he has made more progress on his thinking than he did in the past six decades. And it's proving some of his theories correct and others incorrect. And it just was amazing to me to talk to the father positive psychology and hear him using these language models to validate or invalidate his own research and how much letting go of the ego that has to be in order to do it. That's fascinating. And it kind of mirrors the best use of head for AI in some ways, which is creating an operation where I'll upload a scientific paper I'm interested in and then made a system where I'll get a map of all the other papers that cite it with a bubble showing how cited those papers are and attempting to sort into confirming or refuting. So I'm not doing it on my own research, like Marty is, but taking these papers that I'm interested in or that I've relied on and trying to put them into this huge network and see how do they really hold up in the larger body of research.

13:38Yeah, it is interesting. Sometimes the responses that you get back, isn't it? For sure, because some I find that they just want to flatter you, but sometimes you get these really 180 degree different outputs that you ever thought you were going to get. And those are the ones I really enjoy the most because it makes me want to double down on what they're saying to prove it either right or wrong. Totally. It's very interesting to have a tool that still is often wrong and yet is very useful. So I think it's really helpful if people are being a little bit of an anthropologist about it and really going in there and messing around because you do start to get a sense of where it goes right and wrong. Like I had a recent experience where I was talking to an LOM and it brought up a fact that is pretty obscure about the number of pen pals, the Charles Darwin had. And he had a huge number of pen pals because mainly what he was doing was synthesizing other people's research. And I'm like, wow, I know that fact, but not many people would know that.

14:41And then it got into the research on this and it cited a particular famous researcher named Howard Gardner. And I said, he is a creativity researcher and that's a right fact about Darwin, but he didn't do that work. So it conflated these things. And I said, what's your source for that? And the source was me. Somebody had posted excerpt from something in range on good reads. And it had conflated things that were in nearby paragraphs and kind of smushed them together in a way that was plausible, but not right. And every time I see that sort of thing, it's a little bit of a wake-up call that these things are incredible. And also there's all these edge cases that we really need to be playing with to try to understand. I'm going to jump here, but if you think back to the periodic table, which many of us have studied in interchemistry classes, do you think that when Mendeleev was dreaming it up, he was following the path that LLMS is today in the way that he was

15:45dismantling one of the greatest myths in science? Interesting. Tell me more in what sense when in in terms of following the LLMS, give me that a little bit more. Well, we were just talking about how sometimes they're putting out information that seems like it couldn't be correct, and it challenges us to think about things in a different way. And my point here was Mendeleev really way in front of his time when he dreamt up the periodic table in a way that was going to be so groundbreakingly different than what anyone would have expected at the time. Yes, in that sense. On the other hand, I think some of what he did because he didn't really dream up the periodic table, the idea that it came to him in a dream is a very famous myth that I learned in college that is in Matthew Walker's best-selling book, Why We Sleep, that Casper used in mattress marketing, et cetera, et cetera. And the reality was he had a publishing contract for a two-volume intro to

16:52chemistry textbook, and he had only gotten eight of the then 63 known elements into volume one. So he had to get the other 55 into volume two. He didn't have enough space, enough time. So he had to start experimenting with groups of elements that he could describe together, and he started hunting for families. And that's what led to the recognition that there was this recurring pattern. You could predict the properties of elements and even predict where there would be elements that we hadn't yet discovered. And so he was not even looking for a groundbreaking idea at that moment. He was looking for a way to organize a textbook for beginners. And so I actually think the gap between the Mendeley of Myth and Reality is symbolic of something important, which is that we overvalue the complete freedom of something like the dreaming brain and undervalue the power of constraints to channel us into productive experimentation. So I think he absolutely challenged the receive wisdom, but I think he got into it.

17:55As I think most breakthroughs do with a much smaller problem that opened, first started as a keyhole and then opened into something that changed the world. Yeah, I just wrote a sub-stack on this concept of so many of us are searching for the horizon. And you was reading your book and I kind of got this idea from your book because what stayed with me is that this whole idea that too often we reach for a dream when what we need is a textbook as you were just explaining. And what I wrote about is too often we're looking for the horizon when we really need to be as present in our life because that's where we are today. And so many of us want to look beyond that instead of living in it. And I think your book really got me thinking about that and thinking about it in a different way. So I appreciate it. You got my creative brains working. So thank you for that. Hopefully I can capture it. I'll send you the article after we're

18:57done and hopefully I captured it in the right light. Please do. I think that sounds extremely wise. And late and inside the box when I get into this primarily in the last chapter when I get into this sort of more personal philosophical mode. And I quote the MIT professor Sherry Terkel who described our sort of virtual a lot of our life as virtual these days. And she said that we are forever elsewhere. Right. It's like we're often not where our feet are. We're not living where our feet are. And that's okay. That means that you and I can talk right now and do this. But I think she means it in a more figurative sense, which is that our mind is always on something else. Right now my mind is on talking to you, but often in the virtual world our mind is somewhere else. Not where we are. And I think that's a similar point when you're looking for the horizon, right. You're always looking for the horizon. It takes away from the ability to live where your feet are at that moment, right. When you're exercising,

19:59when you're dancing, something like that, you're living where your feet are. And I think that's part of the reason that those experiences are so helpful for people and so enjoyable. And so I think a lot of that what I'm writing about in the last chapter really dovetails with the wisdom you were just saying. And it sounds simple. And yet it's the thing we rarely do figure out ways to live where our feet are at that moment. Kids are very good at this. We'll be right back with more of my conversation with David Epstein. As you're listening today, I want you to think about one question. What constraint in your life might actually be pointing you toward a new possibility? We often spend so much energy wishing our circumstances were different that we missed the possibilities inside the circumstances we already have. David's work is a powerful reminder that adaptation doesn't always begin by escaping the box. Sometimes it begins by looking more closely at what's inside it. If you'd like to go deeper into the ideas we're exploring throughout the Adaptive Human Series,

21:02you'll find a companion workbook inside the Ignited Life on Substack. It's designed to help you take these conversations beyond insight and apply them to your own life. You'll find links in today's show notes. Now a quick break for today's sponsors. Thank you for supporting those who support the show. You're listening to Passion Struck on the Passion Struck Network. Now back to my conversation with David Epstein. Unfortunately, there's so many things, so many of the gifts and patients that we had when we were kids in how we creatively explore things we lose as we go to and I think that is one of the most important ones. I think it really ties into what I think is a big question that you ask in the book and that is a lot of the obstacle to innovation isn't a lack of creativity. It's asking the wrong questions and I think when we're

22:04kids we end up asking questions all the time because we're just a sponge and we seem to lose that as we get older and I think because we lose that to we end up getting kind of tunnel vision down certain paths that we go on instead of taking the time to really think things out. Is that something you discovered while writing the book? Absolutely and I think that's something I discovered well before even writing this book because I transitioned from training to be a scientist to being a science writer and I think one of the challenges early in my career and that I think persists in among journalism especially science writers is you don't want to ask the questions that you're most curious about or where you least understand because you feel like you want the person you're interviewing to think you're smart and that you know what you're talking about but that doesn't help you do your job at all because you're a conduit for the eventual reader

23:10and so seeming smart in the interview is not in service of the end user and it took me a while to get over that and to get back into that child mind of what do I totally not understand about this person's job and I'm going to ask it on the most granular level I really will not even talk to my seven-year-old son one at school he the everyone gets no got an award at the end of the year and he got king of questions and I was like my work here is done and I'll even talk to him about something I'm thinking about or I'll try to explain to him some idea I have that I'm thinking I'm writing about and he'll start asking these questions that make me realize oh I have to go but I actually don't understand that fundamental part I've skipped to trying to understand and explain at a higher level and so in the words of Austin Clean the great the author and artist who wrote steel like an artist book I love he said that he apprenticed himself to his children and to learn how to free

24:11himself to create art and I feel like I do that with my son just as a reminder to really ask those questions that make me the most curious and not worry about seeming smart to the person that that I'm talking to and it's not they don't care anyway right and if you really wanted to understand what you're talking about you want to be willing to expose your own lack of understanding I love that you and I caught up a couple weeks ago when we were supposed to take this the first time and I told you that my background was in information technology and I have been part of literally thousands of projects and I want to talk to you about general magic but I want to use it through this lens of my time at lows when I was at lows during some of the years I was there I was in charge of all application development for the company and we were working we were working on this initiative called total closed loop and what this would have done had it been successful as we would have

25:12solved the holy grail of retail which is what we were going after how do you create a complete frictionless experience for the customer that no matter what channel they interface with the company they feel seen valued and understood so whether you're coming in through a call center the retail store an app whatever they would be recognized and understood and it was a major multi-year program that probably had 200 projects underneath it and it ended up failing spectacularly even though we were about 80% to getting it correct and it failed because I don't think we ever had the right framing and because we didn't have the right framing the CEO of the company and the CFO could never understand how to explain it and it's so interesting to me now because we spent hundreds of millions

26:17trying to implement this home depot is now spending 15 or 16 billion trying to replicate what we were doing in 2005 to 2010 and it kind of made me think about if General Magic had gotten it right with the smartphone where would they be now compared to Apple and is there any lessons from what I just briefly mentioned about lows and what happened to General Magic that you might be able to tie together based on your own research yeah I mean without knowing more details about the problem of framing because I'd love to hear more about that so if you're willing to tell me a little bit more about the CEO and the CFO would they try to frame it internally externally or what was the problem they were having explaining it externally what this thing is or internally so that the teams could cohere or can you tell me a little bit more about that I think the issue was the CEO used to be the CFO and so he was so used to explaining things from a shareholder value perspective

27:22and I don't think he really looked at it because he wasn't an operator who grew up working on a cash register working up working in the stores and really having to interface with the customer I don't think he understood how difficult it was to interface with lows at that time when you were a customer and for whatever reason at that time they did not want to have a loyalty program they were 100% against it so we were trying to put in a loyalty program underneath the surface but doing it in a completely different way that no one had approached because we didn't want you to have to present a card we just wanted you to feel the loyalty because we understood you so much in fact we developed this tool called single view of the customer that without even having a loyalty program we were able to use data matching and had a 92% accuracy rate on understanding who our customer was regardless if they were in their first home or their second home or third but I don't think

28:27that Robert really understood had had to explain this issue from looking at it from a customer standpoint because he was always trying to explain it from a shareholder value standpoint and their tooths completely different things on how you value the problem we were trying to solve. Interesting okay that problem you're trying to solve issue I think tracks absolutely with general magic so to give people a little quick background general magic this is a story about the danger of too much freedom the absence of constraints this was a company conceived in the 80s started in the early 90s that was so visionary that Goldman Sachs took from public in the first concept IPO in Silicon Valley history they went public just with an idea not a product three former Apple employees in one of whom who'd become the CEO named Mark Perat was in the late 80s was making schematics that if I showed you right now you would think where it was the iPhone right thin glass

29:28rectangle no protruding buttons just a touchscreen rectangular apps it's a phone it's a computer it's an entertainment center etc so he saw the future the web did not exist when he was drawing that he really saw the future and so it drew this incredible collection of talent money to the team and they were going to build basically the iPhone is starting in 1990 totally audacious they were even doing patents that are coming back now because of autonomous AI agents like they were just I read his PhD dissertation for Stanford in 1976 where he coined the term information economy and like this guy was an incredible visionary and they had so much talent they had this like literally I've heard they made a trip to Japan and CEOs Mike Stern who was the general council and CFO at one point recounted to me being chased down a train platform by a CEO of one of the largest companies in the world to please take my money and so they had this huge consortium of international companies and they were innovating like crazy I mean

30:34earliest form of USB precursors to emojis they made a virtual gathering space where agents could interact for commerce and what did they call it 1990 the cloud they were just so far ahead but because they had so much money and so much talent they could do anything so they frequently did do anything and they just added and added the project grew more people more resources when it finally shipped way late it shipped with a 200 page user manual and it wasn't clear to anyone what problem it was solving for them because it had so much stuff that it wasn't really coherent and so it was a absolute flop the company goes from stock price doubling on day one to basically worthless in two years and one of the problems I interviewed dozens of former employees that they would tell me is we did not have a clear customer there's this saying that they called their customer Joe six pack and nobody knew who the heck that was right there's this saying that people don't want

31:38a quarter inch drill they want a quarter inch hole in the wall not what cool thing can I make for them but what problem can I solve and they did not have that and so they were so unfocused because they were not defining a user problem that they were solving and so it was so amorphous every time they added some feature they just retrofit the story to say well here's why people need this instead of actually understanding the customer problem and building to that so people internal to general magic who recognized this problem did some interesting things one guy said oh the web happened he built a website a low level service engineer built a website on his free time he called auction web to facilitate commerce between strangers and general magic had a right to any IP developed by employees so he brought it to the leaders said look it's doing one of the things we want it I know this is way too small this is small potatoes we have this much larger vision so he

32:38took it outside and changed the name and it's that became eBay right piero midi are was a low level service engineer another guy Jeff Hawkins was creating an app that did turned strokes with a stylus into writing and when general magic kept expanding he said you know what there is a clear customer problem busy professionals want to sink their contacts and calendar on and take them on the go so he moved out and created his own device it was just context calendar memo pad period three of the bajillion things that general magic was doing and that was the palm pilot smash hit in the same era so the people who really spent time defining a customer problem they were solving that made so much difference because it channeled and focused their work but sounds like similarly to the experience that lows there was at least for the CEO maybe a lack of really focusing tightly on what customer problem exists and what is the way that we're solving that I think the mistake when I

33:40really think about it is that what we should have done is we should have gone back to like Amazon starting with books or NVIDIA with gaming or deep mind with video games I think if we had solved a smaller problem first and then another smaller problem and not tried to solve the huge the huge thing then it probably would have been a different story because there would have been success stories that could have been touted and we could have trickled the tiny pebbles and so I think it was a sequencing issue we tried to go for the big bang instead of sequencing a lot of small wins that would have been let up and allowed people to see the vision along the way and I think it was that vision gap that probably cost us the long-term solution that we were so really between months from accomplishing that's so interesting and AFF is I want to hear how the decision was made eventually

34:42to kill it because that's often very hard once a lot of money has been spent but that sequencing it reminded me of so in inside the box I write a bit about the work of a Danish professor named Bent Flubier who spent his career studying projects and he kept a database of 16,000 projects and he led a bunch of projects himself and was involved in projects over a billion dollars budget and things like that and he defined what he said was the typical path what he found first of all was that only eight and a half percent of projects came in on time and budget and 0.5 percent did that and delivered the promised benefits and the typical pattern for not working out well is what he called think fast act slow where there'd be a big idea that was so enticing that it would quickly become a big project and instead of starting small doing some experimentation some iteration defining the boundaries biting off that small chunk you just go into big execution really quickly and then

35:44it's act slow because once there's so much momentum it's actually hard to adjust to these lessons that you're learning while you're going the reverse what he called think slow act fast was spending time being small and defining the boundaries and picking up defining those smaller problems you're going to start with and then when you move into execution you can act much more quickly and or as he colloquially he said Pixar planning so one of the reasons I wanted to write a bunch about Pixar is because in his work he said that Pixar planning is like the best way of planning and he was talking about all these sorts of rules and constraints they have but all but things like to your point where they will keep a director in story development for with a tiny team for years forcing him to refine the core of that story what is it about stripping away characters that's why there was no on we and shot in Freud in the first insight out as they were stripping it away making it simpler and it might seem really inefficient to spend years in that mode but the cost only explode once they

36:46move into production and complexity explodes and so if they can stay in that small group then when they really have the boundaries to find they move into production they can act really quickly and they were always finding ways to be small to your point I don't know if you remember but in theaters Pixar used to always start their films and maybe still do I'm not sure I haven't seen one in theater lately with a short it would be like a five to ten minute short and those were always them experimenting with some new kind of animation where it was all right we're going to just try it for a short and see how this works and sometimes it worked out and sometimes it didn't but they were always looking for these ways to be as small as possible while they could it's a very good point when I took over the application development organization everything we were doing was typical waterfall type of development and one of the things I was happy to bring in in and hopefully I think they use it now as the main thing is I've run an agile development because what was happening I remember I met with the SVP of strategy one time and he says to me his name

37:49was Scott he says John you guys are absolutely incredible at delivering projects that have absolutely no value by the time they're done and it was hard to hear it he was right because you get so far into these things that there was so you had so much sun cost fallacy going on that you just wanted to bring it to an end even if it was irrelevant by the time you were done with it and with the agile development really allowed us to do was to really use trial and error and to be much closer with the customer to experiment with what we were actually whether that customer was an internal person or external person is what we're building actually solving the problem and if not it's much easier to throw that away after a couple of months of work than it is after two to three years and tens of millions of dollars. Speaking of which with that big effort how because that sun cost fallacy is so difficult such a strong bias for humans how was the decision made

38:51to end that sort of holistic customer experience program that you were talking about? Painful I so sure at the time this has got to be a very rare occurrence our president and chief operating officer was the brother of my boss the chief information officer you don't ever see that in Fortune 50 companies and it was the brother the older brother who was the COO who eventually made the decision and what it came down to is we're working with a lot of the larger vendors on the outskirts of this project but the inner most the hardest thing to work on was this very complicated data engine that had to keep track of all the requests that the customer was making in real time to run it across all the other data layers that were in the company and we partnered with a startup which we might as well have just bought instead of working with them the way we were and it was really their inability to deliver on that technology that kept being this huge hurdle

39:56it kept you we have to wait we have to wait we have to wait and keep giving them more and more money and after you're giving this company tens of millions of dollars to survive and you're the one who's basically bankrolling them eventually a decision had to be made. Those are tough decisions I always admire it that because in some ways it would be easier if more expensive to bring it to the end anyway although like you said then creating something that what was the phrase that you the the hard feedback that you got great at creating solutions that are no longer needed or something like that they're basically irrelevant by the time they come into production so I kind of admired those anyone who can defeat the sunk-cost fallacy at any time I have some admiration for even though it's painful and I admired Scott for at least being honest with me giving me an opportunity to try to fix it well I want to talk to you about another project so as I was reading the book one of the stories that really stayed with me came out of chapter six and it was a story about UPS and why this was relevant for me is UPS was trying to answer a question

41:03that when I was a consultant and Arthur Anderson I was trying to answer for a different company waste management and that is how do you find the most efficient route for thousands of drivers making thousands of garbage stops or deliveries every single day and what was interesting to me is when we went into this project we went in believing the challenge was building a better route but what we discovered and we discovered this because we actually sat in the garbage trucks and did the routes with the drivers was that the software could optimize miles but what it couldn't optimize was the drivers were carrying years of local knowledge that the system couldn't possibly know neighborhoods alleys traffic patterns customers all the little things that never showed up in the data and so as I was reading about what UPS went through I found myself thinking man I've lived

42:04through this before what was one of the biggest surprises for you that the technology only became powerful when it started learning from the people or was there another insight that you gained I think the first big surprise for me was the extent to which the drivers once they made this had to remap all of America because Google Maps wasn't accurate enough for UPS and made this routing algorithm the first big surprise to me and to them was that the people actually had to use it basically rejected it and their managers rejected it and part of that was because it did weird things it would tell them if they were driving down one road and there was a delivery to make on a side street a human will stop off and make it and it would tell them off and to skip it and come back later or the one that that several drivers told me about was if you go on a cul-de-sac and it would tell you to zigzag instead of walking around the cul-de-sac maybe you're going to save two meters

43:08or something but it's weird to a human then on top of that it didn't necessarily take into account there things like oh someone has to sign for this package they're not home but I know they work right around the corner or something like that right or I know I don't want to be here when camp drop-off is happening or whatever it is all these little things and because of I think the lack of feeling like they could still use their own local knowledge and because of some of the oddities of the way the routing algorithm told them to behave they just plain rejected it and so you had this incredible you know expensive time intensive project that does nothing for you if you don't have the change management right and so it actually did not take them it was intensive but it took them a lot longer to integrate this into the workflow then it did to actually create the routing algorithm and so they had to go back to square one and ended up meeting with every driver eventually showing them all this things worse they started making these video game consoles where you could play with the routing algorithm with Orion and see

44:10how it worked and if you change things and so we're trying to make it go from a black box to a glass box as they said and they started making games like beat the computer if you can do better job do it and remember one of the drivers who'd been working for 20 years it came home the first day tried it and said see I told you your algorithm your algorithm stinks I beat it and when they looked at his data he'd gone 135 miles to the what the suggested Orion suggested route that was like 140 but he'd been going 150 for 15 years and so this competition with the with the machine got him to do better and so I think it was just surprising less surprising now that I've done that research for chapter six but that visions of change are much more compelling when they include visions of continuity I think that's I'm paraphrasing something that Adam Grant has said but it's so true that the change in many cases with technological disruption the change management is the hard part

45:11and I think that's totally true with AI too right I think one of the reasons I've spent a lot of time with one company over the last year and a half that helps other companies implement AI and what's become apparent to me is that one of the reasons why we're seeing despite this incredibly unprecedented rate of adoption that some of the productivity numbers are not keeping up is that we're often doing a poor job of actually integrating it into the known work streams and caring about that and thinking about that and so I think the surprise to me was the degree to which you have to the bigger a change is the more important it is to include a vision of continuity and to ground it in something familiar for people and that that goes throughout design right that's why your electric car has a plug-in where the non-existent gas tank is and why the folder's on your desktop look like old hanging folders because this the what are big changes we're trying to embed them in things that communicate to you what it is that this thing does and how it connects to the things you're used to David one thing I have learned in two decades of doing many technology

46:17projects is technology projects rarely fail because the technology doesn't work they fail because people don't change and you're right because there isn't a change management strategy well as you were living to I think one of the smartest decisions that the UPS team made was realizing they couldn't ask drivers to trust a system that the drivers couldn't understand so I really liked how they brought the drivers in to see how the routes were built experiment with them understand the reasoning and I love this analogy of turning a black box into a glass box it made me wonder if explainability isn't just a design choice is it what actually creates trust to some degree the algorithm right of the code but they did want some hands on experience with this thing just to start to tease out its behavior and understand it and see that there was some

47:17logic to it and some sense to it and so I absolutely I think so for sure that it's connecting with things that are understandable to us that makes sense that really build that trust one thing I really loved about the UPS story is that in the end they thought they were solving a routing problem and it turned out they were solving a learning problem and why I like this story is it's a great example of what happens over and over again in your book we think we're solving one problem but the real breakthrough comes when we discover we've been asking the wrong question all along which is what we started with is that one of the main threads that you wanted to connect all these different stories yes yes I think you put it very well I'm not sure I have anything to add to the way that you just summarize that yes I wanted to bridge from there to I always think our books are influenced by someone great life for me my book was really influenced by the

48:23lifetime of work by a psychologist Gordon Flett and your book as I read it Herbert Simon keeps coming back again and again throughout the book and he wrote in 1970 that a wealth of information creates a poverty of attention today that feels really prophetic why do you think Simon understood something about our future in 1970 that almost all other people missed man if you read his writing about what was coming with AI starting the 50s he was seeing a lot of what was coming he was a founding father of AI but I think Simon first of all he was a massive polymath right he won the Nobel and economics he won the highest award in psychology won the Turing Award in computer science and so one I think he saw across disciplines but at the same time in his own words he was really just studying one problem which is how humans make decisions and he was so enmeshed in that and the workings of the brain and modeling the workings of the brain in some ways he was

49:28fighting neoclassical economic theory which modeled humans as rational actors the more options will always make a better decision and he was saying theoretically that's true in practice it's completely false because of the way our brains work sometimes more options will paralyze us sometimes they will lead to better decisions that will regret more and sometimes they will just read to to worse decisions and so I think he was just so like acutely aware of how the things that seem like they should only be good more information more tools because of the way our brains actually work can often have are an ill fit for the way our minds actually work and I think he was attuned to that because he had studied some economics and then realized that what he was learning did not in many cases represent real human behavior so he was attuned to this kind of disconnect in a way earlier than

50:30than most people were I do have to say I love it when I read a book and it introduces a person to me who I knew who he who Simon was but I didn't understand the death of going all the way from political science to winning a Nobel Prize in economics and it is very wild he's like under every third rock in social sciences that I turn over there's something in Simon using credly prolific also just a brilliant person and one other thing I wanted to quickly talk about is an episode 301 a few years ago I interviewed glory mark and it was one of the most fascinating conversations I've had on this podcast because we spent a lot of time talking about attention residue and task switching and reading your own thoughts on Gloria I realized that as we were talking about Simon he anticipated that entire conversation that we both had with Gloria decades before these things ever existed how did learning this and talking to Gloria

51:35you know change how you think about protecting your own attention some of Gloria's work is the to me sort of the let's say scariest research that went into the book because when she started studying people in the workplace basically 25 years ago she found people were tasks switching about every three minutes on average and then I think 2012 it was every 75 seconds and then by 2022 is every 45 seconds and the trouble is that multitasking is not really possible you can talk and walk at the same time so if one of the things is an automatic function then you can but not between real cognitive tasks and two of the most interesting things I think from her work were one she was monitoring physiological measures and she found that the number of tasks tasks switches someone made during the day predicted their stress as measured by heart rate variability and so this whole area of how attempting a multitask impact stress was new to me and a little frightening

52:36and also her finding that we become accustomed to a certain cadence of interruption where for being interrupted by anything all day whether it's people whether it's notifications whatever you train your attention so that even if you say today I have to focus and I put the phone away you will self interrupt with intrusive thoughts at the rate to which you've become accustomed as if we have this internal distraction barometer that wants to maintain a certain pace and I'm like I found that frightening and I see that in myself when I'm I'll go into these periods where I'm writing a book where I'm like really locked in and I won't even turn on my phone until my workday is done and then I may shift into these modes where actually I'm task switching like crazy and I can feel how hard it is for me to then sit down and read at the depth I want to and things like that and with Gloria's advice I started batching my work where she found people in offices and is a little old now so maybe it's more we'll check email on average 77 different times a day and so

53:40what you really want to do is if you have to answer all those if you really have to answer all those emails can you put that into one or three or five blocks a day where you're just answering email and then you're not what you don't want is 77 different tiny blocks throughout the day and so I've been much more intentional about structuring my work where I'm monotasking in a certain block taking a little break then moving to the next block and for the last thing of every day I do what I call the Hemingway Principle where Hemingway would stop writing in the middle of a sentence so that he knew exactly where to pick up the next day and so my last act of every workday is to write down the important thing I'm going to start with the next day because I think it protects me from what's called the mere urgency effect in psychology where people will prioritize tasks that feel urgent like answering a message over things that are important even if those things are objectively much better use of time so her work really just made me much more proactive in thinking about the way that I structure everything during my workday because I think at this point as her work shows if you're not structuring your

54:42attention it is being structured for you and I want to be the one driving it I love it and it's such an important message I wanted to make sure we got it into this interview. Last question for you David is if someone listening wants to conduct their own theory of constraints audit this weekend or this week what are the first questions they should start asking themselves to potentially uncover the bottleneck that's limiting everything else. I think the first thing I would probably say the first thing that I did for myself because I launched a some months ago video operations I made a YouTube channel it's a total workflow that I'm not used to in all these kinds of things and the problem early on was I just could not move enough scripts through to make it viable basically and get ready for filming and so I stole something from a genetics lab that I visited where I made all of the steps of the process visible by with post-its on a wall and I said which of these steps where is the work piling up because it's

55:43not all these steps and it actually became pretty clear where some of the work was once I laid it out that way I said it's definitely not here it's definitely not here it might be here where's the work piling up and once we identified that spot where work is piling up it doesn't matter how efficient all the other steps are because the work just piles up at that one step and so once we had that identified I could talk to the team and say all right this is the bottleneck how do we widen this stop spending effort on the other steps because it's just making those steps faster and piling even more up at the bottleneck and we need to focus our energy on how do we widen this one step so I would make every step in the process visible just you can look at it on the wall say where's the work piling up and then don't spend energy on the places where work isn't piling up because that's just running the machines faster so they can pile up at the slowest step awesome well David for a listener who wants to learn more about you where's the best place for them to go David Epstein.com I have a free newsletter that you can find there and a free YouTube channel where I turn a lot of these

56:44things into sort of video lessons that's new so yeah but it's all you can find it all at David Epstein.com and I would add an excellent sub stack that I subscribe to personally. Oh thank you thank you I try to pretend that nobody whose brain I admire reads that sub stack because for me it's a place where I want to be a little free to be very imperfect so I appreciate that I will attempt to forget it. Awesome well David such an honor to have you today thank you so much for joining us. Total pleasure thank you for having me. I hope today's conversation with David Epstein gave you a new way to think about how you approach change uncertainty and the problems right in front of you. One of the ideas I keep coming back to is David's story about Mendeleev. We love the mythology of the breakthrough the genius who suddenly see something no one else can see but the reality was much more ordinary. Mendeleev wasn't sitting around trying to change the world he had a textbook to finish and that constraint forced him to experiment look for patterns and ultimately see something much

57:46bigger. I think there's a powerful lesson in that sometimes we don't need more freedom more options or a bigger vision sometimes we need to pay closer attention to what's already in front of us. Because the problem you're trying to solve right now may actually be the doorway to what's next. If something from this conversation resonated with you I'd love to hear about it send me a message on substack linkedin or instagram or share this episode with someone who could use a different perspective on how to navigate an unpredictable world and make sure you're following passion struck so you don't miss what's coming next because next week we're taking everything we've been talking about adaptability uncertainty constraints and the ability to keep moving forward and putting it to the ultimate test. I'm joined by Cole Broward. Cole became the first American woman to raise solo non-stop and unassisted around the world. 27,000 miles alone at sea facing storms

58:48exhaustion isolation and moments when everything could have gone wrong but what's most fascinating isn't simply that she finished it's what she discovered about herself along the way. At one point after being told she was too short to compete in the southern ocean Cole found herself at what she describes at one of the lowest points in her life and then something changed she decided to go anyway. She went for being told she couldn't do it to sailing around the world alone and there's one moment in our conversation that I think captures the spirit of this entire series. When you hit rock bottom is exactly what I need to go back up. While I'm at the bottom all I can do is go up I can't really go any lower. I wish I didn't have to hit rock bottom to find exactly where I am going in life but maybe that's kind of the beauty of it too. That's next week on Passion Struct because David reminded us that we don't always need to escape our constraints. Sometimes we need to learn how to work within them

59:51and Cole shows us what happens when you take that idea to the edge of what you believe is possible until then remember you don't have to know exactly where you're going to take the next step. Sometimes you just have to be willing to take it. I'm John Miles and I always stay curious stay intentional and stay Passion Struct.