AI meets real estate, with Zillow's engineering SVP Toby Roberts

0:00The home buying journey is the most expensive and one of the scariest transactions that people are going to go through. And they're lives, actually like 50% of people cry during a home purchase. I mean, I'm sure hopefully there's tears of joy, but there's a lot of tears of frustration in that, you know, at times. If the AI agents are enabling real estate agents in terms of like doing a lot of that kind of fundamental blocking and tackling work that, you know, that the real estate agents are doing, they can spend more time focused on what is that human connection that has to kind of exist in this really scary transaction. Hello and welcome to Geekwire. I'm Geekwire co-founder Todd Bishop. And I'm Geekwire co-founder John Cook.

1:00We are coming to you from Seattle where we get to report each day about what's happening around us in business, technology and innovation. What happens here matters everywhere and every week on this show we get to talk about some of the most interesting and important stories and trends in the news. We are recording this week's show live on location at atmosphere Seattle. It's an event venue here on the shores of Lake Union in front of a room of real estate professionals and invited guests. We're here in partnership with real estate at work. That is a meetup and professional networking group led by Lake A. Deva, Seattle real estate developer and broker. Yes. We've got a great crowd in the audience. It's a mixture of technologists and real estate professionals coming up on this week's show AI meets real estate. Seattle has been a key intersection for technology and real estate. In fact, we've been covering this mega story in the region for many years now.

2:04John, I kind of feel like you broke this entire story back in the day. I set this multi-trillion dollar industry on pass. Actually, I was looking back to one of my first stories I ever wrote in the Seattle Post Intelligencer. We're taught and I worked a long time ago, a newspaper base here in Seattle. I was the first reporter to break the news that Rich Barton and Lloyd Frank were starting a new company out of Expedia. This company at the time, in fact, I said it was tentatively called Zillow.com. So they weren't even sure on the name. But back in 2004, and the first headline was, former Expedia exec is back. And then a few months later, we had another story on the PI where it said, tight-lipped Zillow pulls in financing. And we had a quote from Rich Barton at the time that said, we really think this thing can be big. That's all he said. Well, that's the history. And tonight we have a special guest to help us get our minds around what's next.

3:06Our guest is Toby Roberts. He's the senior vice president of engineering at Zillow where he's been for 10 years. He runs their core technology, the apps, the website, the data, the AI, with a team of about 650 engineers in the US, Mexico, and India. He's the one who moves Zillow off its own servers and onto the cloud. And before Zillow, he worked at companies including Amazon and Cox and Electronic Arts. Please welcome to the show Toby Roberts. Thanks for having me. Great to be here. Toby, great to see you and great to have you here with us for this conversation about where AI is meeting real estate. Something I think we're thinking a little bit about these days. Just every day. Every day. Every day. Over and over and over again. Yeah. So, you know, Zillow was born 20 years ago, just celebrated your 20th anniversary. And, you know, really you kind of came of age in the mobile era. Now we're switching to a new technological era.

4:06That's right. Moving to the era of AI. AI era were immersed in it in many ways. And just to start us off here, I'd love to hear your thoughts on how you feel AI is changing the way we find by and sell homes. Yeah. I mean, it is a complete watershed moment really. It has changed the full ecosystem everything from how I understand what I'm even looking for to understanding my financing options now, to really narrowing down where my search is and really what's the type of home I'm looking for. Am I ready for renting or am I, you know, am I moving from that rent to like kind of the buy scenario? And so, AI and our capabilities to understand our customers more so and deliver the context and meet them in a more personalized experience has just been this massive unlock. You know, we're in a room here with a lot of real estate professionals. And one of the big questions I know that's on many people's minds is what happens when agents start doing the work of agents when AI agents start doing the kind of work that's traditionally done by real estate professionals.

5:15What have you seen from your purchase at Zillow so far? It's going to be a lot of talk about agents here. Yes. You know, we're going to have to clarify that. I think like what we see for real estate agents, I mean, I don't have to tell the room, I think, that they are hustling day in and day out to make those deals, understand their customers, keep up with the trends. So what we see is the opportunity for AI agents, I'm going to have to like preface this, I think through the whole, you know, next hour or so. We see the opportunity for AI agents to really be that copilot for the real estate agent, right? Like, understand how do I keep in touch with my customers? What's the next best action for them? Like, oh, what was that call that I had? And like, what were the follow-up action items that I took for them? So how do those AI agents really enable and supercharge our real estate agents?

6:18That's where we see the big opportunity and what's changing. And are you seeing that in practice with some of the real estate agents that are out there that they're actually automating those parts of their jobs? We are. I mean, Zillow owns a real estate agent CRM and we see and have delivered the capabilities to drive the company. So when you're trying to juggle all of the things that you're doing as a real estate agent, checking in on, hey, where is that loan? Like, where's the loan status? Is my customer ready to move to the next step? Have they submitted all their documents? How do you kind of juggle that and remember that all the time? So we're actually seeing that in practice where now these summaries and these action items and these follow-up steps are available for the agents. You know, Toby, we were talking with some folks in the reception before we started recording. And there's some angst, really. Real estate is not just a business, it's also a creative pursuit. And especially when it comes to marketing materials and things that people would have taken great satisfaction in in the past,

7:24there's a lot of people who still want to do that in a bespoke kind of creative, personal way. Where do you draw the line between what's productive, what's creative, and how do you think about AI agents in that context? People should always have the avenue to escalate up to a human. And so while maybe customers start in a chat experience and try to understand the market or try and understand what they're looking for or try and understand their options, we'd, Zillow always want to make sure that real estate agent is available or that loan officer is available as well. So we think about that. It's not quite human in the loop, but what's that level of escalation that we can kind of provide? And make sure, because the home buying journey is the most expensive and you know, one of the scariest transactions that people are going to go through. And there are lives actually like 50% of people cry during a home purchase. And it's not, I mean, I'm sure, hopefully there's tears of joy, but there's a lot of tears of frustration in that, you know, at times, right?

8:30And uncertainty and just like, oh my gosh, like especially if you're going through this for the first time and it is a really challenging first time and there's a lot of ambiguity and uncertainty. And those real estate agents and loan officers at times are really those that unlock, you know, and really kind of share with their customers. Okay, I'm going to walk you through this. I'm going to like kind of help you. And so we feel like if the AI agents are enabling real estate agents in terms of like doing a lot of that kind of fundamental blocking and tackling work that, you know, that the real estate agents are doing. They can spend more time focused on what is that human connection that has to kind of exist in this really scary transaction. So that's the external world and obviously the agents that you're working with and the transactions that you're enabling by connecting home buyers and sellers with agents. Can you give us a sense of the current state of the art of the technology inside Zillow, the products that you and your teams are developing and where things stand in terms of AI and how you're implementing that in your products and using it in your engineering inside Zillow?

9:44AI is not brand new to Zillow while generative AI is having its moment, right? I don't know, 22, 23, I, 2022, 2023 when we really kind of started to see that spike. We have had ML traditional ML models in production now. Now, Zestimate is the one that everyone is familiar with. That's what we're kind of most famous for. That's a traditional ML model that's been in production now and continually evolved and improved, but for 20 years. So we've continued to, like from the start, from when Rich was saying, hey, we think this could actually be, you know, this could be big. We've got something here. It was a founding capability out of the gate. And so since then, and then if you go back the, you know, the last few years, we have continued to invest in how do we bring a better understanding of the home to customers as well. So if you think about that, a lot of times I remember when I was buying my first house, I would go into the back of a real estate agents car and drive around for days on end, you know, going and looking at different homes and different homes.

10:55So we see the ability now with AI capabilities. How do we provide people the greatest sense of home, of the home that we can in a 2D format. So you can kind of understand that before you choose which ones you're going to go and tour. So now real to scale four plans based on pictures and capabilities that are now moving to phones, for example, that's there. We have the ability to with drone footage that is captured by professional photographers. You can now fly through the home. You're not constrained by just a movie. And so you can kind of just like fly through the home and really get a good sense of that. And then I was talking about before on the real estate agent side, how do we enable the real estate agents to understand their customers that may be shopping on Zillow. So how do we kind of provide with permission that context to the real estate agent so that and then provide that in summary capabilities and action items and follow up those types of things. Yeah, just you talking about the imagery.

11:56So much. So much there. So much talking about the imagery of homes that just got me thinking about there was a story in the New York Times just in the last few days about the headline was something along the lines of is that fireplace real. Right. And so it gets in this whole idea. Yeah, you can make those homes look really great. You can. But you can make them look so great that they actually don't really exist. So we're going to tease that ahead to our trivia question later, which is going to be really fun that we hope you stick around for. But where do you draw the line on that and where does Zillow come down in terms of the use of AI and creating content in listings? So really it's a really great and important question. There goes the plan is the flip plan. Yes, there's the flip plan. We're on a power right right on here. All right, we'll ignore the future ones. Yeah. So it's a great question and we've been pretty we've been pretty public on this. We think that the listings should accurately represent the state of the home.

12:56We understand that there's capabilities with AI now and you know, yes, that fake fireplace and it's happened that fake fire in the fireplace that's happened for you know for years and years with photoshopping. But we really truly believe that the listing photos should accurately represent the home itself. Now we do have the capabilities. So if you're on the seller side, we again like those listing photos should accurately represent. So by side, we do have generative capabilities that we have on the by side if you're looking to buy that home. And so we have the ability, for example, virtual staging. And so in that case, we are very transparent and our customer has selected to take that photo, which should be the stock listing photo and then apply generative aspects to it, AI aspects to it to make it look, you know, in whatever, you know, mid century modern theme or whatever theme, you know, is your preferred preference. But we think that that should be a very transparent and very much that the customer has selected to make that change, not that the stock photos are altered at the gate.

14:03Do you see Zillow as having a responsibility to be the trust layer in that in other words to ensure that those kinds of generated images in listings, especially our transparent that the people who are viewing them know that they might be generated by AI. So at this point, if you were going to try to rein in the faux fireplaces, yeah, talk about technology, run a muck. But do you see a responsibility there on the technology side? I think we will, we've been an advocate for it. We'll continue to be an advocate for it with all the, you know, the various amaleses and what are expected, you know, of kind of listing agents and what listing photos should be. Whether we're the police, I think like more so that I think we'll be public and we expect that and we believe that customers who are looking to buy their next home or, you know, rent their next home, they should receive accurate understanding of what that home is. You mentioned the Zestimate earlier and that really is in many ways the flagship data feature than some ways the flagship feature of Zillow.com, especially.

15:10How has AI changed that feature and what is it allowed you to do with the Zestimate that you might not have been able to do in the early days of machine learning, especially? I think it probably goes back to that computer vision capabilities again, if I were to, you know, kind of just like heart back to that. I think by a greater understanding of the home, the finishes, did you renovate that kitchen or did you renovate that bathroom? You know, those were something initially we weren't able to necessarily like kind of pull that in and try and factor that into the Zestimate or the value of the home. As we get that content and as someone may upload those types of pictures associated with the home, we may be able to then pull that in as part of the Zestimate. Do you feel like the Zestimate is more accurate because of AI? I think in general we've tried to drive accuracy of we continue to drive accuracy as much as possible over and over. AI is certainly from a visual perspective, gives us more understanding of that home, which will lead to a better ability for us to get a Zestimate on that.

16:15So another really interesting place where Zillow is kind of a litmus test for this new world is your interactions and your integration with open AI and chat GPT. And I want to turn this over to John here because he actually had a really interesting experience with this. Tell the story John. Todd I'm moving. You're leaving Seattle. Are you moving to Georgia? Yes, that's right. I'm leaving Seattle the tax infrastructures to crazy for me. No, I'm just kidding. I'm not leaving you Todd. Don't worry. No, but I was playing around with with the Zillow app in chat GPT yesterday. It was a really cool experience and it was pretty fun being guided through the whole process. So I started looking at properties on would be island and it was very easy to use. And I just clicked that green button to set up the home tour within the app and literally within two minutes an agent called me. And I was like, whoa, they were aggressive in terms of getting on the getting on the horn and calling me to get my sense.

17:21Set up the tour. Yeah. It worked. Right. It worked well. So I want to be clear. So glad to hear that. So this was you were interacting with chat GPT the Zillow app inside chat GPT was not the Zillow app on mobile. And the agent that called you was a was a human was a human that called you within how how soon. I mean almost instantaneously and I don't think he knew in fact I asked him later. I was like, how many leads are you getting through the chat GPT function? It's like, yeah, this is really the first time it's been brought up with me. And I don't even know if they know on the back end. They probably just see that the lead came into them. It just came through Zillow and I'm not sure if they knew it was through chat GPT or not. No, they wouldn't know that it came through necessarily chat GPT. It essentially would would get we'll get into the tech as much as you want here. But it will essentially use our come through our same lead funnel we expose essentially an MCP and capabilities out to be able to support the open AI embedded Zillow app.

18:23But we support any of the functions within that largely through a lot of the other infrastructure that also supports our app or web experiences as well. How was that integration gone with chat GPT? What have you learned from it? And are you seeing provide us some numbers on how it's going? It's still early. So I mean we were a launch partner with open AI. So we as they launched their app store. I think they hadn't decided on the name at the time. But we were the one of the launch partners there. And so we saw this as an opportunity as a new surface area by which our customers. We want to meet our customers where they are. And if our customers are in open AI and they're trying to understand their next move or their next rental or their next flip. I heard a bunch of flipping talk earlier in as part of the intro. We want to meet them where they were. And we want to be prepared and ready for if people are starting to use that surface area.

19:26And so we did some really unique things. We basically have our we had delivered our canvas. We were able to like render our own map within the chat GPT experience. And so the pins and everything that you're familiar with if you're in the Zillow app or if you're on the Zillow website and our map and look and feel those are all there and we rendered them live within chat GPT. But a lot of the infrastructure on the backend is like I said is all essentially the same foundational aspects. The traffic is still relatively new as customer behavior has changed. But we see high intent individuals when they come through that avenue. Like John wanting to move to would be island. Apparently would be island. Yeah. You mentioned MCP the model context protocol. And I think it's interesting in that you're kind of a litmus test here for future websites for future web services that might do their own integrations with chat GPT with Claude with Gemini in the future.

20:33What would you say the level of sophistication was with the platform like how would you compare developing a Zillow app for chat GPT with developing a Zillow app for iOS or Android and like what can people expect if they go down this path themselves. Anything with you have to make sure you're also keenly aware of any of the security and governance controls that and MCP when we first started using it was relatively new. And so getting that off authentication authorization support was still very much in its infancy. And so we had to kind of work to build aspects there to make sure that we were understanding our customer as they were coming in. We were protecting their data. We were only kind of having very explicit allow kind of prompts back to our customers whether they were going to share that data within other areas of the ecosystem as well. Zillow's regulated industry. We are a mortgage originator. So we have to abide by fair housing fair lending all of that construct. And so we had to ensure that we had the right governance controls in place for

21:42all of those experiences as well. That was like you know it was a it was a absolute must day one for us. We are here talking with Toby Roberts. He is the senior vice president of engineering at Zillow. We are on location in partnership with real estate at work here on Dexter Avenue overlooking Lake Union in Seattle. You can probably hear the float planes behind us. We're going to take a quick break and we will be right back. Some of the world's biggest companies have brought jobs and wealth to the Seattle region. That growth has come with some real pains to you. There are these hotspots. Seattle has the potential to be one of those tech people are getting replaced by AI and you can't replace us with AI yet. I'm Monica Knicksburg and I'm Joshua McNickles. We host booming a podcast about the economic forces shaping our lives here in the Pacific Northwest. We get into the stories that hit your wallet and our region. Listen now on the KUW app or wherever you listen.

22:48Welcome back. It's Todd Bishop and John Cook. We are here in partnership with real estate at work. A professional networking group led by Lake a Deyftha, a Seattle real estate developer and broker. She's got such a great community here. It's great to see all of you here in the audience. Our guest this week is Toby Roberts. We are talking about AI and real estate. It's such a fascinating intersection here in Seattle. It's been a long time tradition in the tech community. And as I was saying before we started recording the level of entrepreneurship that you see among people who are interested in real estate and technology is really just a big deal. It's kind of a great alchemy that we have here in the crowd. I was talking with some people who are doing some fascinating things all across tech and real estate before we started. There has been Toby some fascinating news just in this past week that we'd be remiss if we didn't talk about. And that is this larger push in the industry to actually slow down AI. And we actually heard about this directly on the show from Bill Gates a couple weeks ago.

23:59We got to do an interview that we played clips of on the show. And it was such a remarkable moment to hear none other than Bill Gates saying this stuff is moving too fast. The guy who has been pushing technology you know probably further than it should have gone in the past if you think about some of the things that Microsoft put out there. And here he is saying no this is potentially dangerous. And I'm wondering just in general Toby how are you reacting to that. How is the overall push in the industry for manthropic and others to slow down AI. How could that potentially impact you Zillow and the larger market for real estate and technology. We have not we still see plenty of opportunity for us to continue to use AI to enable our customers and real estate agents. So at present we still see plenty of runway for us to use. We're in a different scenario. We're not building large language models.

25:04You know again I talked about we are doing ML. We've built you know traditional ML models. We're not building our large you know kind of LLM's you know foundational LLM's internally. For us we are leveraging we are partners with anthropic we are partners with open AI we partner with Gemini. So we are partners with a lot of those those companies. And so I think in that in that sense I also mentioned a few minutes ago we've just been so aware of any of the governance and safety measures out of the gate that they have just been a day one must for us. So quality and the safety has to be there before we're launching any of these you know particular models into into a production. So that's just been a must and we'll continue to make sure that that's foundational in our use of AI across Zillow. I am curious are you seeing a use for some of these more advanced models like Fable from anthropic. You know it feels like I used Fable once or twice and I felt like I was sort of basically I would have needed a garden hose and I was you know applying a fire hose to it.

26:13It was just too much it was too much for me. Are you seeing the Ferrari to go get you know some milk. Are you seeing places inside Zillow where these most advanced frontier models are actually providing value that you wouldn't have otherwise. You're talking kind of Fable level models and astral level models those we have seen mostly use in internal productivity use cases. So if you are a developer for example maybe you're planning you know your plan section would be leveraging some of those you know kind of Fable 5 1 type models or astrotite models. Some of the coding itself may be down in kind of the sonnet or some of those other kind of models that those those providers provide. And so that's how we're seeing Fable used we haven't used Fable for example in a in a customer facing whether that's a B2B real estate agent customer or someone buying or selling a home we haven't used it in those context yet.

27:14So to be you've worked across the tech ecosystem for a long time now EA Amazon Cox. Have you ever seen a moment like this before where the tech industry has come out and said let's stop innovating or let's slow down our innovation I'm curious just put this in perspective for folks no I can't think of a time where the tech industry said you know and to your point we've seen. You know PC and you know desktop to mobile transition and different flavors of mobile to kind of phones and tablets and iPads and then the big transition to cloud that we had you know over the past you know decade or so a little more than that I guess but I have never seen one anything move quite so quickly and change the industry quite so quickly I think you can all of those you know have changed the tech ecosystem. But I think AI has changed that ecosystem so quickly from levels of investment to skill sets to you know how we're changing roles and if you know becoming more efficient and providing new capabilities and I've never seen anyone or anywhere else say like hey we should actually slow down why do you think this new technological areas so different from those past ones and why do you think there's so much fear around it I think

28:35I think I think it's fear the unknown right I think like there's an aspect of it's so new right I think even within tech companies you probably have you have individuals that are really leaning in and understanding the technology and then you have, you know you could probably segment out the population of any of the major kind kind of tech companies or any, you know, even the population of like, are you really leaning in and understanding it? And then there's a lot of fear of unknown right now and what are, you know, what are the potentials and what are, you know, what are the protections that are in place? And I think that's what's driving. It's so interesting because at the same moment, so many of us are finding all sorts of efficiencies and ways of using it, you know, I don't need cloud to engineer a bio weapon for me. I just need cloud to check my email and, you know, do new things and all sorts of interesting things. You know, we've got, keep up on all my slacks, you know, exactly, exactly. You know, we've got it doing basic website updates for us,

29:38just as an example, if we have a new funding on the site, we now have a scheduled routine that will work through cloud to add that to GeekWire's funding database. Just a quick plug if folks out there don't know, like we've got a whole set of databases that track the news that we cover, everything from funding to new startups to the GeekWire 200, and the ability to just work through our coverage and turn that information that we're reporting into raw data. So it's just my long way of saying, like, I don't, I guess I'm okay with a slowdown. It feels like there's so much on the table right now that to be gathered with just the capabilities that we already have. Yeah, there's, we haven't tapped, you know, the full capabilities of the models that are out there. Now, we want them to, you know, I think we want all of those companies to continue to make sure that they're safe and usable and have the right protections in place, and you know, see what unlocks

30:38the next versions of those models provide. We have some technologists in the audience today too, and you've really moved the organization to an AI-assisted coding company, really, in a big way. So 97% of Zillow engineers are now using AI as a copilot alongside them as they're developing. How has this changed the shape of your teams, what you get done, how do you think about this from an organizational structure? Yeah, there's a lot of theories out that, I mean, we haven't seen a lot of them come to fruition yet. We hear a lot of, okay, we are people going to move to more kind of pod-like structures or organizational structures like that. We haven't like fully kind of seen that, even though we are largely, like you said, 97%, we are shipped MR rate is up year over year, north of, you know, around 40% or so. What is MR, I'm sorry. Oh, shipped Merge request. How many merges are actually making it?

31:39And so that's fully out into production. I'm the guy who asks the dumb questions. Yeah, that's my role in the show. Okay, so you're, that, again, that shipped MR rate is, it's essentially showing how fast our code is flowing out into production at that point. So it's much, much faster. Much faster, much faster, yes. Yeah, 40% over year over year is essentially the increase for us. Are you finding that you need fewer people doing the engineering or are they in different roles? How has the shape of your team changed? What's interesting is you free up, I mean, it's the same with any, you know, kind of optimization, you free one bottle neck or you accelerate one area, certain other ones present themselves and we continue to work through those. So now, how do we get code reviews faster, right? And so if your code generation has accelerated, okay, how do we make sure that code reviews are accelerated as well? Otherwise, we inundate our developers with just reading more and more like code reviews,

32:40you know, day after day. And so how do we continue to develop a ways by which we kind of accelerate the full cycle, the full software development cycle? One of the struggles we've seen with the companies that we talk to is this idea of the individual contributor using AI and really creative and interesting way and they're kind of operating in their own bubble and they're doing amazing things. However, the real struggle seems to be and I was talking to a real estate broker here who had five people in her company and five agents that she's working with and like getting it operationalized. And across the entire organization that's helping everybody so you're not just operating on as an individual. How do you get that? How do you solve that problem? And someone's not building the same thing over and over and over again, right? Talking to your, you know, your Fable 5, you know, Fable 5-1 usage, you know, over and over again. We, there's a few ways. I mean, for us internally, we have really leaned

33:41into also a bit of a community aspect within Zillow. How to developers and other ICs is not just engineering and closest, obviously to engineering, but this is happening in other areas as well across the business. How are they sharing learnings? What worked? What didn't work? You know, we have large communities that are kind of leaning into that. AI champions are spread out and embedded across all of the organizations at Zillow so that they can kind of be that champion within one for adoption and usage and two just for, wow, how could I think about using this in a way to accelerate my workflow or improve the customer experience that we're able to deliver those types of things as well? And then we've invested in means by which we could share, for example, AI skills. So you can now build skill if you think about, I don't know, maybe a financial calculation that has to be done over and over and over, but you want one authoritative way to go and do that.

34:43Well, we've enabled our finance team, for example, to build an AI skill and then share that across the company. So we're not having different conflicting and competing financial calculation skills. It strikes me. Zillow was very early on during the pandemic in declaring remote work is here to stay, the Cloud HQ. When we talk with Rich or Jeremy Waxman, the CEO, they give us a really hard time when we say Seattle-based Zillow group. Cloud HQ. Cloud HQ, come on. And it's a big argument that we have with them, being the Seattle champions that we are at Geekwire. But it strikes me that that might have actually laid the groundwork in some ways for you to collaborate more easily. You don't have to be in a conference room to share these kinds of skills that you're developing. No, yeah, exactly. Like you said, we call it Cloud HQ. Affectionately, we call it Cloud HQ. And we leaned right into that on March 2020.

35:44And never went back. And so we had to learn how to operate that way full time. And that takes a much more intentional means to connect across, basically, across the business or connect with your teammates. Or you can't just wander into a conference room and start whiteboarding together. It's a much more intentional. And so we had to really lean into a little bit of that uncomfortableness to learn how to operate in that space. Do you have a favorite way of using AI personally right now for you in your day to day work? Oh my gosh. My life is probably sick of me talking about it every day for the last three or four years. Yours are mine, both, but no. I think probably every day I have talked about it. I think for me, I have found what might be a really interesting one or a good one that I use a lot was instead of asking for a bunch of data analysis to be done,

36:45I'm able to do that often by myself. Instead of disrupting what someone is doing because we need a data point. And I do like all of the things that I look at to maybe check by a data scientist in that area. But I think there is the ability and a bit of an unlock where I can look at some of the metrics you were just talking about in terms of AI and engineering, use of AI. Those are ones that I'm very interested in and I can basically go and pull those real time and live. We've invested a huge amount in making sure that our data is internally both secure and very available because that's really what powers all these AI experiences. Well, I want to share with you in this segues into our next segment what my personal favorite use of AI is recently and that is creating fake zillo listings which is a teaser for the GeekWire Trivia Challenge

37:48which is coming up next. Applause Can you really destroy a forever chemical? Spoiler alert, yes. What's the greenest thing you can do with your body after you die? And is limitless clean energy finally within our reach? I'm Lisa Stifler and I'm Laura Scott. We host Positive Charge, a new podcast from GeekWire about the innovations and the people protecting our planet. hopeful surprising stories about the technology taking on climate change. Positive Charge. Listen and subscribe now at geekwire.com slash positive charge or wherever you get your podcasts. Some of the world's biggest companies have brought jobs and wealth to the Seattle region. That growth has come with some real pains to you. There are these hotspots. Seattle has the potential to be one of those. Tech people are getting replaced by AI and you can't replace us with AI yet. I'm Monica Nicklesbury. And I'm Joshua McNichols. We host booming, a podcast about the economic forces shaping our lives here in the Pacific Northwest.

38:51We get into the stories that hit your wallet and our region. Listen now on the KUW app or wherever you listen. Pop Chris hot shot. This is GeekWire's trivia challenge. What would you prefer a good game? All right. Welcome back. Yes, it is that time. It is time for the GeekWire trivia challenge. It's Todd Bishop here with John Cook. Our guest on the show this week is Toby Roberts, the senior vice president of engineering at Zillow. So for this week's GeekWire trivia challenge, it is a challenge for both people here in the audience at our live show and also on the broadcast itself. For this week's question, the theme is industrial and commercial properties that were repurposed into homes. So for this trivia question, I will be describing for you Zillow listings for four extraordinary properties. But here's the catch. And this is where the trivia comes in.

39:51Only three of these are actual properties that appear on Zillow. One of them is completely fake. Your challenge is to identify the one listing that is not real. And for our listeners at home, you can try to figure it out just from the audio descriptions, just like we normally would on the show. But as a special treat for our audience here in our joint recording with real estate at work, we have passed out handouts showing visuals of each of these listings. Three of them are real listings. And one of them is a complete and very convincing fake. It was created by ChatGPT with some help from Claude with lots of prompting from me very early this morning for about an hour. So Toby, I just want to assure you that if somebody is going to get into the Zillow deep fake business, it's going to take a heck of a lot of work

40:53and a lot of prompting. All right, so does everybody get the premise? We're going to have four listings, your choice and your challenge this week is to pick the one that is not real. Anything you want to say, John? I just love how much time you spend on this today. Great one. All right, and I should say, by the way, if you're listening on the show, you can go to the show notes. I will link to the PDF from there. But I think you can still play along from the descriptions. So here we go. Four extraordinary homes. Which one is fake? A, the first is in Kirkland. It was Fire Station 24 on Fin Hill, built in 1993 and closed in 2012 when the city annexed the neighborhood. It's a four bedroom house now and the engine bay is still big enough to park two fire trucks. It sold for more than $2 million. And I love this listing. You can see the bay is basically where the fire trucks used to go.

41:54Wouldn't it be cool to live in this house if it were real? Okay, so let's see. So that is A, the converted firehouse. B, the second is in Winachee. It's an Apple packing warehouse from 1942 that sat empty after the 1980s. It's a three bedroom house now, 3,120 square feet, and the cold storage room is the wine cellar. Listed at $525,000 with a zestimate at the same price. Okay, the third C is in the Poulouse. This is South of Spokane. It's a schoolhouse built in 1908 and the last class graduated in 1958. Almost 8,800 square feet and the gymnasium comes with it. The original maple floor, a stage, and a gallery looking down on the basketball court. I want to live in any of these places. Except maybe the fourth. Yeah, we'll get to that.

42:56So it's off the market, but Zillow puts it the zestimate, which we know is highly accurate at $747,200. Okay, the fourth D is Southwest of Spokane. It is a nuclear missile base on 24 acres, most of it underground. A retired engineer bought the empty silo in the early 90s and spent 30 years turning it into a house. It's listed at $1.245 million, more than $1.2 million, and it was built to survive a one megaton blast from a mile and a half away. Now that is the total prepper's house. All right, so there you have it. You have A, the firehouse, B, the apple packing warehouse, C, the converted schoolhouse, and D, the former nuclear missile base. Let's ask the crowd what they think. Who thinks it's A? Clap if you think it's A. We'll get some audio here. Okay, that's the converted firehouse.

43:57Who thinks it's B? That's the fake one. Who thinks it's C? This is the converted schoolhouse. Okay, and who thinks it's D, the former nuclear missile base. Okay, okay. Yeah, I was thinking D to me Todd, I think I remember reading something about that. So we should ask the expert what he thinks. Toby, any ideas? These were pretty well done. So wait a second, Joe, before we jump to Toby, you think D is not the answer. What do you think the answer is? Let's get you on record here. Okay, Todd always likes to call my feet to the fire here. I think I've heard about D before. I think that's an actual thing. Although there's no zestiment showing in that, which is interesting. I don't know why there is an zestiment. Maybe you can't do a zestiment on. Because it's on market. There we go. I think to me, casual, the audience won't be.

45:00I'm going to go with, man B looks really nice. I think I can say it. You think A is fake. Okay, all right, Toby, I'm really curious to see. You've got the visuals in front of you. You know what a zilla listing looks like. What do you think it is? I actually thought I was going with A. The pictures in B look quite AI-generated. They really. Okay, okay, before you go on, I was going to ask you. So in the audience, the people who thought it was B. Why did you think it was B? Why don't we bring somebody up who thinks anyone want to come up? The represent team B. Brittany, Robinette with Real Residential. I think it's B because the photos looked excellent. And it looks AI-generated. And as a realtor, those numbers just don't match for me.

46:02So. Toby, you have some thoughts on that. Yeah, I mean, I'm, I'm, the pictures are what is kind of the most questionable aspects of that. The roof, is that a rooftop deck? That is. It's whatever you want it to be. That's what I thought. I'll go with B. I'll go with B then. You're getting swayed by the audience. Yes, yes. Okay. So let me just say that this is perhaps one of the smartest audiences we've ever had. Because the answer was in fact B. The answer was in fact B. The converted Apple warehouse. I thought the images looked real. I thought that even though no, okay, I, I got to tell you, the reason it took me so long was that the first ones looked like a clearly AI. You know, just like, it was like, come on, come on, chat, you can do better than that. I'm trying to fool a bunch of real estate people, you know, come on.

47:03Well, this has been a lot of fun. Toby Roberts is the senior vice president of engineering at Zillow. A big thank you to Toby for being here. This was great. I appreciate the invite. And thank you to real estate at work and to Lake Adave, the Seattle real estate developer and broker behind this great community. It's been so much fun mingling with all of you. And really, this is the entrepreneurial energy. I'm like, I'm ready to go flip a home. Todd, your reality go to do another trivia challenge of fake images. Sure. You have a new job here. Thanks everybody for listening. Kurt Milton edited and produced this episode. Our music is by Daniel LK called well. Until next week, I'm Todd Bishop and I'm John Cook. We'll talk to you next time on the Geekwire podcast. Thank you.