Flock is Just the Beginning: Inside the Era of AI-Powered Policing

0:00Hey everyone, it's Tristan and we're gearing up to do a new Ask Us Anything episode for our podcast. The stakes of the AI race have never been higher, the dangers increasing, we know you have a lot of questions, we'd love to hear them. Please send us your questions at undividedathumaintec.com that's undividedathumaintec.com. Thanks. I spoke with a man named Luis Martinez who was on his way to work, it was a cold Minneapolis morning, all of a sudden, massed federal agents boxed in the SUV that he was driving and he had to come to a dead stop in the middle of the street. Now this mass agent came up, told them to pull down his window and then scanned his face using his phone. Luis didn't know what was going on and the agent kept asking, are you a US citizen? Now the system that the agent was using on his cell phone to scan Luis's face, picked out his irises, the shape of his face, the curve of his lips,

1:00but it did not correctly identify that he was a US citizen. The only reason Luis was not taken to a detention center was because he happened to have his US passport in his glove compartment and pulled it out and was able to prove his citizenship. This app is called Mobile Fortify, it's being used right now by ICE. And what we found is that according to a lawsuit that was filed against DHS by the state of Illinois and the city of Chicago in January is that this Mobile Fortify app had already been used in the field more than 100,000 times in January. Hey everyone, this is Tristan Harris and welcome to your undivided attention. So the conversation you're about to hear, I recorded while I was in Berlin just a few blocks away from the Stasi Museum, which was dedicated to the history of East Germany's surveillance state. And what struck me about the kind of surveillance that's on display is just how laborious it all was.

2:00Every file had to be recorded, it had to be examined by a person, every tape was listened to and transcribed by hand, every informant was someone who had to be recruited, paid and managed. It was expensive and time consuming work. And it reminded me of an argument that my friend Yvonne Harrari made in a TED Talk in 2018 that democracy prevailed over fascism and communism in the 20th century, not because of any inherent advantage, but because the technology of that age favored open democratic societies. It was simply too difficult, too costly, to centralize power with pre-digital technology, but Yvonne warned that with the rise of AI that calculus might flip, that it might actually become far more efficient to run highly centralized authoritarian governments. The greatest danger that now faces liberal democracy is that the revolution in information technology will make dictatorships more efficient than democracies.

3:03Now eight years later, I worry that we're starting to see exactly what Yvonne was warning about. Right now, police across all levels of government from small town sheriffs to massive federal agencies are adopting AI technology at a staggering rate. And they're using it to build out the kind of efficient centralized surveillance state apparatus that you've all warned about. So take for example, you know, the recent deployment of hundreds of thousands of automated license plate readers or ALPRs across the country from companies like Flock or Axon. And on their face, these cameras are just there to help police track cars by license plates. But as you hear in this conversation, they actually are just the most visible frontier of an AI-powered digital drag net. That gives police the ability to analyze extraordinary amounts of data about private citizens. So today on the show, we've invited two experts on how police are using AI technology today and what we can expect in the near future. And what protections we need to create now before it's too late. Andrew Ferguson is a law professor at George Washington University who studies how policing technology collides with constitutional law.

4:10He's the author of your data will be used against you, policing in the age of self surveillance. And Garonsburg is an investigative journalist at the Associated Press where she leads high-impact projects on the intersection of technology and society. And her investigations about the global surveillance industry were part of a package that won the 2026 Pulitzer Prize for International Reporting. Andrew and Garons, welcome to your undivided attention. Thank you. Thanks so much for having me. So before we dive in, I just want to ask you both just to share a little bit about your background. How did you come to this topic and Andrew will start with you? Sure. I used to be a public defender in Washington, DC, trying cases in front of judges and juries. And a lot of times in my trial work, it was determined that my client was in a quote unquote high crime area. That's a term of art in the Fourth Amendment. Involved data, involves crime, involves numbers.

5:11And I thought to myself, someone should probably be keeping track of this information. And so in my cases, I would go subpoena the crime analyst to figure out whether this was a quote unquote high crime area. I would cross examine police officers. I'd commence judges about the role of data in policing. And no one else was doing this at the time. And so when I transitioned into academia, I started studying the impact of data and new technologies on policing. I happened to arrive in academia at the rise of predictive policing. So I followed that. So I'd have transitioned into facial recognition and the Internet of Things and smart surveillance. And now we're in the AI age. As the technology has changed, the law has pretty much stayed almost the same. And so my academic career has been sort of studying that puzzle between a new technology and an old law and all of the interesting issues that arise from it. And Garant, how about you? Well, so I actually began my work as a journalist living in Mexico and then came back to the Bay Area, where I did a whole course of study in statistics and economics and had always really had a data driven bent to my investigative reporting.

6:25But then in 2018, I began noticing there were just so many billboards up around San Francisco, talking about AI, the promise that it would bring to our daily lives. And I was able to take a year out to study at Stanford, where I took a lot of programming classes and really delved into AI and its intersections with civil rights, human rights, and many, many different areas of our lives. And so since that time, I've been leading investigations on that very intersection at the Associated Press. Well, and you're referencing something which is just the felt experience of driving through San Francisco and seeing all the billboards of the promise of AI. Well, I hope that what we do here is we both expose for listeners is where is this really going? What are all these technologies? How are they being applied? Where's the dark side of all this? And then hopefully have a vision as well of what would a good version of this look like that is preserving the values of open societies?

7:29So I want to start our conversation first by taking people to the frontier of the technology and people may not know this, but every year, there are these massive expos where police departments and federal agencies, they go and they get the latest tech. And I want to just kind of put our listeners at that frontier. If you were to go to one of these expos in 2026, what kinds of technologies would you expect to see and what would surprise people the most? And Garanz, I'll start with you. Well, I think it's interesting for folks to understand just how pervasive these technologies are these days. If you were to go to something like the Border Security Summit that's coming up in San Diego next month, you would see lots of vendors with little booths advertising all kinds of surveillance technology to be used on the US Mexico border. That includes surveillance towers that can be driven around on trucks, ground sensors to detect people and vehicles, biometric technologies that capture the shape of people's irises or quadrants of their cheeks.

8:31I think it's just so important for people to understand that these technologies are very much being deployed in our communities today. And this is increasingly spread far from the US Mexico border and they embedded in a lot of our cities and towns and used daily. And how about you? What are some of the frontier technologies that you see police investing in today that maybe people haven't heard as much about? I think the best way to visualize where we are on the frontier right now is to go inside a real time crime center. So if you can picture a centralized police building filled with screens and computers, camera feeds coming in and you see the convergence of three forms of AI power. The first is sensor data so automated license plate readers, ALPRs, gunshot detection. Second, you'll see camera feeds. So camera feeds from CCTV cameras, camera feeds from police body cameras from drones and the rest.

9:34And you'll see that these feeds are now going to be captured and analyzed with AI analytics. So every object within those video feeds, every sensor, every historical fact can now be recorded, separate out and analyze in ways that we've never seen before. So you can find a particular car in a particular place, you can find an object across a city, you can track a backpack across a city just by object recognition technologies. And then you have like the physical AI of robotics like drones are robots. You literally have some not terribly successful robots patrolling the streets. You see these three different types of AI converging all in a centralized real time crime center, which are now in over 300 cities in America, big cities and small. Where police have a new tactical advantage, but also have a new surveillance advantage to watch everyone everywhere all the time. I would just add to that as well, you know, one of the ways that I keep track of all of the AI tools that the Department of Homeland Security is deploying sometimes in these real time crime centers, sometimes along the US-Mexico border.

10:47Is I look at what the agency calls is AI use case inventory. Now this is a public accounting of the AI tools that you can look at yourself. And there's nearly 200 AI systems that are used currently, including some in law enforcement activities. And that includes nearly 50 AI tools that the agency itself said were high risk that are currently being piloted or in development or already out on our streets. The agency calls those tools high risk if they're key in making decisions or if they have a legal effect on rights and safety. Well, one of the things is you were speaking about that first center was that I realized I have no idea and I've heard of this call a real time crime center. Yeah, they're in 300 cities across America. It's basically a centralized command center where analysts and police officers can watch the city using video feeds. And if you watch what happens, there can be a call where 911 calls says man with a gun at a corner.

11:49And police can click on the camera that is nearest to the call and see the scene. They can fly a drone with a camera running to be able to get a tactical advantage before police arrive. And then they can click back and forth between those cameras and the cameras on the police officers chest, the police wearing body camera, and the car camera to see in real time what is happening and have a two way communication where they're feeding information about what the officer might see before they get there, which again from police department perspective is incredibly helpful to be able to have that kind of tactical insight. Not just about what you're seeing there, but also what's happened in the last weeks or months because they have historical doubt. They might know this is a location of a person who has like mental health issues. And so maybe they don't need to send the SWAT team in because they've already had three calls at the same time. And all that data is now collected and can be analyzed in different ways in different forms. So someone hearing this might say well, that sounds great. We can just better catch the bad guys. We can better distinguish that person who's been just mentally ill, who's not actually a threat from someone who's actually a criminal.

12:58What's the real problem here? How do you articulate what the problem here is, Grants? Well, I think one of the things that we've found over time is that the data that often feeds these systems contains some of our own human biases. If you're looking for example at historical arrest records, you would see more people of color in that data than you would proportionally to white members of the community. And then if that data then powers the AI system and is used to train the model to figure out where these high crime areas are, it will likely be sending officers right back to the same street corners in essence, hardening that bias into the system. I think also when you're talking about object recognition, say the model is trained to look for a gun. Well, there are a lot of things that could look like a gun that are bulky in somebody's pocket, right?

13:58And so what we've seen, for example, when some of those object recognition models have been deployed inside schools, you'll have alerts that go off. So those are a couple things I would point out. I'm sure you have other examples as well, Andrew. Yeah, I mean, there's a case where there was a video analytics system running in New Orleans and around the different parishes. And it's such that they could do what they called a virtual patrol. They go from camera to camera to see what might be suspicious on the streets and just pause there. That means they could watch all the streets all the time. And they saw a guy who they thought had a bowl turned out it wasn't a bowl, but they use that pretext to actually have actual officers stop him, chase him, arrest him. And he was eventually found not to be guilty of what they accused him of. But the reason he was tackled and tased was because of this sort of virtual patrol. But to your question of, isn't this a good thing? I think the point of this discussion is that there is a duality to this. Like we are giving police incredible new powers and they can use it for good. In the sense of there's a card jack and they want to find and get people on the street there.

15:06But that same surveillance technology can watch the abortion clinic, the mental health clinic, the protest against police brutality. Because in order to work, in order to have a real time crime scene that watches the streets with sensors and video, you have to be watching the streets all the time. And that kind of flips the default. It means that everyone is being surveilled. And 99.9% of those people are absolutely innocent. There's going to be about their business. But they're now also being watched by police without set rules, without regulations, without even a whole lot of transparency. And thus we've kind of reversed the default. We are in that world where you can be watched all the time. And the police can choose when and to whom they want to sort of target their surveillance. And the technology exists to do that now. Yeah, that was what was coming up when the grunts was speaking earlier of just increasing technologies are being used on us. The 200 technologies you mentioned, the 50 that are high risk. Most people have no idea about this. So there's not a symmetry of understanding.

16:07Those in law enforcement have way more information and federal agencies that weigh more information about us. We don't have information about what they know and how they know what the increasing technologies that they're using. How do we just continue to sort of map out an ontology of these different risks that we're trying to address here? And then as we get to the end of the episode, we talk about solutions. I want to make sure that we're dealing with each one. I would just add one more to your list of going, it is this idea of what changes when everyone is being surveilled and watched all the time. How does that chill democratic action, association, where you go and who you go with and what you're doing. There's an interesting story on social media where a gentleman who owned a gun range had to apologize to his customer because he'd bought one of these ALPRs because he thought it was good for safety. Not realizing that it meant that all of the people going to the gun range were now under surveillance. Hey Tristan here, just a reminder that ALPR stands for automated license plate reader. These are the AI enabled cameras from flock and others that have popped up all around the country in recent months.

17:12As we'll discuss, they do a lot more than just read license plates. But again, if you think about putting it together, you don't need a database of all the people owned guns if you have an automated license plate reader outside the gun range. You know who owns a gun. And you might even actually find someone who's involved in criminal activity if someone who owns that car isn't supposed to own a gun and is at the gun range. You might have just created the evidence against your clientele simply by purchasing a technology. And again, it just reverses that default or revealing who we are and what we're doing without rule sets about whether it can only be used in the cases that maybe we could agree it should be used for. And I would add to that as well, I think in a democratic society, these tools can be used in ways that really would surprise people. They can be trained to optimize for things that we might not find palatable in this country. For example, in China, some of the tools used in Xinjiang predict if a person had facial hair or a Muslim sounding

18:17name, then they would be more likely to be deemed to be a terrorist. And also, I'd say another risk is sometimes the tools just simply don't work as well in one situation as they may. And another is if we're thinking about AI powered transcription software, which many of us use in our daily lives, it works much better in English than in other languages. But yet, if that same transcription software is used to translate important surveillance intercepts, say, for example, audio in Gaza, and it mis-translate something in Arabic, well then that ends up being a big problem if that software is part of the data that's feeding a military targeting system. So I think thinking about what these systems are optimizing for or predicting on is another way in which the public can think about the kinds of risk that may be present depending on where they're deployed.

19:23Grants, there is a story from your reporting that just illustrates how this tech is being used by police, something that illustrates more of the human stakes of what we're talking about. So one thing that my colleague and I by Rantau found was pretty surprising was back in late January when the public, I'm sure, remembers the immigration enforcement push in Minneapolis. I spoke with a man named Luis Martinez who was on his way to work. It was a cold Minneapolis morning and all of a sudden, mass federal agents boxed in the SUV that he was driving and he had to come to a dead stop in the middle of the street. Now this mass agent came up told him to pull down his window and then scanned his face using his phone. We didn't know what was going on and the agent kept asking are you a US citizen. Now the system that the agent was using on his cell phone to scan Luis's face, picked out his irises, the shape of his face, the curve of his lips, but it did not correctly identify that he was a US citizen. The only reason Luis was not taken to a detention center was because he happened to have his US passport in his glove compartment and pulled it out and was able to prove his citizenship.

20:38This app is called mobile fortify. It's being used right now by ICE and I think what that pointed out to us was that, you know, even if these tools have already been deployed, it doesn't necessarily mean that they work as advertised right it was a cold dark morning. Luis's amount of color it somehow didn't pick up his citizenship. It could be that some of the databases that ICE is pinging to it might not be shared properly right but I think that what we found is that according to a lawsuit that was filed against DHS by the state of Illinois in the city of Chicago in January is that this mobile fortify app had already been used in the field more than a hundred thousand times in January. I think that encounter really kind of encapsulates some of the concerns that have arisen around the Trump administration's immigration crackdown in Minneapolis, which was described at the time as the largest of its kind and drew national scrutiny after federal agents shot and killed two US citizens.

21:41Now DHS says that its enforcement efforts are really targeted and focused on serious offenders, but what we're seeing increasingly is federal agents working in tandem with local officers leaning heavily on this kind of biometric surveillance and these interconnected databases in order to determine people's citizenship, which of course, you know, then can end up having much bigger consequences. And that would have gone totally differently if he didn't have the passport in his golf compartment. It grants other other examples that we should be worth. You know, one of the things that we were looking at when trying to understand the Border Patrol's predictive policing program was how it was used out on the highway and through going through court records in Texas, we found the case of a father in the San Antonio suburbs named Alex shot who was pulled over actually under the Biden's law. And actually under the Biden administration, this predictive policing program has been present since then.

22:44And he basically had his truck ripped apart for no apparent reason. He was pulled over by local law enforcement officer and wanted to know more about why he was found to be suspicious. So he filed suit. And as a part of that discovery, we ended up understanding more about how the Border Patrol and local law enforcement were coordinating doing what they call whisper stops that kind of. Wall off the reason for why this man and others like him are pulled over. And it was basically Border Patrol and local law enforcement were trading messages on a WhatsApp thread saying, hey, you know, I see this truck traveling southbound at this time. You know, local law enforcement, can you pull the truck over for me basically concealing border patrols involvement? And I think that what's been so interesting about that particular case is this this San Antonio man basically says, you know, hey, I'm nothing to hide. You know, there was no reason for them to pull over my truck. What I object to is this surveillance program that even concealed the fact that the border patrol was involved in it in the first place.

23:51Now Border Patrol says that they monitor, you know, driver's patterns. He had been close to the border, but there were no other indicia according to what we've seen in the lawsuit thus far that would have signaled that he should have been pulled over. So I think that case really illustrated to us that as Border Patrol's capacity extends far from the frontier, you know, it could basically be anybody who finds themselves suddenly in this kind of digital drag. One of the interesting things about that story is that it shows how the technology can be hidden from the court system. So a lot of times this technology is created for law enforcement for particular use. So we want to find a car or a pattern of travel, whatever it is, but they don't want to have that exposed in court. But of course, some of these cases do end up in court. So for example, if that truck did have drugs in it or whatever it is, it might actually end up in court. And what was interesting is why the police stopped was largely hidden, but in a court, it might actually get revealed that there was this predictive policing pattern recognition system that was part of it.

24:59And then they'd have to justify about whether it was reliable, whether it made sense, whether it had any basis there. And that can be part of the problem. A lot of times the technologies are built for particular use case, but not really thought about how it might play out in court. So maybe that system also has a scolvatory evidence or impeaching evidence that might undermine the case. And if the system isn't built to find that and surface it for police and prosecutors as it goes to the court system, you may well be creating a tool that's very helpful for investigation, but will never allow cases to go forward into the court system for a conviction because you haven't built it to withstand the scrutiny of a judge or the lawyers who would be litigating it. Back in summer of 2024 during the previous administration, we actually interviewed on this podcast immigration lawyer, Petra Molnar, and she argued that the border between countries was often a laboratory for the most is to open into surveillance technology because it had the most gray area in what, you know, what you could do.

26:00But that they would eventually be turned inward on all of us. And that prediction, how does that prediction ring true for you, Grants? Well, I think, you know, one important thing that's been happening in the last couple years is some of the enhanced role that the customs and border protection agencies could play at the border have increasingly been showing up in the interior of the country. So for example, the border patrol was once limited to policing the nation's boundaries, but the surveillance system that the border patrol runs that my colleagues and I investigated last year that relies on automated license plate readers is actually been deployed far from the US Mexico border. And pick up motorists near Gary Indiana or Chicago or Michigan. And so I think that we're also seeing this sort of quiet transformation of some of these agencies that, you know, might have been confined to the US Mexico border or maybe a hundred miles from the frontier now into something that's more like a domestic intelligence operation.

27:08And so, you know, the authorities can now monitor American cities at a scale that would have been difficult to imagine just a decade or so ago. Agents can identify people on the street through facial recognition, trace their movements through license plate readers. And in some cases, use commercially available location data to reconstruct their daily routines and associations. So, I think we're going to be able to outline what AI is doing to exacerbate the things we've talked about so far. Is there anything else you'd add to that set of equation? I think there's going to be the, you know, the next stage may well be sort of the agenda AI. It's taking the existing data that we already have and allowing police to use it. The stringent, so we've had a lot of this information before we've had the data, but it was either siloed in the sense of police couldn't connect the dots or it was too expensive. So, we just didn't have the capacity to take a police department, which is usually not your most tech savvy of institutions and have them be able to use it.

28:09Again, a great, just concrete example is if you sort of built these platforms for policing using tech companies like Axon and other companies that are sophisticated data players, Axon and other companies sell AI assisted police reports. So, the audio from a police body camera, right? So, as you are speaking to the officers being recorded, that audio can be put through a chat GPT turbo like LLM and turned into the police report that the officer reviews and signs and hands into court that becomes a document for the prosecution. And that's a good example of how AI is taking the data we had. We've always had the audio from police body cameras. That's been something that has been there since we've had police body cameras. But now we can do something new with it. We can do something in terms of generating the police report that then becomes the affidavit, the search warrant affidavit, just because it's the same documentation in a different format. That then becomes how the prosecutor evaluates the case and might in a criminal justice system where 95% of cases are resolved by police might be actually the sole source of information that puts someone in jail for a long period of time.

29:19Let's double click on the idea of agentic policing because I feel like that's a sneak preview into the future Andrew that most people aren't really tracking. And we all heard about AI 2025 to 26 is going to be the year of agents. We're going to have agents that are doing all of our work. You know, you could say start a whole business for me file the IP, create a brand for it by the website, built by the social media ads. And suddenly, you know, a thousand AI agents get to work, right? The legal agent that drafts the LLC for the corporation, the programming agent that writes the code for the website, the social media agent that does these things. So people are used to this idea of agents giving us all these magical powers. But what happens when a police force or a single police officer can fire up an army of a thousand AI agents to do this kind of long form, but would normally take a month of research and work. What can go wrong with that? Sure. I mean, I think the promise of AI agents is to create the answer machine for police investigators, right?

30:19So if you think about how AI works and the way you use it almost every day, it used to be when you like Google something, you were searching for something, you got a whole bunch of links that gave you a way of finding the information. And now when you Google something, you get the answer, right? So the same thing happens in policing, right? So if the police have all those databases, we were talking about ALPR data, video databases, people's addresses, 911 calls, gang members, all the social networking connections that Grants was talking about. It used to be that an officer would have to track down each one of those clues and sort of connect the dots. In a world of agent policing, you asked for the answer. So take a concrete example, but you have a car jacking, you have a gray card, you have a partial license plate, right? So you had to figure that out. So in a traditional world, a police officer would go look at all the partial license plate that they could identify all the gray cars, try to link them up with maybe people who might be involved in car jacking and those things. And now in a world of agent policing, you can either like create an agent to do each one of those things, which is like the simple version, or you can just ask for the answers, like give me the 10 most likely suspects of people using the database.

31:27You may not even know how the agents will go about and figure out which of the data sets are going to and bringing back your information. And like in today's world with Google, like if you ask Google for a question, you get an answer, like it is smart to go double check the links to make sure that it actually is accurate and just like find some reddit blog and give it back to you that it's actually the answer. And you would hope that police might do the same thing that they would do the double check. But what is happening is that we are having officers now like work backwards. So we're hoping that they will do that double check. We're hoping that they will make sure that of the top 10 people who might have done this car jacking that they've double check through the data to make sure the person that the agent thought was most likely was in fact most likely. And that radically changes what police do. It creates a whole host of questions about well, how do we think about error and bias and a sculpatory evidence that might be in the system really open questions that are just now being thought about or at least I'm thinking about them because I spent the summer writing a lot of article on it.

32:32But we'll be in the future because we've seen it in all these other industries like this is the future we know we have the technology and the question is when does it get applied to the policing data sets that we also know we have. And is the genetic policing something that police are using right now. I don't think they're calling it that per se although I'm calling it that but I think even to you know the wired magazine just has this great expose of how flock is using pattern recognition within its flock cameras at least has the they have the capacity to do so I don't know if they're actually doing they sort of walk back where they were doing it. But they said look we can find all of the cars in this block that have also passed the street in this street and they have these sort of pre program prompts already in their systems even at this early stage which means they were thinking about it like months ago or whenever they created the system it's not years ago. And so we are seeing the seeds of this being built out in part because it's what police do anyway if you ask an off stick okay get a car checking what are you going to do they can tell you the five steps that they take on every carjacking and there's no reason why you can't create an AI workflow that sort of automates that in a way that is doing just what they are doing and have been doing for decades it's not rocket science to recognize that this is what police will be doing because is what they have been doing.

33:52What we're seeing now is companies are kind of putting it within the architecture of their systems because now AI can make useful all the data that has been collected. So I just want to stop here for a moment and really absorb what Andrew's talking about here. There's there's one surveillance story that's all about you know data tech companies data brokers the government they have a mountain of data on every citizen and now they can use AI to analyze that data and really get the data. So I think it's a real time to build out a surveillance network. But I think Andrew's pointing to another version of a surveillance story that's actually all about AI agents where you enter into a world where police and law enforcement are going to have access to thousands of agents or analysts that they can deploy to do their surveillance for them all running at superhuman speed never sleeping never complaining. So I think the classic detective's court board you know there's the person sitting there the detective and they've got a court board up and they've got photos and pins and red yarn that's running from one node to another.

34:57And up until now a human being had to sit down and make all those connections you know one of the time over weeks or months or years. But now imagine that there was a thousand detectives with millions of red yarn strands connecting every possible point to every other point. Today police could search for using flock you know a blue pickup truck with a kayak in the bed. But in the future they can have an AI agent search for a blue pickup truck belonging to someone who voted a particular way whose dog has a microchip registered at this address whose phone was at this intersection last Thursday. So this isn't just a story of more surveillance. It's a story of a different kind of surveillance entirely. Because for most of human history the constraint on surveillance wasn't law it's been the expense and time required to do it. And once that constraint is removed with AI it's going to create an entirely different paradigm shift. So just want you to hold that in your head and we proceed with the episode. So far we've been talking a lot about the United States and how technology has been applied here.

35:59But Garons I know in your reporting you've written a lot about how American companies have actually been building out surveillance apparatus for the rest of the world. Ironically we talk about China being this is to be in the thing and will never build that here. But the US may be helping to build that surveillance state in a way. Yeah I mean I think particularly in this moment of the AI race between the US and China was important for us to pick apart some of that history of how these. Not only AI models were built over time but some of the very basic computing infrastructure that also fueled surveillance networks. And it turns out that American technology companies notably IBM to a large degree actually designed and built China surveillance state. So some of our reporting uncovered just how Silicon Valley had played a greater role in enabling human rights abuses than was previously known. We also looked at how the American Congress had through democratic and republican administrations alike really failed to curb American firms from selling technology to Chinese police government agencies and surveillance companies.

37:14And then also we took a look at how AI is now powering warfare. So to look at how models that have been developed right here in California have been used in the war in Gaza to help make targeting decisions as well as how some of the surveillance technology has later been turned back into the United States sort of with new Chinese amalgamations and used to track down dissidents. Here in the US, so I think part of what's been important is to understand how systems are maybe piloted in one particular location in the globe and then adopted much more widely after that. These are really global technologies and so just because they surface in one place for one use case doesn't mean that they can't be applied or you know, misapplied in another kind of very troubling scenario.

38:17Which gets back to the power of these tools divorced from their context normally its laws and norms and transparency that protect us from those tools being misapplied in context to create worlds that we wouldn't want to have. And it gets back to the question Andrew of what are there for the protections norms or transparency measures that we need in order to deal with the range of threats that we talked about so far. Yeah, I think that in addition to constitutional protections where the courts can continue to sort of strengthen the fourth amendment in response to these new threats and again just to keep talking about ALPRs there's ongoing litigation about whether a system of ALPR surveillance these license plate reader cameras is a violation of the fourth amendment because it reveals location data. And where you are and what you're doing whether you're at a health clinic or a gun range it reveals where you are and Congress or states could easily take a lead in sort of regulating this regulating either requirements for audits or transparency which we don't even have in most of these situations we don't even know if AI is being used.

39:24Two states have regulated whether AI is being used in police reports Utah and California but again having knowledge about whether AI is being used or at least being relatively easy and benign for step and then you could set up a warrant requirement by statute you don't even have to go to the fourth amendment floor of the constitution and say look you know this is obviously a very dangerous technology that could be misused it has good uses but maybe we won't only want to use it to find serious issues. We could have a system of laws in place that require that kind of warrant requirements say hey look you want to go figure out which car was at this location this time good comments a judge why you need it. It's not perfect it might not be all that protective but at least it's something that we don't have right now so that's just a couple ideas about how we can sort of update our laws and rules and regulations around this technology which we haven't done. I'm sure you speak to a lot of legal experts and advocates in your work what was you at what have you been hearing from them about what we need as well.

40:30I think you know at this point AI has gotten increasingly political right and so the current administration is not in favor of those kinds of state and local AI regulations and I think we're going to see that playing out as the administration matures in its AI policy. We'll see what kinds of restrictions may come up at a federal level. But I think again what's been interesting in some of the reporting that I've done is to see that these very personal issues of privacy should authorities know where I am as a result of where my car is sometimes do really unite both Republican and Democratic lawmakers. For example, after the Border Patrol investigation I mentioned we had lawmakers from both sides calling for more scrutiny of the Border Patrol's use of this ALPR predictive policing system. And I think that the way that flock safety one of the companies at the center of his public debate frames it is that it's all about privacy versus security.

41:40How much privacy will you give up in order to get safety or security? But of course that looks very different for different people. What safety feels like in one community may be very different from another. And I think that what's been interesting to watch is that this is an area where Democrats and Republicans kind of come together. We found both parties tend to value privacy a certain amount which is why this is kind of kicked up in recent months and weeks. But of course automated license plate readers can be linked up to all kinds of other information that track people's shopping habits that track the driver's licenses of the people who the cars are registered to their voting records that track their citizenship and the public. And so it's kind of this I think it's become a touch point in part because in the United States many of us have cars and that's often seen as like a symbol of freedom.

42:43But in fact it's just one way in which we're all being tracked through this mass surveillance. Do you think that there is a moment now that because maybe ironically because of AI and because people are afraid of AI the same surveillance technologies that have largely been in existence for the last five years are getting a fresh look. Honestly we've had ALPR systems for a long long time they gave tickets and DC for over decades. But the fact that AI has changed the conversation in terms of a new fear, a new power it has given some of these systems of surveillance a different look. And I think adding to that is the overreach by the federal government in the context of immigration. I think the flock debate would be different if we had not seen ALPRs used to target protesters and Minneapolis. I think everyone realizes that the surveillance aperture could be turned against them in the future.

43:47Our cars, our smart cars are connected cars are also revealing that exact same data. If you are driving any modern car you're being tracked everywhere you go. And the Bluetooth signal from your car, the GPS that allows you to press a button and get emergency services. Guess what? That's all trackable as well. And so it's interesting to me that we've had this new moment of a particular company and particular technology which actually had for years now. We've had ALPRs for years. Has sort of created a spark where people are starting to push back on this idea of mass surveillance. Even though there are so many other places in our lives, like every smart device you own is a surveillance device. Like we're putting wiretaps in our home called Alexis and we have ring doorbells that are capturing ourselves and all these things. And so we are actually trackable almost everywhere we go because we have these wonderful digital technologies everywhere. And ALPRs are a piece of that puzzle but it's only one piece.

44:50Well, and there may be a response by the sinners of like, yeah, well, I have a fitness span and it tracks all these things for me. But that fitness span, this business model is not to sell that data to other surveillance companies. It's interesting how this actually interlinks with other new AI risks which is that the assumption that the data that is trackable is that the data that is tracked. About me is safe is suddenly completely in question because you have AI systems like mythos that can hack into literally to classify systems of the United States government have been demonstrated as such. But you don't even have to get there. Every single thing you've created in a digital world is available to the government with a warrant and sometimes without a warrant. There is nothing there. I just wrote this book called your data will be used against you on the takeaway is there is nothing to secret to private that the police cannot obtain. With a warrant they can go you know your smart bed your period tracking app your car the smart heart sent in your heart detectives can go to your doctor and with a warrant and get that information use against you in a court of law.

45:52That's the rule. That's the law as exists and we don't think about that way but it's part of it. And so all that exists now and when you add AI which just allows you to make more use of these digital trails that exists everywhere we really are in a very vulnerable situation these are the government. What are some of the tools that this should know about today to protect themselves so I'm thinking of things like deflock which I believe is a map of where flock is deployed so that there's actually some transparency because most people just don't even know that there are flock cameras in their town. What are some other tools that people should be aware of to protect themselves and fight back. One thing I think it's interesting about the deflock movement and the maps that have that is that I think part of the reason people are reacting to that is that it's like a tangible visualization of a fear about surveillance. You literally can see the camera that we have cameras all over the place right behind those AI billboards in San Francisco or whole series of cameras in downtown San Francisco they go directly to the real time crime center in San Francisco including all the

46:53bells and whistles of new policing that are there. But it's hard to see that you don't actually see that as a physical manifestation of surveillance and these cameras just symbolically are standing in for that fear. So I think one thing is to open your eyes about how vulnerable you are to this world of self surveillance whether that's democratically mediated self surveillance meaning that it's your tax dollars that are paying for those cameras. It's being done in your name and because it's being done in your name it means you have a rule to do something about it you actually can call your congress people it is fast into me as someone who studied this whole industry for over 15 years that they're ordinary people in city councils who are taking the time of their busy lives to go protest a surveillance camera. And I think part of that was education we saw a movement of educating yourself about how these cameras were how they can be misused and I think if we expand that to other areas of new policing technologies we might have a stronger conversation and movement for our elected leaders to do something about it.

48:11So we spent most of this conversation focused on what's been going wrong with AI and surveillance but now I want to ask you both the opposite question which is what is an alternative vision for our open societies and AI policing tools would work in a way that's not dystopian but actually desirable you know not this sort of Robo cop or minority report vision but the vision that I don't know if we actually have so Andrew one is to start with you. It's not an easy question because the system that could be designed requires a lot of trust in how police will use that power on that new power and many communities do not trust the police to use that power wisely. And the concern is that if you build a system that can be used for good reasons and sense of like finding the murderer or whatever in the city it can also be misused if the category of crime you're targeting say people who criticize the president gets changed and that's what the dangerous that's really where the concerns are I do think there's a compromise that could be made.

49:24Even though I also think that that compromise may very well not work as well as we'd want which is to essentially assume the worst and begin from there I end my book saying that we should focus our attention on police surveillance technologies by envisioning the tyrant we should have I call the tyrant test and we should picture the tyrant reading your most embarrassing Google search and then begin from there like what would you do to do. You protect that information from being used against you in a way that is either you know criminal or embarrassing or a way to silence you politically what steps would you take and there isn't an easy answer but it has to be an all of the above approach including legislative changes like I discussed from legislators. Court decisions that enhance or the privacy of our information and our communications has to be sort of community base because it might be different in different places people have different sort of a sense is about their levels of privacy and their trust in police and you have to kind of have an all the above approach and you know with checks and balances and if that sounds sort of familiar well you know America was kind of built on the tyrant test because we realize that trusting power and one king was a problem.

50:41And so we build a whole system that tried to limit that power I think we should do the same thing with policing technology and policing information that police can use or misuse to easily right now. Grants have you seen any models of a police that would actually help serve justice and protect our rights. You know I think one thing that I keep coming back to is body worn cameras so body worn cameras were rolled out if you remember after the protest in Ferguson in 2014 president Obama at the time said that that would bring more transparency to communities who are really trying to understand what police were doing so that was really held up as a way for communities to engage with the actions of law enforcement. Now if you add AI powered facial recognition to those body worn cameras enabling police to identify people on the street that brings in a whole host of other privacy concerns right.

51:42I was going to mention a story that was not part of the big surveillance package last year but that I thought was important which was about the city of Edmonton in Canada that decided to run a pilot of live facial record. I think that was a lot of the time that I was able to identify facial recognition in body cameras so that was with the company acts on enterprise and it's thought to be the first time that a police force has done live facial recognition inside body cams and I think that that raises a lot of really interesting questions because the body cams of course were introduced to bring more transparency to policing and now there's a sort of novel use of those very cameras that were meant to bring people to the city. I think that was a lot of time that I was able to bring about more community transparency right. Canada you know of course was one of the first countries that had an AI strategy back in 2017 but I don't think that the AI strategy quite contemplated exactly what happened in Edmonton right so I think that one of the things that I think is important again is just for the general public to keep pace with what law enforcement tools are being used where

52:56they're being used you know wet law enforcement says those tools will accomplish and to really engage in that conversation because it can change from you know Ferguson Missouri to the city of Edmonton in Canada to very different uses for essentially a similar technology now with AI layered on top of it. One other interesting point on AI assisted body cameras is you actually can enhance police accountability. I just wrote an article that you're in a Schwartz from UCLA that says that the ability to watch police encounters with citizens as they go about their daily business can be automated through AI to find problematic behaviors of police. There has been a rather difficult path to enforce civil rights lawsuits against police for police misconduct brutality whatever is gets hard to find a pattern and practice of police misconduct but companies like Trulio and other sorts of companies that sell AI analytics for police body cameras can actually identify those very problems automatically.

54:05And so in one of the early studies essentially every time the officer used a racial epithet or a curse word or used handcuffs or there's a use of force it would be picked up by the AI and so all of these like millions of hours of body camera footage could be isolated to find the problematic behaviors that was afforded to the supervisor. So the supervisor had noticed that their officer had gone rogue and they could look into it for police accountability for better training in the rest and that's a use of AI to improve policing through the same technology that exists that might be a way of also improving the way police practice works every day. That's an interesting example because I think that it all depends on how technological progress is defined you know how these tools are built to optimize or predict actually shapes these very real world outcomes if you have a tool that is aimed at bringing more accountability to police pattern and practice as opposed to analyzing historical arrest data that may have biases built into it that's two very different use cases for our people.

55:18So it's arguably a similar tool right so I think that it's in those kinds of questions that will continue to grapple with these bigger issues of civil rights democracy power and accountability. Yeah I mean it's sort of the classic who watches the watchers and this is sort of essentially AI that helps watch the watchers. You know this is also bringing to mind Estonia which has some laws that could provide a decent model for this they've had a data tracker apparently since 2017 where citizens can see which agency access their data and for what reason. So citizens basically are not passive users of services they own their own data and unauthorized access is actually a criminal offense and their criminal code also has a post surveillance notification requirement so once a surveillance authorization expires the agency must immediately notify both the person that was surveilled. And any person whose private or family life was significantly affected and it was identified during the proceedings I'm just curious if either of you have ever heard of thought about this example.

56:18What's so fascinating about that is that we have the technology now to do that in fact when flock got pushed back they just changed their sort of default policies about LPRs and one of them was to say you know we actually have this audit system that can look at every time the police are using this for inappropriate ways and we can catch them and fight. And if they do so and the question of course as well why weren't we leaning into that to begin with right and maybe the answer that we don't have a law that's on the books we're sort of reliant on the tech company is to sort of make sure this audit ability is enabled and used. And I think the way so interesting about that set of regulations in Estonia is that it really sounds like it empowers the public to understand how these technologies are being used whereas in the United States mostly what we have is opt out so you can decide that you don't want your data shared with xyz agency or xyz tech company or ad broker but that doesn't actually give you the information of how.

57:23You know this data is otherwise being used which in Estonia sounds like you're just getting much more transparency into how these systems work which in turn makes for a better informed public. I love that example just by having transparency and more of this visibility it's actually educating the public about the nature of what is surveilling them and the kinds of information that can be inferred from that one sort of basic rule of principle that came to mind when thinking about this is. Is the AI tool or the technology that's being employed is it used to check a human decision or is it used to make a human decision and I feel like that is what least one distinction as well that can can help with steering us towards that more pro human democratic open society that we want to see your towards. Well, Garant's and Andrew thank you so much for coming on your divided attention and lightning everybody about how these technology trends in the law are intersecting in this new world thank you so much. Thank you pleasure to be with you.

58:25Your divided attention is produced by the Center for Humane Technology we're a nonprofit working to catalyze a humane future our senior producer is Julius Scott our executive producer is Josh Lash mixing on this episode by Jeff Sudier. And a special thanks to the whole Center for Humane Technology team for making this show possible you can find transcripts from our interviews and bonus content on our sub stack and much more at humane tech.com. And if you liked this episode would be truly grateful if you could rate us on Apple podcasts or Spotify it really makes a huge difference in helping others discover this podcast and join the movement for a more humane future. And if you made it all the way here let me give one more thank you to you for giving us your undivided attention.