AI for Litigators: Leveraging AI

0:00Hello everyone, welcome to today's program, AI for litigators, leveraging AI. I'm alla Policaster, Director of Education and Professional Development, and today I'm delighted to introduce our speaker, Sam Davidoff. Sam is the CEO and founder of Align, a digital binder platform he built to give litigators a paperless, secure workflow from intake through trial. Before founding Align, he spent nearly two decades as a trial lawyer, handling complex civil and criminal litigation in federal and state court. We're so excited to hear his insights today, so I'm going to turn it over to him. Great, thank you very much. Yeah, so that is the brief introduction of me, and I was a partner for a long time, a trial lawyer at a law firm Williams-Iconnolly here in Washington, DC, which is where I'm based. But about actually almost seven years ago now, I started a software company first doing

1:03the binder's product that was mentioned, and then we actually now also make an AI-based legal research product called Align Research. Really excited to talk to folks about AI today, and I'm going to share some slides. But just to say that I, if pitch this presentation, sort of with the idea that it is for litigators who are, you know, anywhere from starting to use AI to, you know, having used AI and become more sophisticated as AI users. And so if you're on the latter end of that spectrum, some of the stuff, I'm going to go over some basic concepts and bear with me on that. But I'll also get into some more advanced things. And one thing I would say is if questions come up during it, I think you folks all have access to the chat and can put questions in the chat, please do. And I'll keep an eye on the chat, and if things are questions that are topical to what

2:04I'm talking about, I'll stop and address them. And that's not quite having the conversation, but it will make this more dynamic. And I really appreciate it if folks do that. I love trying to explain this stuff or address real questions that people have on the ground. So feel free to do that as I'm going to just put that in the chat. Okay. I am going to share my slides. And hopefully everyone can see that now. Okay. Great. Okay. This is what we're going to talk about. I'm going to talk about some strategies for using AI as a litigator. And this is going to be platform agnostic. In other words, the strategies I'm going to talk about are general to the use of AI, specifically, large language models that are what really all the AI tools these days, or nearly all of them are using. And these kinds of strategies will work if you are using a legal specific tool, like a

3:05Harvey or a LaGora, or if you're using a generic commercial tool like Chatchy PT or Claude or Gemini or any of those. I'm not going to speak directly to particular products, except a couple of little places I might offer a couple of tips on things that work. I know Harvey pretty well. I know LaGora less well, so I may offer a few tips on that as we go. Okay. So let's start with some basics. And just to sort of level set on AI what AI is, let's start by some basic terminology. So first of all, when we talk about AI, AI these days is used as a term for really what is sort of more technically, sometimes called generative AI, or it is really machine learning models that have been trained on huge amounts of text.

4:08And these are called large language models. And the models are sort of the kind of black box secret sauce that you access when you access the Chatchy PT or Claude are also when you access one of these legal specific tools like Harvey or LaGora. So the models have names like in Chatchy PT. Now they have names like Soul, and Terra, and Luna, and Claude, you might have heard names like Opus, and Sonnet, and Fable. There are other ones. And these are the names of this sort of trained set of data or technically vector embeddings that when you give them a prompt, they generate text in response. And that can be as simple as you ask question, and it generates text answer, like we're sort of familiar with from Chatchy PT going back to the very early days, or it can be more sophisticated that you give it a document and it generates back the text of a new document.

5:13But in general, the general concept of the model is you put text into it, and it gives text out of it. Maybe with a little qualification that on some of the more advanced ones, it doesn't have to be text, it can also be sound and pictures in video. And they can some of them can even generate sound and pictures in video. But the idea is you put something in, and the model gives you something back. And what it gives back sort of depends on what it's been trained on, but as we've all seen, it's pretty impressive what they can give back. They give back things that look a lot like what a human might give back to you. So those are the models. Now I want to distinguish the model from what is often called the harness, which is the tools that kind of go around the model. And these are the applications that you actually interact with, like a claw desktop or a clawed co-work or Harvey in the case of Harvey, Align Research, My Product, could be other things

6:14like co-pilot in word. And what these things are all doing is they are wrapping around the model. But they are sort of in a certain sense kind of giving the model a, you know, it's like putting the model in a battlesuit or something like that. It will take in your question. It will feed the question to the model. But it will also give the model the ability to do things like reach it. Again, if it's enabled, reach out and access other materials, search the internet. Maybe have the model call another model. Maybe have the model reach out to a legal research database. Maybe have the model interact with Microsoft Word and edit documents. And these concepts are kind of, it's often useful to keep them distinct and understand that the model is sort of the engine, the thing that is able to understand text and generate text. And the harness is the thing that allows the model to sort of interact with the outside

7:18world. It will be getting information from you, getting documents from your system, searching the web and doing other, even more sophisticated things like writing programs and running programs, things like that. And so there's both things are very important. Which model you use is very important. And you sort of hear press releases over time in terms of, you know, a cloud released a new model. They released Fable 5. They released Opus 5. You know, a GBT released a new model. But they also important are the tools that go around them. Harvey is a tool that can work with a number of different models. And you can select different ones when you're in it. And the same is true with, with LaGora, with co-counsel, with things like that. They can work with different models to maybe make things even a little more confusing, but also illustrate the point. You may have seen recently that some of these companies like Harvey and Thompson Reuters that have make these tools have actually now started releasing their own models.

8:21And that's a whole other subject. But again, illustrating the point that there are these two different things. There are the models. And then there are the harness or the tools that go around them. The prompt is a generic word for the text that you put into it. And the question that you ask or the things that you are following up with the model. But the prompt can also include the things, documents that you put in or whatever. Context is a sort of more general term that is used to talk about everything that the model has access to when it goes to generate a text. So these models in general, they don't really remember things. When you have a writing conversation with the model, each time it gives you an answer, it's actually sort of taking the whole prior part of the conversation, your original question,

9:23its original answer, your next question, its next answer. It's taking all of that, looking at it all again, and then giving you the next answer. It might also be looking at more than that. It might be looking at the whole history of your conversation with it and some files that it's found by searching the web using the harness above and files that you've provided to it by uploading them to chat GPT or by accessing them in a Harvey Gold. It's taking all of that and using that to generate the next answer. That sort of whole bundle of material that it's looking at is referred to as the context. It is all the context that the model has when it goes to generate the next thing that it's going to save you. It's useful to distinguish a team prompt in context because one important thing that I'm going to talk about is while it can be important to how you phrase your prompt, how you ask your question to the model, what's actually much more important is the bigger

10:26picture, not only how you phrase the question, but all the other material, all the context that the model has available to it. Those are some basic AI terminology. I think that's enough AI terminology to certainly get you started and to converse on most things that lawyers using AI will need. I'll stop there, but I may use those terms as we go through. Let's first talk about things that are risks in AI usage. As lawyers, what are the dangers? What are the problems that we face in using AI as in terms of actual legal risk exposure that we may have in terms of sanctions in court or anything from malpractice claims to just concerns or complaints from our clients? It also to just risks about things that are annoying, things that make us inefficient, things that make us liable to make mistakes and things like that. The first one that if you've used AI at all, the read about AI you have almost certainly

11:30heard about is hallucinations. Hallucinations is the idea that the model will say something to you that a message isn't true that is made up. Hallucinations are really kind of an inevitable part of how large language models work. Large language models work by they take in all of the context that you give them and they then use their model to generate a response that is essentially what their model dictates the response should be based on all of the entire context. There is nothing in that process that has any concept of truth or reality that's just not sort of a thing in that machine. The machine takes in text and it generates other texts. There's a lot of things that are done to try to make that reliable, but there is no sort of a concept of saying to it, don't lie, it's not a thing. There is always some risk that when it writes its response, what it writes isn't correct.

12:35And in legal, that risk is particularly pronounced in a few places. And one of the things I attach to the materials are some cases that talk about them and there's unfortunately these days hundreds if not getting on thousands of cases that discuss problems with hallucination and briefs that people submit to court. But these can span the range from the obvious things and that have gotten the most press of an AI writing a brief or writing a memo or writing a complaint and citing a case that doesn't exist. But it's really important to realize that hallucinations can go a lot further than that. It can be and be things that are much more subtler and harder to detect. It can be an AI citing the case that does exist, but with a quote that doesn't exist, putting a quote in the parenthetical that doesn't exist. Or even more subtler, AI can cite a case that does exist and not have a quote but have

13:36a summary in the parenthetical that isn't really what the case stands for. Or even subtler still, it could actually cite the case and have an accurate quote from the case. But in the discussion of the case that it writes in the paragraph, it can make an argument that kind of goes too far. And these are the kinds of things that need to be aware about it. And it doesn't only apply to cases. The same thing is true if you ask an AI to summarize a document. If you ask it to find things for you across documents, what it writes back to you has some increment of untrustworthiness. And that is basically, at least in the form of sort of response, you know, whenever an AI writes something from you, there is no way you can guarantee 100% it's correct. And so you need to take that into account as you're working with AI. So that's one risk, the risk of hallucinations. Second is model competence. But what I mean by model competence is just the notion that these models are trained,

14:39the way they are trained in essence is by having them suck in enormous amounts of data. At this point, the frontier models that are really what we're talking about, the models created by OpenAI and Therapeutic and Google, have read pretty much every piece of thing available that's been written by humans ever. And a certain sense learned a lot from that. But they, but you know, they have a cut off on their training date. It's only up to a certain point. There may be things that were excluded from their training. There may be things that they were asked to focus on more. So there's kind of a limit to what they have. And you could sort of see that, again, now models search the internet using their harness. So this happens last. But in the early days, you would see this of, you know, you could ask a model a question like, who's the president now? And it might not know, especially if you asked it around the election year because that wasn't part of its training data. Again, this is important for legal because it means that when you ask even a very sophisticated,

15:43the latest greatest model released by your provider of choice, when you ask it a question, if it doesn't have anything more to go on, if it doesn't search the internet, if it doesn't, you know, you're not giving it other documents, the answer it gives is going to be bound by its training data. In other words, at a certain point, it can be as smart as it is, it, there is a smart and fluid as its answer seems to be the reliability of what it gives you is going to be limited by sort of what was it trained on and what it should to access in that training. There are also differences between models that is something that people focus on. My own view is if we're talking about sort of the top level models that, you know, the programs from Claude or Anthropic or from Harvard or LaGoura give you, those differences are unlikely to make a huge difference to you as a lawyer in your daily work. They're all going to be pretty good, but they're also all going to have these limitations that we're talking about. Another risk is the risk of psychofancy.

16:44And what this really is and anyone who's interacted with these, you've seen this, right, when you talk to these models, they really want to be helpful. They want to be your friend, they want to encourage you, they want to say that your ideas are great. And this is actually a feature of how they are trained. So part of the models training is that they suck in all of this, you know, enormous amounts of data, but then part is a, what happens after that is the creators of the models have people go in and sort of score how they answer questions and give them essentially reinforce answers that are desirable and the suppress answers that are not. And the result is the model's kind of developed what appears to us to be a sort of personality. And the personality of models in general is they want to be agreeable and helpful. This is a problem for lawyers because it, that's not always what we want.

17:49And we are trying to do legal analysis. We, you know, in the days before AI, I think of an example, I would always think of when I would hire an expert witness. I didn't want the expert witness who said, yes, of course, your damage is serious, right. Of course, your client is correct. Of course, you know, we can, you know, we can show causation the way you, you've said it. Those experts made me nervous because I thought, are they just agreeing with me because, you know, I'm paying the bills. That's not what I wanted. I wanted the expert who said to me, you know, I read your damages, Terry, and it doesn't actually totally work. And you're going to have to change X, Y, and Z, right? That's often what you want, you know, in legal or if you ask an associate to, you know, or a colleague to read a case and tell you, can we use this case in our brief? You want, at certain times, you want that person to come back and say, no, we absolutely can't use this case in our brief. It doesn't stand for the proposition we said. AI is not great at that. It has this desire to please, and I've actually done some experiments, which if somebody wants

18:52to reach out to me afterwards, I'd be happy to share there. I've made them public where, you know, if you, if you've given AI a brief and a bunch of cases and you ask it, you know, can we use these cases to argue a point that we want to make in this brief? Its tendency is going to be to tell you, yes, you can, and to try to find some way to make what you want to do work. And that's often dangerous in the law for reasons that we know. So again, this is another risk, and it's different from the risk of hallucination because it is the model sort of being overly agreeable with you and overly trying to sort of lead you in the path that you want, but that may not be right. And then the fourth risk that I want to talk about with AI is what I call context raw. I don't just call it that. This is a term that's used. And it refers to a kind of feature of AI that as their context grows, and let me pause and say, how does their context grow? Well, the context grows in the first place by just you continuing to chat with it, right?

19:54So you ask an AI a legal question, let's say. You say, here's my situation. You know, is there any law in California that would prevent my client from being able to do x-blinds? And then the AI kind of gives you, based on its model, you know, some unanswered. And then you say, well, read this California statute and tell me if that changes your answer. And it does that and it gives you an answer. And then you say, well, can you put that into a memo? And it produces a memo. And then you say, well, you know, can you edit the start of the memo to do that, you know, so that it addresses this more? And the memo doesn't really address that. And so it gives you another version. What's going on as you go through this iterative chat process with the model is all of that, that context is just continuing to build up, right? So it's not like when a person chats where, you know, we sort of have an idea of where we are in the conversation and we're not thinking back to the first thing that was said every time. The model is literally at every stage in the conversation, effectively rereading the whole

20:56thing and then giving you the answer. They don't have any concept of state. It's always, every answer is just kind of reading the whole context and giving you back. And so as these chats grow and length and as more things get added to them, not just your chat, but the documents that you put in, the things they pulled from the internet, there is a phenomena which is, and you, which is that they get worse. They start to forget the things in the beginning. They start to be less good at remembering instructions that you gave them. And this is a phenomenon that's been familiar, I think, to anyone who's engaged in a long chat with one of these things where, you know, you ask it to write something, it writes a few paragraphs, you tell it to edit the last paragraph it does, you tell it to make another change, you know, it does that, you then ask it to edit the first paragraph and it undoses the edit from the last paragraph or something like that, right? These kinds of frustrations, what you're experiencing is a form of context, context rot. And again, what's important to understand about this is there is a part of this that is

21:57just inherent to what happens with these long contexts. Again, like the hallucination risk, these are things that are getting better over time and are ameliorated as new models and new tools come out, but they're not eliminated. And so again, it's something to keep in mind that these lengthy chats, lengthy things have a cost to them. And okay, so these risks are, I think, general risk of using AI, but again, they all have particular application in legal and you want to think about them as you are going to work with these tools. So what I want to do now is talk about what are some strategies that I have found to work, both in my own work and in work that I do. I also work on a consulting basis with a number of law firms and can share in general things that I've seen and strategies that I've seen to be effective in trying to counter this risks and get the best results out of your AI tools, whatever they may be.

22:59And for each one, I'm going to give some an example that I hope is meant is reasonably real world in terms of what you might encounter in the day to day of litigation. So the strategies I generally put in the category, these five, and let me just go through them one on one rather than go through the roadmap. So first, don't chat, plan and execute. And this is really meant to address the context problem, but it also in general, I think it also helps with some of the things like hallucination as well. And let me illustrate it. So let's take an example where you've received from opposing council a set of requests for production. As we're all familiar with, it's one of these requests for production that as almost they all do these days, you know, starts with a laundry list of definitions and then a laundry

24:03list of instructions. And then each RFP starts with some little preamble and then there's the little kernel of the thing that you're actually being asked to produce. And then maybe there's more language and additional instructions in there and whatever. And the thing is 80 pages long. And what your client wants to know is like, okay, what are we actually being asked to produce? Okay. And you know, a common task to do would be to create some kind of summary chart that sort of boils this down to here are the key categories that were being asked to produce documents in, under each category here are the actual requests and the specific things that we want. You know, here are requests that overlap, things like that. So you go in and you approach you might think would work is you go into the AI and you say, okay, here are these RFPs I just got from opposing council. Can you give me a summary of them? And it goes and it gives you a summary. It says these are the categories they're asking. You know, this is what RFP want is about. This is what RFP do is about.

25:04And you say, okay, that's not exactly what I want. What I'd really like you to do is rather than just summarize the, you know, boiling down what each RFP is, categorize them not by the number that the other side assigned, but organize them by topic. Right. So give me the ones that relate to personnel, give me the ones that relate to, you know, financial statements, etc. Whatever. And it says, okay, here are the categories of documents. You know, here are the RFPs under each one. And then you say to it, okay, now does this uncover every RFP? And it says, well, let me look. And may, you know, it's spinning wheel goes for a little bit. And then it says, okay, actually here is all of them. I had missed some. And so actually there's four categories. And, you know, and now here's the newly organized thing. This is the kind of back and forth chat that you get into with AI's if you approach things this way. And then that result is you end up with something, you have a lot of frustration. You sort of feel like you're fighting with the AI. You get something back that maybe is kind of what you want.

26:07And you're not totally sure if it actually covered everything. And you need to go back and check it. The needing go back and check it thing is going to be there no matter what. I'm not going to give you a, there's no secrets also for avoiding that. But a strategy that I would recommend in a situation like that is not to engage in a chat with the AI, but instead to kind of follow this plan and execute way. And so instead, how I would approach something like this is to say to the AI, first of all, think through in your own head. What does it you actually want and give it clear instructions and say, and in other words, don't think about this as you're going to engage in a conversation with the AI. Think about it as you are giving a task to a junior lawyer or a paralegal and assistant and they are going to go away for two days and do the task and come back to you. That's not what's going to happen. It's going to happen very quickly. But think about it in that way, right? You're trying to give them a set of instructions that they can then go and execute on their own. So write it out like that.

27:07Here's what I want. I want an organized summary of these RFPs by category rather than number. It must comprehensively cover all the RFPs. In each category, identify the numbers that RFP numbers that follow that category as well as the description of what they cover. And then here's the key point. Instead of just saying, now go do it. Tell it, give me a plan of how you're going to do this and what it will look like and then come back to me for approval. And what it will do is it will then take all this. It will read the document and it will write out for you a plan. Here's how I'm going to go about it. And here's the output I'm going to give you. You'll read that plan and that may lead you to think of some improvements. I actually don't like the way it's approaching this. I forgot to tell it what to do with the instructions. I forgot to tell it what to do with. But you're going to be much better off revising that plan with it and then telling it to go execute that plan, then sort of trying to iterate back and forth on the work product that you're

28:08having to create for you. I would say that iteration too can be a little dangerous for the same reason. And so if it comes back to you with a plan and you say that plan's pretty good, just, you know, I make sure that the headings are bold. That's probably fine. If you get into a back and forth with it over the plan, you're going to end up in the same problem. But the easy solution there is to get back and you know, get in the back and forth with the plan, come up with a plan that you want and then start a new conversation. Take the final plan that you've given it and open up a new chat and say, okay, here's my RFPs. Here's what I want. Here's the plan I'd like you to follow. And again, I would say, tell me what you're going to do. Let me confirm it. Hopefully it will give you back the same plan that you put in. Say go and let it go. This strategy, again, you're still going to want to check what it does. You're still going to, you know, it isn't perfect. But I think you will find that this strategy of instead of chatting with it to get to the work product you want, instead organizing your thoughts, giving you an instruction,

29:13having it come up with a plan and then having it execute the plan leads to much more consistent and reliable results. I would also say that for some of these kinds of tasks, for example, a task like this RFP thing, if you end up with a plan that you like and that you see worked well to get you summary of a particular RFP, that's a great candidate for reuse, right? You don't need to have that conversation to generate the plan with the AI. The next time you get another set of RFPs, you can reuse the one that you have. Couple things to say about this. Some of the tools that are already starting to encourage you to do this or building in features that help you do this. So they're helping you by sort of trying to plan first. And you're seeing this getting built into the tools part of it, the harness of these a lot of these tools. Now, and that's great. I think that's a great thing. But if they're not in the particular tool that you're using or it's not doing it the way you want,

30:14just do what I'm saying here. In other words, any model, even without any tooling, will be able to do this kind of workflow. Give me a plan. Let me have me approve the plan and then execute the plan. And that's much better in my experience and much more reliable than having the back and forth chatty. Okay. Let's talk about another strategy, providing context. This is really meant to address a number of things. The hallucinations, the fact that AI is limited by what they're trained on. As well as those two in particular. Let's take another example of a common workflow summarizing a deposition. This is a task that we often have to do for various reasons, including updating our clients as soon as we get out of the deposition. And a sort of naive or simple way to do it would be to say, I just got out of the deposition. The court reporter sent me the rough transcript. And I give that rough transcript to me.

31:16AI and I say summarize this deposition. The problem here is the AI does have a certain amount of context. It has the deposition, of course. If you ask that to summarize the deposition without giving it the deposition, it would do very badly. But it doesn't have anything else. And so what you're going to find is you're a little at the mercy of the AI might misunderstand some things about what the case is about. It might not know what some of the key terms mean. It might confuse the names of different witnesses. And as a result, how it chooses to summarize, what it chooses to cover is not going to be necessary what you want. And you're going to find yourself again in this kind of battle with it where you're going to say, hey, you didn't even mention the partner's background where he said he'd been an expert for the other side in a similar case or whatever, right? Things that you thought were important. You can try to tackle this problem using the first strategy, which is give it

32:19good instructions, ask for a plan for how it's going to summarize, etc. And that may give you somewhat better results. But the problem here is not just a problem of clear instructions and chatty. It's a problem that the AI just doesn't know what you know about the case and what's important in this deposition. But you can give it to it. And so a much better approach would be to give it the deposition, but also something like that. Please review the attached, give it other things too. So for example, in this case, please review the attached copy of the operative complaint. Here's an attached memo on the background of the witness. Here's my notes on what I observed during the deposition. Now review the transcript and provide a summary of the deposition and make sure to provide citations to every page online. So what have I given it now? And I've picked these examples, you could give it other things as context, but I picked these examples to kind of illustrate a few things. I've said, give it the complaint.

33:20In other words, I'm trying to give it some material on what this case is about, what's at stake, sort of to have it have some understanding of what is important and what are the important issues in this case. I'm also giving it some background on the witness so that it has to help it with what might expect to see in here, what might it, what things might come up, might be expected to come up in this deposition or might clarify things that the witness is assuming when they talk. And then the third thing is something that I think is very useful, which is sometimes you go through a deposition if you're the one taking or defending the deposition or an observer, and you may take your own notes about what's important. And what always drove me crazy when I was litigating and pre-AIDA is was the way I like to take notes was to take notes about what seemed important to me as it was going on. If I was taking the deposition, I like to jot down what were the things that I got,

34:23what were the admissions that I got, or what were the things that were unexpected, what were the things that I wanted to make sure to kind of come back to, things like that. Or if I was defending, similarly, I would take notes on, I wish the witness hadn't said this, or oh wow, they got a good answer on this, or oh, it's interesting that the plaintiff seemed to be going after this theory or whatever, take notes like that. I didn't really want to take comprehensive summary notes. To me, that was distracting and wasted time. With AI, you can sort of bring these two together in a great way. You can take your sort of what's important to me, I want to make sure to relate to the client, what do I want to make sure that my team focuses on in future depositions, give that to the AI, as it makes the summary, and have that inform its summary, so that you don't just get generic deposition summary where, you get a summary that highlights the things you'd want it to highlight, and that you were seeing live when you were there. Then I'm giving it the instructions, and again, also telling it to provide citations. In this latter point, I'll say more about this a little bit, but it's to also help me do what

35:28you're going to need to do no matter what, which is validate what it says. If you've just been in the deposition fine, but if you are asked to summarize the deposition that you weren't there, and you haven't read yet, this is not going to help you get away from reading it. You're still going to need to read it to make sure it's right, but this will help you double-check, you know, what the AI is doing. So here, by providing context, even though your question is still ultimately summarized in deposition, that's what I want you to do. But by providing the AI with context, you are now going to get a summary that reflects an understanding of what's at stake in the case, an understanding of what's going on with this witness, and your lawyer's perspective on what should be shown in summary. That's going to be a much better summary. And again, this is sometimes called, so in the early days of the AI, people will talk about prompt engineering, which was sort of getting your question exactly right to the AI so that it didn't hallucinate.

36:30And that is in general much less important to people. Think and often misleading. So for example, if you say to the AI, please don't hallucinate. That's not going to make it not hallucinate. And by the way, it doesn't help if you put it in all caps and tell it that the world's going to blow up if it hallucinates. It's still not going to prevent it from hallucinating. However, giving the AI robust context, giving it a kind of sense of what matters in the case, what's important, not going to eliminate things, but it will make the answers much, much, much better. You can tell this to your needs. And then the tip I sort of put here, which is similar to the tip I gave on the execution, which is, you don't have to do this from scratch every time. You can, you could have a prompt like this with the complaint and the witness is back right, and you could reuse that. Another strategy you can use is actually to have AI first create some context documents. And this is a strategy that I've utilized, which is say to the AI, read the complaint, read, you know,

37:34interrogatory responses, and write up a summary of what this case is about, who the key players are and whatever. And then for future depositions, you could actually just have it give that AI generated summary to the AI that's going to summarize and the deposition. Again, that's just sort of a way of kind of bookmarking or sort of saving your place rather than asking the AI to reread everything each time it summarizes the deposition. You can sort of generate a kind of context module that you're going to read once, but more importantly, you're going to feed it to the AI every time it goes to summarize the deposition. And you will find and you will sort of develop a sense for how creating those leads to better summaries and you'll improve them over time and end up with better work product. Next, I want to talk about show and don't tell. And to me, this strategy is, I think really needs to be used more. And this is actually in a sense, in my mind, the only way to really reliably

38:42cut down on it, even in some cases, eliminate hallucinations. And it sort of all stems from the notion that any time an AI is talking to you, it could make things up. So if I say to an AI, what is this case about? The AI, what it says back to me, this case is about, you know, two parties who didn't this kind of dispute and the court ruled that home summary judgment was appropriate for whatever reason, that's potentially wrong. What it's saying to you, anytime it says something to you, it's potentially wrong might not be likely, but it's potentially wrong. On the other hand, if I say to an AI, give me the quote from this case that is, you know, the judge is ruling on X issue. And tell me the line that that's on. And it gives me that. And I then go and look in the case, and in fact, that quotes there and that lines there, that's not a hallucination. You know, you may see that that what it said, it could still lie to you, you could go and look at that quote

39:44in the quote wasn't there. But by having to show you what is there in the case, rather than summarize things for you, you're going to get more reliable results. And let me give you an example. By the way, this idea of show don't tell and having it point to things in a document is you see this in some legal research providers now where they put actually a number of them where they were put in citations, end of things in the citations, you can click on them that will show you things in the actual document. I think that's great. That's actually what the premise of our legal research tool, align research is as well, which is we don't summarize cases. We simply have our AI actually going in, highlight the actual cases. So there's really no way to hallucinate because it's actually just highlighting the text in the underlying cases. But the concept is more generally applicable. So let me give you a couple examples or an example to walk you through this. I go to an AI and I say, you know, what's the law on the ninth circuit on fraud by omission?

40:46Okay, there's a lot of problems with asking the question this way. First of all, I know the AI has the most current law on the ninth circuit, you know, et cetera. Right. So that's a bad one. Better is to say, look, here are the cases, either because I've gone and searched and found the cases and put them into the context. Or because, you know, I have a connector to in my harness to, you know, some tool that has access to a legal research database. But either way, I'm getting at the context. And now I'm saying summarize the law on fraud by omission. That's better, but because I've asked it to summarize, to tell me something about the case, I'm still at a risk for hallucination. Right. It could, you know, it, it, it, it's summary of the current state of the law might be wrong. And I'm going to, you know, be at risk with for that, even though I've reduced the odds of that. Here's what I would suggest is instead writing your query along the lines of, here's a set of cases of fraud, review each one of these cases and just give me the names of the cases and pin sites or

41:50citations to those that deal with fraud by omission. And then for each of those cases, the ones that were actually about fraud by omission, give me verbatim quotes with sites to the pages. So what this AI work product will now look like is not a summary, but a list of cases with each case a quote and with these case a page. Now I can now go into the cases, find the page, look at the quote, and see if it says what the AI said. Is that more laborious than getting a paragraph summary? Yeah. But what you've done is, but it's less laborious than reading all the cases yourself. And, you know, reading all the parts of the cases, if what you're trying to do is just find the keep, you know, zero in on the key parts of cases and, you know, the parts that deal with fraud by omission, it's less laborious than that. And you're building into it an easy ability to check by having the AI point you to quotes and citations rather than having it summarize, you know, for you when you get a summary, you know, even with citations to case names, there's no, your way of valid is to go

42:58through and reread the whole case. If, if instead the AI gives you a quote and a page number and you want to check that, you go look at the page number and you look at the quote. For certain kinds of tasks, again, I gave legal legal research as an example, but, you know, finding documents that might be relevant to a particular point that you want to have a chronology on or things like that. Having the AI points you to places in the document rather than summarize things for you, is going to both cut down on the hallucinations and maybe more importantly, make it easier for you to validate and also is going to discipline you to go in and actually do the validation, right? And in my mind, given that, you know, AI's have this problem of not being, of always having an hallucination risk, it's a discipline that you want. Sure, we would all love to not have to read everything and, you know, to get perfect summaries, but that isn't the state of the art and we should not convince ourselves that it is, the state of the art is that these things can make mistakes.

44:00And so given that, I think, asking the AI to give you an output like this that is validatable and that forces you to go in and validate, right? Doesn't give you kind of a shortcut is good discipline in one that I definitely recommend, you know, for cases where it can be made to work. The last example I want to talk about in terms of strategies is categorized on analyze. And, you know, this is kind of wraps up a number, the hallucination and the psychofancy problems. And, you know, boils down to the fact that in my experience, even the top of the line AI models, you know, the latest model from clot or open AI or whatever, in my view, they're still not as good as lawyers. We could have a debate about whether they'll eventually get there or not, but the state of the art in my

45:03view today is that they're not as good as lawyers, particularly when it comes to, you know, analyzing and making legal judgments about things that are important holdings of cases, the importance of significance of things and documents. And they're okay. It's impressive that they can do it, but they aren't as good as good lawyers. And so if you are relying on them for judgment and analysis, I think you're you're walking into a trap and you're setting yourself up. I mean, in the worst case, you know, if you don't check it, if you don't validate it, you're setting yourself up for, you know, some embarrassing, potentially sanctionable failures. But even if you are, you know, as you should be good and disciplined about, you know, checking and evaluating work, you're setting yourself up for wasting a lot of time. You say to the AI, what are these documents should I use for the deputative, Mr. Smith? It says, oh, like here's 10 great documents that you definitely want to use for Mr. Smith. You go and read them and you're like, wait, these 10 documents don't help at all. You go back to the AI and you say these ones didn't help. Like, you know, I need you to find documents on x, y and z. And then it goes

46:04back and it finds those and you read those and it turns out, you know, they were about x and y, but not z. And so you go back and say, well, now, right? So, you know, great, you validated things. You didn't make the mistake. You didn't submit a brief, you know, or misadocument in the deposition, but you wasted a lot of time and AI actually made you less efficient. And that's because I think that there's summarization analysis, there are analysis and judgment sort of powers are just not where we are, spoilers. By contrast, their ability to categorize things, their ability to answer a simple binary question, is this document about, you know, the financial statements or not? Is this, does this case deal with fraud or not? You know, or may, you know, it doesn't have to be binary. It can be, you know, which of the following three topics is this document about? They are much better at that. Much better at that. Are they infallible? No, you're still going to have to validate, but they are much, much more likely to get things right. And so, the way I would go about asking a deposition,

47:08a deposition prep question like this is not give the AI, you know, 500 documents and say, which of these should I use? And not even to do that with more context. You know, here's what's important for Mr. Smith, blah, blah, blah. That's not what I would do, because you're giving the AI then an analysis task and you're delegating to it the task of figuring out, you know, making the judgments about what, you know, what would be a good thing to ask a question about. I don't think that's where you want to be. What I do think is useful is asking the AI to help you, you know, bucket the documents, categorize the documents into areas that may be useful for your deposition. So asking it, which of these documents discuss, you know, Q2 sales at retail stores in the Midwest, if that's an issue that's relevant? If you ask it that question, it will do a very good job of going through and saying these 10 documents relate to Q2 sales. And, you know, these 90 documents are about, you know, our marketing department's work in, you know, Q4. It will be very good at that

48:11categorization decision. And that will make you more efficient. Now, you're still going to have to read those Q2 documents and decide what you want to do in the deposition. But by the way, I think that's where you're earning your money. That's what you should be doing as a lawyer. But it will really help you focus on those documents. Now, is there some chance it missed some documents? Yes. And you're going to have to take them to account and you're going to have to have some double checking strategy or whatever for that. But my point here is if you can break AI problems down into categorization problems rather than analysis, judgment, summarization problems, you are going to get much more consistent and reliable results. A tip I will say is that a number of AI products, Harvey's one that I know in particular, but others have this as well. Have these kind of tabular review features, which let you put a bunch of documents in a table and then ask a question for each, you know, that will be asked to each document. This is a great way to use this kind of categorization strategy, right? Put in a thousand documents that you think you might

49:14want to use for deposition and ask the question, not the ultimate question, how should I use in the deposition or what topic should I ask about on this. But instead, a categorization question, which of these documents discuss earning, which of these documents discuss communications with Mr. Smith, et cetera, and have the AI do that categorization for you at a minimum that will give you a great starting point on which documents to dive into first. Okay. I see we have about 10 minutes left. I do have one other section that I want to talk about on some other ways of using AI, but given that we have about 10 minutes, let me stop now and see if there are any questions on any of what I've discussed so far or anything that, you know, in general on this topic of using AI and litigation, I'd love to sort of stop and take those questions now if they're there

50:15and use the time for that. If there aren't any questions or if the questions don't eat up the time, I will talk for another 10 minutes about the last topic. But let me pause for a minute and see if there are any questions on this. And let me just make sure I'm looking at the right place. I think I am. Maybe this is the quiet crowd. She's okay. Okay. I'm sorry. I was not looking at the right place. I'm sorry. And I missed a number of questions. All right. Great. So let's do that. This would be great. And I apologize that I missed these. I'm sorry. I got the wrong spot. Okay. First question was how careful do you have to be about confidentiality? This is a great question and you know at a certain point I'm going to give a little the warly dodge answer. You're going to have to make your own sort of legal ethics decisions.

51:18But I want to tell you some important things in my view. First of all, all of these with a very narrow exception that you're almost certainly not using. All of these AI providers involve the concept of when you ask a question, your question and in other context you provide goes out to somebody else. All right. It goes out to the AI provider to whether it's Anthropic, whether it's you know, JTBT, whoever it goes to them. By the way, this is true even if you're using a Harvey or a Ligora or a co-counsel or whatever, at least up until very, very recently, they even though you would send it to them and then they would send it on to an Anthropic or a Google or a Club. So that's happening. Sometimes you hear people talking about closed loop AI. That's basically not

52:19a thing. You could in fact, you could buy the hardware yourself and run certain models on that hardware. That would really be a closed loop system, but nobody's doing that other than very edge cases. So everything we're talking about AI is going out to some AI provider. That's just a fact of life. Is that a confidentiality problem? I think the answer is it's not any different than the same confidentiality concerns we have when we use, you know, Outlook, Microsoft Outlook for our email or Gmail for our email or I manage for our documents. The truth is everything we do electronically these days goes out to some cloud provider and what are the confidentiality protections we rely on? They're contractual. And so you absolutely do need to at a minimum. I mean, you need to read the terms of your contract with these providers and I think what you're looking for there is number one that they're not going to train on your data. By the way, if you're paying for your AI, every provider that I'm aware of for paid plans for the AI does not train on

53:24the data. So that's going to be an easy one. You also want to look at, you know, what do they say they're going to, hello, they're going to retain their data and that there is a variety among them. Some of the plans, you know, retain them for lengthy period of time, some retain them for less, some, and this is some of the advantages of some of the legal providers like Harvey and Ligora is, they will offer you abilities to have, you know, what they call zero data retention, which is almost deleted almost immediately. Those things may matter to you in terms of confidentiality, but these are not technical limitations. These are contractual limitations and they're important and you should look at them, but I don't think that in my view, I have not heard a convincing argument for why this is a new class of problem in terms of a lawyer sending information to an AI provider. To me, it doesn't seem like a different problem than a lawyer, you know, allowing their email to go through an email server. Different catalog fish and there've been some famous cases on this, including a judge write-off opinion on, you know, if a client asks a legal

54:29question to an AI, that's a whole separate catalog fish, but a lawyer using AI as a tool with proper contractor restrictions, you know, make your own legal judgment, but I don't see the issue, and I haven't heard a compelling argument why that's a concern. Okay, next question, West Lock Code Council hallucination risk, absolutely. It's great that West Lock Code Council and other legal AI providers are grounding their AI as they say, grounding their AI in their legal databases, their legal precedents, etc. That's great, but that is fundamentally not different than the examples we talked about before, where I give the AI the cases and then ask it a question, does that cut down on the hallucination risk? Absolutely. Does it eliminate it? No, it doesn't eliminate it, and there's in fact some famous examples, and I actually, if I didn't include the opinion in the materials, you can find it, or if you email me, I can send it to you. There's even examples of

55:30cases where people have got sanctioned from courts for hallucinations from some of these well-known providers. So it's not, no, it's not a panacea that you're using one of these legal research providers, and then they will tell you that as well. Okay, if using Claude, can you set the case context once and then refer to it again and then bounce from case to case? This a little bit gets into some tool-specific stuff, and I guess the answer is in general, yes, but it depends. It depends how you set it up. So a simplest answer that I hope will answer this question is Claude has a feature called Claude Co-Work, which is available in their desktop app. That allows you to point the AI to a folder of materials, which could be cases in this, and you can do that, and then you can ask

56:31questions about the different cases, and it will go in and find what it needs and come back and give you the answers. It'll do the franny documents. That's an example of I think what you're asking. Google has a product that I think it's thought they're called Notebook LM, which had a similar feature. You can also do it by uploading things to a chat and having it within the context of the chat, it will look, can look back at the cases, but I don't think that's what you're asking. I think you're asking about Central, and I would look at Claude Co-Work, it chats you, it's a similar thing. Harvey has vaults. I'm sure LaGoura has a similar thing. That's kind of what they're meant to get it. Okay, the next question, building on your point about prompt engineering, what's your view on loop engineering, and its potential application and litigation? Okay, this is a question, and I'm going to say congratulations to this questioner because they're creating a I think a sophisticated user of AI.

57:41So what this question is about is sort of a lot of the advances in AI, especially in non-legal fields, but it's coming to legal too, is in what are sometimes referred to as agentic workflows, or as the questioner asked, loop loops. And the idea here is that AI is now are capable of, in effect, you could think about it as having conversations with their selves. So you could ask an AI a question, it could say, okay, the answer to this question, I need to read these documents. It'll go out and read the documents, and then it will say, based on reading these documents, I think I need to have another AI agent go out and look at some other things and give me a summary of that. And sort of chain these things together, and this is really the model and the harness working together. And at certain points, it will know to prompt the human and say, hey, human, I'm not sure where to go at this point. I figured out this, and I figured out this, but what do I need to do? And then the human will respond to the AI, we'll sort of go off in a different direction.

58:47This is 100% important in the future, but what I would say, so to answer the, to be in my definition mode and answer the question, this is important, yes. And I think is a way to streamline repetitive task. At this point in legal, I would say it's a fairly advanced topic. There are not, in my, say, I'm David off to you, great off the shelf, easy to use tools for doing this in legal right now. So if you're going to do it, you're going to need to sort of level up on AI, right? I don't think you need to do it to get a ton of great value out of AI. But if you're asking this question, my best advice to you is, that's exciting, it's fun. And you're looking at a couple of weekends, I've really digging in on how to use CloudCode work, how to use CloudCode, maybe how to use other tools like that and kind of developing the skills for that. I'm not going to go beyond that

59:51in this CLE, maybe one day I'll do an advanced CLE, but it's advanced topic. It's definitely important. I think we will start to see more commoditized tools. I think we will see firms building centralized tools around this. It's not a pool, you can just dip your toe in that. That's definitely that is the cutting edge of things right now. Okay, next question, to ask about citations for specific cases, does it need access to Westlar Alexis or are there enough cases on Findlaw? This is a controversial topic. So let me start sort of with simple, uncontroversial things. It needs access to something. It definitely needs some way to get cases. That's true. Second of all, Westlar and Lexus have great databases. The best, right? They don't, by the way, even Westlaw, Lexus don't have everything and Westlaw has things. Lexus doesn't have and Lexus doesn't think about them, etc. But they're great databases. They are, in effect, the gold standard for legal research databases. Those two things,

1:00:57I think are uncontroversial and true. Okay, here's the issue. You can't just give your AI access to Westlar and Lexus. That's not a thing. They don't make that access available to AI. They offer their own AI products and those AI products have access to their databases. They have connectors to certain other AI products, but those are not actually access to their raw databases. They're access to their AI and then their AI has access to their databases. You are implicit, I think, correctly in your question is, can you just say to your AI, you go search Westlaw, the answer is there isn't any way to do that within Westlaw's terms of service. And Lexus's terms of service. Really, I think what the question boils down to is, is your then your only strategy to pay Westlaw or Lexus for their AI or use one of their

1:02:06connectors to their AI that you get through a Harvard or a LaGoura? Or can you use your AI and just connect it to a public source like, like, fine wall, court listener, which my view is that your AI connected to court listener is going to get you, you know, 90 to 95 percent of where you want to be and in many cases 100 percent. I think it is great. I think it is a great resource. And I think the uplift you get from being able to connect your AI directly to it and just direct searches rather than having to go through some AI that a Westlaw or a Thomps, you know, or Lexus is trying to sell you to me, you know, outweighs the fact in most cases that, you know, there may be, you know, a random unpublished district court opinion that you miss. I think courtless and databases are great in getting better every day. The other opinion is no. If you're not boiling the ocean on Westlaw and Lexus, you know, you're

1:03:11not doing your job. I think in most cases, in many cases, that's overkill, but not in every case, you know, it depends on the stakes of your case and it depends on, you know, how, how, how obscure your issue is. But unfortunately, there's no great answer here, right? In other words, my perfect world is that Westlaw and Lexus would make their databases available directly, you know, from not for free, for the pay for and your AI could use it. It's not the state of the world now. So my own view is that the free law stuff is great. The court listeners have a great full disclosure. That's what our product align research is based on. So I'm biased. Take all that bias into account. But that's that's where things stand right now. Next question. What's the greater danger to our profession? Choosing between AI, hallucinations are implicit bias. If AI is using the internet, does it create

1:04:15the risk of AI becoming biased? So I guess what I would say is I think these are, I think in terms of our profession specifically, I think hallucination is the more immediate risk. The greater risk, which I think is, is really not specific to legal AI, the question of sort of, is there some things about AI training that would sort of make it not good in general and maybe not good for legal? Is a really interesting question and I don't know the answer. And in other words, I understand the question. I understand the notion that perhaps by the AI reading millions of Reddit posts, it's now somehow not good at answering contract while questions. I don't have an intuition as to whether that's true or not. I think it's something to think about when you're looking at these. And I'll add a further wrinkle. As is models created in particularly in China are becoming more popular and actually are the basis for

1:05:22training, you know, the models that have been released by Harvey and times and raiders. I think that question is also relevant. In other words, do we know what biases are baked into those models? And so, but I don't, I wish I had, if I was at a crystal ball on a great answer to that question, I think it's a good question. I think the immediate risk for us now is hallucinations, but as more and more stuff gets delgate to AI, I think the question of, you know, what are the subtle biases in AI is definitely important. Okay. What do legal models like Harvey offer that Claude doesn't? Can any of them prepare first draft of discovery responses? Okay, let me take the last one. They could all prepare first draft. The question is how good is the first draft? And I think in both cases, they're not going to be good enough for you to just send it off. I think if you follow some of the strategies you

1:06:24talked about today, they may be good enough to save you time to save, you know, to let you do it. I think they will. Both be good enough to do, you know, less time than, you know, you would have taken on your own. What does Harvey offer that Claude doesn't? My honest view on this is that mostly what Harvey offers that Claude doesn't is a kind of safety and comfort for lawyers that they, and this is that, you know, at the level of individual lawyers, right, it makes it harder for it to make some catastrophic, you know, for example, it has this vault feature where you upload documents they're in the vault and it does things. Whereas with Claude, you can give Claude access to your file system and it can run wild. You shouldn't, but you can. So Harvey sort of put some safe guards around these things that I think are helpful. It gives auditability and enterprise level controls

1:07:28for law firms that, you know, have not yet really come to Claude or these other, you know, frontier providers. So I do think it has some benefits, particularly when you're at an enterprise scale, you know, big law firms needing to manage lots of lawyers. In terms of the performance of the models, the legal reasoning, my own view is they're either the same or I think that the Claude's and Chatchy-Betese have the edge. So, you know, in my view, these products are offering a, they're often a service to you in terms of making AI easy and safe to deploy to your lawyers. I don't think the technical performance is better. And in fact, I think in many cases, at the very edge of technical performance, I think the frontier models with every time. Let me just ask a question to the moderates because I see there's like a stack of questions here. I can talk about this all day, but I think we probably are going to hit a hard limit at some point. So. And, you know, as long as you'd like, we can adjust the

1:08:29credit. Okay. Okay. Anybody drop off if you get bored. I find this stuff fascinating. What affordable products protect against the breach of confidentiality, Claude Pro versus the free version. Are they discoverable? You know, here's what. Okay. And then, you know, can AI use be detected? Let me take these kind of in a couple. In general, and, you know, I would say that cybersecurity is much more, you know, it's always been important and it's even more important in the age of AI. I think for two reasons, I think there are new risks. I attached a case in here about prompt injection, which is where in this case, it was a litigant actually tried to add instructions secretly to a brief to have the courts AI make a wrong decision didn't work, but that was the idea.

1:09:35But that class of attack is a real class of attack. And the danger for lawyers are, you know, that AI could go out and read something on the internet thinking it was finding a case and instead read instructions to do something malicious. That's a real risk of it's a new risk in the world of AI. Old risks too, just, you know, bad passwords and things like that are sort of magnified by AI, because as you may have read in the news, you know, attackers have access to AI and it leverages, it scales up their ability to run attacks. So cybersecurity is important. Beyond the scope of the ceiling, but it's definitely important. There in terms of the specifics of like your contract agreements, you know, pro versus free, it's sort of addresses before. Yeah, you definitely want to tear that, you know, doesn't train on your data and typically the free ones do. So you don't think those are, I think they're confidentiality problems around that. I don't know that a court has ruled on that. I don't think a court has, but I would not be comfortable using that for legal work.

1:10:36On the flip side, I think the paid versions that don't train and with appropriate restrictions on data retention, you know, I do think are you want to know plenty of lawyers and big law firms that are using them, you know, under those contractual terms. Um, discoverability, I don't think discoverability is really a new issue. I think, you know, the discoverability of your chats with AI are similar to your discoverability of, you know, documents that you put into Dropbox, for example, um, if you're a lawyer and you use Dropbox for your business and you have, you know, an enterprise plan and confidentiality, which is probably fine. If you're a client and you stick your incriminating documents into Dropbox, they're still just as discoverable as they were in any other time. So in general, I, I don't think there are really many new discoverable things, but, but, but certainly people are going to start requesting

1:11:37AI chat logs in discovery and, you know, some of them may be privileged and some of them may not be depend, you know, or work product more likely. But certainly, you know, your clients who are having chats with, you know, AI about all kinds of things and they're not lawyers and these aren't, like, that's going to be discoverable and people are going to ask for it and they're already asking for it. The, the question of can AI detect AI? I'm not a big expert on this. My sense is that there are tools that can do it. They're not 100% reliable. And, you know, there are times when those may be worth using, but they're not 100% reliable. And, you know, in both ways, they can miss things and they can detect things as AI written that aren't. So, but again, I really don't, I don't have a lot of them that I have those tools. Do you find cloud Trump's chat GPT? I don't think there's a binary

1:12:38answer to that. I think, first of all, you know, they're both releasing new models and new unupdates to their harnesses all the time. I think it goes back and forth. I, you know, I think, cloud sort of, I think it bounces back and forth. I will tell you, I these days, I use cloud, chat GPT and Gemini for legal and, you know, a lot of the work I do these days is non-legal. And frankly, even within particular providers, different models give you different results. So, I don't think one is better than the other. I'm blunt about most things. I thought one was, I would tell you. But I, I also think that for like, you know, 95% of what lawyers are doing, it doesn't matter that much. There's a difference between, you know, the top tier, you know, cloud opus and cloud sonnet opus is better than sonnet, but the difference between opus and touch of Pt. Sync front, the vast majority of what lawyers doing is not going to matter. And,

1:13:40you know, using a, you know, a frontier model is good and they'll give you good, not perfect results. And the difference is among them. I don't think matter that much for lawyers. Ha ha, this one's a great set. Is there a product other than from Westler Lexus that can access these legal research? Absolutely there is. It's called Align Research and you should go use it now. It is Align.Loyer for Susp and Research. But we're not the only one. There are, there are a number of them, many of which are using SBR, the courtless center database, which for US case law, it's phenomenal. Others have built their own databases. And, and, you know, I have been at some pains to do them. There are companies like describe and I should know these are my competitors. Anyway, they're doing interesting things and they're good things and frankly, so yes, there are definitely other options. And I think it makes a lot of sense to explore those.

1:14:45Google wants to have an enterprise for legal. Anyway, yeah, I think that you're just pointing this out. That was a very recent announcement this week and I haven't used it or tried it or anything. I'm eager to see it. I think it's great that they're in that space. I've never used Westlaw co-counsel. The next question is on Westlaw's co-counsel. The question doesn't find it very useful. You know, I haven't used it, so I don't have an opinion on that. I definitely, I definitely don't like tools that sort of constrain you to, you know, just sort of this chat format. Like, you ask a question and it gives you sort of a memo answer. I like one of the things I like about a cloud co-work or, you know, other tools is for general-purpose AI tools, anyway. I like my general AI purpose, general purpose AI tool to be general purpose. So if I want it

1:15:51to, you know, have a chat with me or generate them a long written summary, great, but I also like you able to ask them to do other things and you certainly have that flexibility with, um, Claude. I don't want to speak to the other providers, but in general, I've sometimes been frustrated. I'm not talking about co-counsels particularly, but in general, I have sometimes been frustrated with legal tools that purport to be general purpose tools, but in fact, I'm kind of limited in kind of what I can do with them. Um, I'm confused that it was mentioned that co-counsel sells data to Anthropic or Chatty PT. So just to be clear here, they don't sell it to them. That's not what I meant to say. I just mean, at least up until this week when Thompson announced a, um, that their own model. I don't know how much they're using it, but in general, all of these models were are using Frontier models. So you send something to Harvey or co-counsel and they then send it to, um, you know, a Chatty PT or an

1:16:59Anthropic. They don't sell it to them. They have contractual restrictions that say that they, then in fact, the opposite, that Anthropic or Chatty PT can't use it for anything. They have to lead it right away, et cetera, but, you know, they are a provider to them, just like, you know, they run their, you know, software on Azure or AWS or whatever. So they are a provider to them. They're one of their, their service providers. Um, again, at least until very recently, when some of them have started finding tuning their own models, but they're not selling their data to them. And, you know, in other words, they're still complying with what they've promised to you that they would do and, um, and, but they're using third party providers, um, including for their AI models. I don't want to really get into selling my product, but yes, there is a, I've probably crossed that line, but there, there is a free trial. Feel free to try it out. We are using court listener.

1:18:01My email address, uh, you can reach a couple of places. I think I put this on the slide, which are, but anyway, it's Sam at either MFB tech.com or a line.lawyer, um, and, uh, and, uh, you can also find me on LinkedIn. Please do, please follow me. I write about all these topics ad nauseam. So if after, you know, an hour and a half here, you're not tired of me going LinkedIn and you can read my thoughts on that, um, any of which will sound familiar now that you've heard me talk. Um, um, again, what are the free legal database use? Yeah, court listener, I think it's great. I think those guys are doing great work. I think they're project. It's not just case law research. They, uh, have access to paste materials and other things. And I think for us, law, it's really phenomenal. Um, not really a question, but very cool that you have a vintage Mac on your desk behind you. I think it's cool too. And the pathology goes even deeper. I have many, many, many vintage Macs, but the less said about that, the better, um, in terms of revealing too much about my own mental health.

1:19:07And thank you all very much for having been a great audience and for all the great questions. I love that. Uh, this is great. The one part I did not get to in my slides, but you'll have the slides is just I had a section, um, that followed that had just a couple of, really the point I wanted to make is, you know, yes, using AI just sort of help you do big picture legal things is great. But don't forget about the simple things. Um, and I gave some examples here, um, that I think illustrate the point. I think that, oh, um, you know, using AI for sort of just kind of, uh, automating or avoiding sort of simple, uh, annoying things in your legal practice. Um, it is actually a lot of benefit too. So just don't forget about those things was really my last tip. And with that, I will, uh, let's wrap it up. Thank you all very much.