Emerging Litigation Podcast

Transforming Legal Workflows with AI: Sara Lord Interviews Tara Emory and Wilzette Louis

• Tom Hagy • Season 1 • Episode 80

Share episode

“Of all the opportunities legal operations teams might identify to save time, money, and resources while potentially improving quality, Robotic Process Automation may currently offer the biggest and most immediate opportunities.”

That is from the forthcoming book, "Legal Operations in the Age of AI and Data," specifically the “Automation in Legal Departments” chapter written by Tara Emory, Wilzette Louis and Adam Poeppelmeier of Redgrave Data, and Kassie Burns of King & Spalding.  (Available for pre-order now from Globe Law & Business.)

Automating repetitive tasks and workflows required to effectively advance litigation frees litigators and support teams to focus on “strategic, analytical, and high-value work,” say the authors. Boosted by AI technology, like natural language processing, these tools can conduct data extraction and analysis from volumes of documents, create new documents, summarize documents, or initiate document drafting.

How can litigators best leverage these capabilities? 

Listen as our first-time guest host Sara Lord interviews Redgrave Data's Tara Emory, SVP, Legal AI Strategy, and Wilzette Louis, Director of Client Solutions. 

Tara is a highly regarded legal industry executive and recognized expert in legal AI, ediscovery,  information governance operations, and consulting. She plays a leadership role in The Sedona Conference and was contracted to serve as eDiscovery Lead on the House of Representatives Select Committee to Investigate the Jan. 6th Attack on the U.S. Capitol. Tara holds a JD and LLM in International and Comparative Law from Duke University School of Law.

Wilzette is an ediscovery expert and advisor focused on approaches for using technology and workflows to  maximize effectiveness, efficiency, and overall client satisfaction. Wilzette has a BS in computer science from the New York Institute of Technology.


I welcome as guest host for the podcast Sara Lord, a former practicing attorney with a decade of experience in data analytics. Sara applies her talents in large and small law firms and businesses to explore and address the cultural and practical barriers to diversity in law, and client-first business-oriented practices. As Managing Director of Legal Metrics, she leads a team of experts focused on providing the tools to support data-driven decision making in legal operations and closer collaboration between law firms and their clients through automation and standardization of industry metrics. Sara earned her J.D. from New York University School of Law. 

Listen as Sara speaks with Tara and Wilzette about the game-changing potential of robotic process automation and AI, and how these are not just futuristic concepts but practical solutions to today's legal challenges.


*******

This podcast is the audio companion to the Journal of Emerging Issues in Litigation. The Journal is a collaborative project between HB Litigation Conferences and the vLex Fastcase legal research family, which includes Full Court Press, Law Street Media, and Docket Alarm.


If you have comments, ideas, or wish to participate, please drop me a note at Editor@LitigationConferences.com.


Tom Hagy
Litigation Enthusiast and
Host of the Emerging Litigation Podcast
Home Page

Tom Hagy:

Welcome to the Emerging Litigation Podcast. This is a group project driven by HB Litigation, now part of critical legal content, and VLex Company's Fast Case and Law Street Media. I'm your host, tom Hage, longtime Litigation News editor and publisher and current Litigation enthusiast. If you wish to reach me, please check the appropriate links in the show notes. This podcast is also a companion to the Journal of Emerging Issues and Litigation, for which I serve as editor-in-chief, published by Fast Case Full Court Press. Now here's today's episode. If you like what you hear, please give us a rating Of all the opportunities legal operations teams might identify to save time, money and resources, while potentially improving quality.

Tom Hagy:

Robotic process automation may currently offer the biggest and most immediate opportunities. That's a quote from the forthcoming book Legal Operations in the Age of AI and Data. Specifically, it's from the Automation in Legal Departments chapter Written by a group of experts, including two of our guests who I'll introduce shortly. Automating repetitive tasks and workflows required to effectively advance litigation, freeze litigators and support teams to focus on quote strategic, analytical and high-value work, the authors say, written by AI technology, like natural language processing. These tools can conduct data extraction and analysis from volumes of documents, create new documents, summarize documents or initiate document drafting, and soon they'll write podcast introductions. Who are we kidding? They already do. They ask how can litigators best leverage these capabilities? Well, let's find out. Let's listen.

Tom Hagy:

As my first time guest host, sarah Lord interviews Tara Emery, who is Senior Vice President of Legal AI Strategy, and Wilzette Louis who is Director of Client Solutions at Redgrave Data. Tara not to be confused with Sarah, tara is an experienced, accomplished and highly regarded legal industry executive. She's a recognized expert in legal AI, e-discovery and information governance operations and consulting. Who else do we have on here? Come on, what do you think we're running? She plays a leadership role in the Sedona Conference and served as e-discovery lead on the House of Representatives Select Committee to investigate the January 6th attack on the US Capitol. You've heard of that. Tara holds a JD and an LLM in international and comparative law from Duke University School of Law. Will Zitt Lewis is an e-discovery expert and advisor focused on approaches for using technology and workflows to maximize effectiveness, efficiency and overall client satisfaction. Will Zitt has a BS in computer science from the New York Institute of Technology.

Tom Hagy:

This is Will Zitt's first time on a podcast, so welcome her to my world and, as I mentioned, my guest host is Sarah Lord. This is her first time guest hosting for the Emerging Litigation Podcast, and so can you. Sarah is a former practicing attorney with a decade of experience in data analytics. Sarah applies her experience in large and small firms and businesses to explore and address the cultural and practical barriers to diversity in law can you imagine? And client first, business-oriented practices. As managing director of legal metrics, she leads a team of experts focused on providing the tools to support data-driven decision-making in legal operations and closer collaboration between law firms and their clients through using automation and standardization of industry metrics. Sarah earned her JD from the New York University School of Law and, may I add, in time I've known her. I've learned that Sarah is whip smart and not without sass. I mean that in a good way. So listen as Sarah Lord interviews Tara Emery and Will Zett Lewis from Red Grave Data. I hope you enjoy it.

Sara Lord:

Will Zett and Tara, thank you both for speaking with me today. Thanks for having me, and a really big hi-dee-ho to all the lovely people in podcast land. Welcome to the show. The clock is ticking so I guess let's just dig in. Will Zett and Tara, you co-authored a chapter on automation technologies in legal for the forthcoming book Legal Operations in the Age of AI and Data, which is available for pre-order from Globe Law and Business. Can you tell us a little bit about the book?

Tara Emory:

Yeah, we're really excited about this book. Will Zett and I co-authored this with our colleague Adam Papelmeyer and with Cassie Burns from King and Spalding, and the book is edited also by Olva Mack, umaira, norristani and May-May on Woody Wei.

Wilzette Louis:

So our chapter dyes into automation and technologies and the use of these tools in legal departments, and how we streamline processes and boosts efficiency across law firms, vendors and e-discovery companies has covered in our chapter. Solutions like robotic process automation can deploy bots to automate repetitive, rule-based workflows across systems. Web scraping tools. Such can also be used to extract data from websites, and it is a great example of how automation is being utilized today. For instance, law firms can actually use robotic process automation to capture evidence on websites to cases that they are currently working on and modern, and they'll be able to keep that information for trial or additional analysis. So other forms of automation technology utilize machine learning and artificial intelligence, and that's for tasks or processes that require some intelligence to extract information and analyze documents or provide summary.

Wilzette Louis:

Ai has been a major focus point for the past few years. Everyone's talking about AI. We've been working with AI for decades now. We have a data science team that has decades of experience of building machine learning models and analyzing. We also have a development team that works closely with our data science team in building these custom solutions for our team and, thirdly, we have a professional services team that brings everything together. Now the reason that we were so excited of collaborating in this book. We're excited to be able to help everyone out there that's having issues figuring out how do they implement AI, machine learning or all of these new tools into their day-to-day processes. We have framework works, we have best practices and we have skilled individuals that's been doing this for years, not just companies that just popped up in the last year when AI and chat GT became a big thing. That's exactly why we're so excited about this book and we have some great information in it that we can share with everyone.

Sara Lord:

Okay, so that's very exciting, and you've been talking about AI and automation, so let's just deal with the basics first. And when you talk about automation and automated technologies, can you explain what that is for the uninitiated?

Wilzette Louis:

So automation technology is taking these repetitive, role-based tasks that humans perform on the computer, on systems, and having an automated workflow. So you can have robots, which is a terminology that's used for robotic process automation that can actually mimic a human's processes when it comes to opening up the Word document, extracting data, coded information, creating a report. So automation technology is any rule-based task that we can utilize software to do for us, so AI can give attorneys more free time to focus on other things and we'll have more accuracy on the results of the end product.

Sara Lord:

So how is that different than the AI automation that's built into Google, apple, microsoft or, in law, e-discovery, document management, legal research tools, or is it not any different?

Wilzette Louis:

Well, it is different and that's because these companies, these technical companies and these vendors, when they create a product, it's solo, it's based on their particular tasks, the particular workflow, the particular service they're trying to provide. Automation technology actually takes all of the different aspects to an entire workflow and brings that into a set of processes that a software robot can take care of. So, for instance, if you have to go onto a website, grab some information, put in an Excel document, generate a report and send it to a partner at your firm, you can actually use a robot to follow those exact tasks and send that information. But keep in mind, this is technology and you always have to verify and check your information. So the big difference is these tech companies are less flexible as far as what you can do with it, and automation technology can bring in different system, different application and get an entire process completed and not just one task.

Tara Emory:

Yeah, the thing to keep in mind when you're looking at any product in technology is that, no matter how shiny it is, it may or may not be the best investment for a law firm or a company to make, and the types of automation technologies that we are working on do include those types of tools, but it's really wherever the pain point is, wherever an organization's team is spending the most time on manual tasks. That's where you can further automate with more technology. So our approach and what we look at is really need-based as opposed to product-based, although products can be part of making sure the need is met.

Sara Lord:

So, specifically in the litigation space, what benefits could litigators enjoy if they effectively implement automation and advanced technology solutions?

Wilzette Louis:

Time and time is a big thing, I know, for litigators. I've been working with them for about 20 years and time is important. Well, they'll have time to build their professional career, work on more strategic workflows, work on things that requires and uses their skill set.

Sara Lord:

So there's never enough time or money. I think we can all agree on that. So how do those limiting factors affect attorney's adoption of new technology solutions?

Tara Emory:

Attorneys are very reactive. They're used to dealing with a problem when it arises. If the need is great enough, they'll go seek a technology solution. And usually to be great enough, the need has to be so great that they don't think they can possibly solve it in the context of their case without technology. But otherwise, even just learning how to use a new tool can be.

Tara Emory:

That's an investment too. That's time that you're spending learning how to deal with a tool you've never used before. When you have an old, reliable way that you know will get you where you need to go, and without taking the time and risk. To try something new in the middle of an urgent need is not the way to go, and that means that people really have to be proactive in how they come at this. They have to say these are the general needs of my group and my team. These are needs that might come up in an urgent situation and prepare for those, but also, if you really wanna start, I think that most legal teams, if not all, have room to reduce their manual processes in some form, and so the time available to go deal with that and explore options is not something most people want to make room for, but I do think that it is essential to remaining competitive, especially as the tools that are available become more and more sophisticated and other groups are gonna be using them.

Sara Lord:

So let's say we get over the hurdle of time to look into these products. Consider the products, the tools, the resources. How would an attorney get started testing the efficacy and appropriateness of an automated tool? Do they hire consultants or work with vendors, or do they just purchase a product off the shelf and trust it?

Tara Emory:

Sometimes, yes, to all. It depends, and I think that this really starts with what their needs are. I look at this as some groups. If this phrase resonates with you, then you have a need, and that is shoveling data by hand. If you are moving data from one place to another, or you're constantly digging through similar stores of data, looking for new things or things that you need, then you're shoveling data by hand and you have perhaps a need that can be solved with automation and looking for where your team or yourself, you're spending the most time on those types of tasks. That's what you go out and do. So once you've done that, then, yes, you can go start looking at products.

Tara Emory:

One of the worst ways to approach this in general is seeing a shiny object and deciding that it's really pretty, and now you want to go explore that because it may be a very good product, but it may not be the thing that is best for you to invest in at that moment.

Tara Emory:

So start with evaluating where you're spending your time and have the most need for savings and then, I think, from there, go find solutions and then, once you have one if you're talking directly to a provider that has an off the shelf product that appears to meet your need, then ask them for something that legitimizes that their product works, if it is something that is a little bit opaque to you.

Tara Emory:

So, especially when using AI, you really do need white papers, research studies, whatever they can give to you. If you don't know how to interpret what they sent to you, then you might need a consultant anyway. But I also think that you can dream up a solution, and that's where we do custom things. So we ask people, think what your pain points are, and then you tell us and then we can see if we can design a solution, and so in that case, you are hiring a consultant to help you work on what your need is and what it will cost to solve the problem that you're talking about and that might include one or several products, or there may not be products available.

Tara Emory:

Wherever something's already built, that is usually a good option, but we wanna make sure that it actually works.

Sara Lord:

So when you say shoveling data, you're not limiting data to numbers, to excels, right? Is that? You just mean information generally?

Tara Emory:

No, let's be more specific. Well, Zett, do you want to talk about some of the examples of how automation can help? We've saved some clients money and we know how automation can solve pain points.

Wilzette Louis:

One example is we had a client that came to us and they were monitoring over 800 regulatory proceedings on a website, the government website. That's definitely related to their cases. But what they were doing to access this data attorneys were logging in every morning and looking for any changes or modifications that were made on these sites. Obviously, this was very expensive, time-consuming and error-prone. They wanted our team to design a workflow or a procedure to make it easier and faster for them. Our development team used a robot process automation. We created robots that were able to go on these website and scrape the relevant information.

Wilzette Louis:

We extract documents. We also extract metadata that was related to the document. So metadata is the user, the date or any information about the document that they felt was important. Once we pulled that document the documents down, we were able to create a structural text where we can load it into a database system where they can log in and run their searches. They didn't have to go in every single day and run these searches and look for any updates or changes. They were able to just go on this website and to see what was changed and how the information was changed. This way they were able to go back to their partner and give them a status on how the case was going, if something was new or if something changed. This particular workflow saved them a lot of time, a lot of headache and definitely saved them a lot of money.

Sara Lord:

Tara talked a little bit about understanding the automated tools. Are there any other risks, pitfalls that people need to be aware of before they implement a solution or build a custom solution?

Tara Emory:

Yeah, you need to make sure, when you're looking at what it does, that you understand the intended use. What is the use case? That should be what you test and be aware that people are enterprising and they want to do their jobs well and they may find other use cases as time goes by or not understand that they are confined to a particular use case and that they don't understand what flows through. You really need a training process, you need an onboarding process to make sure that everyone's familiar with how to use the tool correctly and what is an acceptable or proven use and what might be an off-label use of the tool. If there are off-label uses that are cropping up, you really have to address those and see if they're addressing additional risks.

Tara Emory:

As an example, with the blow-up of generated AI, we're seeing a lot of incorporation of RAG systems or retrieval augmented generation.

Tara Emory:

The way that these tools work is you could submit a query and before it writes a response to you, it's got to decide what it wants to rely on for its response and it will go and find some source material.

Tara Emory:

To summarize to respond to you, that's the retrieval part. Retrieval could be five files, it could be 20, it could be a thousand, but it really depends on what you're doing. And if you were under the impression that it was going to search a set that it isn't going to include, maybe that it's actually going to summarize everything, an entire corpus for you, but it's really only checking 20 documents, then you need to know that so you can plan a workflow where, okay, sometimes this is really useful but we can't take it as it's really run across an entire set. We might want to check what it's actually pulled and what it's referring to and see if we think that's a good step for it to be relying on or something might be missing from that. That's just one example where not understanding what the technology is doing under the hood in some fashion could cause you to misunderstand what it's intended to be used for and where the limitations might lie.

Sara Lord:

In Will Zett's example of how firms and lawyers have up to their game using these solutions. She described a public data source, but if you're building a tool or using a tool that's working with a private data set proprietary data set, confidential data set you're always going to have issues of data security and privacy considerations. Can you talk a little bit about those in the implementation of tools and a legal workflow?

Tara Emory:

Yeah, for the most part. The idea of sending private or confidential data to a cloud is not new. It should not be new and is generally dealt with through contracts. I'm old enough to remember when people used to come check data centers and there were arm guards.

Tara Emory:

You could go visit and say this is where your data lives and you couldn't really check that's where the data was. Certain things you still have to take it face value, but today you can't really go do that. In fact, there's some ways of setting things up where the data goes places but never really exists except in an ephemeral way and certain servers, or it's serverless. There are so many ways of setting this up. Now you have to go on. Is this a reputable vendor? What is known of their reputation? Have they had any incidents? What's been done about those incidences? Your contractual provisions, who's assuming the liability and what representations are being made? Does that give you enough information to tell your own clients you've taken the necessary steps? You can look for certifications, like SOC certifications, and try to get the paperwork in place to show that you've made reasonable decisions and who you rely on. You can't plan for everything and what's baked in the complexities that have been added by Generative AI? One thing that people have really focused on for this one is that it could be true for any AI system, but the Generative AI in particular is aggressively using any queries as new input data. The prompts could become part of the model, and this has a lot of people scared.

Tara Emory:

If you are in one of the current collaboration platforms that's popular, I promise someone else has access to your company's data at all times. They just generally are not going to actually go use it. One of the things you have to keep in mind is who has access to this data, even within the company that is receiving it. Do they have employees that are doing quality control and they've taken steps to lock down? Who has access to things and how much do you care? How much risk is there in an internal employee who has their own confidentiality requirements going and looking? It really depends what you're dealing with. You have to take into consideration, as an attorney, how sensitive is this data? You also have to keep in mind actual regulations. The EU Data Privacy Act is always something that we are worried about. Gdpr, because those are strict requirements. Confidentiality to our clients is a strict requirement. Privilege protection is a strict requirement. You have to think about all those separately and think about what is reasonable to take on word of your provider versus maybe it's not acceptable.

Tara Emory:

If you're dealing with something really confidential, then you may have to go further steps than if you're dealing with something that is confidential in a client sense but isn't something that runs a huge risk, for example, if you're dealing with government contracts or things of national security concern. There are other options offered by some of the AI providers that will not. They'll even take off the table that. They'll hold your data for a small amount of time where it might be reviewed if they think there's a violation of their policies. They won't even do that.

Tara Emory:

So you pay more for that and that's all part of the decision. But at a high level, it's never been a good idea to go put your client's confidential information into a Google search because you have no contract with Google in that sense if you put it into a Google query. And so same thing with prompts, like do not put things that are confidential into a prompt in an unsecured public version of an account. You have to go and pay. I don't think there's anywhere you could do it without paying, so you have to pay for a higher level of security.

Sara Lord:

So it sounds like there's a lot of potential value to these tools if you do the homework required to make sure that it's doing what you think it's doing. You understand what information it's capturing, how the data is being used and appropriately utilizing the data on the tail end. And so, in light of that, why are some lawyers making use of these types of tools, while others have taken no action to implement them?

Wilzette Louis:

We have some lawyers that are excited about technology and they wanna use it to increase their workload, work more efficiently and see how these tools will make their work, day-to-day work, easier. But and these are attorneys that are considered tech savvy they love technology, they think it's a good opportunity. But there are other attorneys, on the other hand, that are a little bit afraid of technology. They have their processes that they've been using for years. It works, they have no issues with it. So they do not wanna bring. They don't wanna learn something new. They don't want to have to have their process in a tool that they can't see the back end. They don't know what's going on. So they're really nervous about should I even bring this tool in?

Wilzette Louis:

There's also issues of costs and resources. Some small companies do not have enough money to invest on these tools. They're not sure how to bring them in. Even if they are interested, there's no one in the department that would be able to analyze these tools, see what works best and see what process will they can actually implement these tools on, even though that technology at the point right, that automation technology is not being utilized at the same pace. But eventually we're all going to move to a industry legal industry where technology is not an option anymore. It's actually you have to start bringing these tools in so you can be competitive with these other law firms, with these legal departments and vendors, because they're using the tool. So you'll have to start thinking about how can I start implementing, at a small level, some of these tools so I can actually provide better client services based on the task that I select.

Sara Lord:

So the same process we went through when we were moving from book research to electronic research, where initially it's that black box factor and you slowly moved through that as an industry where honestly I think for some professionals it's much less about eliminating the black box than recognizing that everyone's in the black box. I don't know that anyone can really tell you with confidence what's happening behind the scenes of their research vendor sites. I mean, I think we know in theory but auditing that can be really difficult but we've all just gotten comfortable with it. Is that a fair assessment?

Wilzette Louis:

Yeah, yeah, that is fair and, like Tara said, being able to speak to these providers and having them give you documentation that this is what's going on and you have proof that you actually took that extra step.

Tara Emory:

Anything to add, tara, over 10 years ago, when I would talk to attorneys about technology assisted review and use of machine learning and document review, there was a lot of mistrust about this, and still is, and because of the black box factor. And so what we would tell them is look at the validation metrics. We just wanna see if the process worked and so you don't have to worry about the weights that happen inside the formulas that go into the model. You need to see that the model did its job correctly and that's what we're gonna focus on. It shows your process was reasonable.

Tara Emory:

And to demonstrate this, I used to ask people how many of you today would get onto a plane if there were no pilot, and nobody would raise their hand. And I would say well, what if I told you that we're like 20 years in the future and it's just normal that you get on a plane and there's no pilot, like they're flown by robots and there's been no incidents and this is the most safe thing ever. How do you feel now? And they'd say, oh, that's okay. And I'd say okay because now you have evidence that this is okay.

Tara Emory:

And I had to stop that way of talking to people because more and more hands would go up. People are totally fine flying on a plane. Now I still wanna know there's a human there to take over if the robot fails. But I think now we have self-driving cars at a level that wasn't even, I think, on people's minds at that time, and it looks a little weird to us today to see if we were to see a car that doesn't have a driver. But we'll get used to that too, and so I think what you're saying, sarah, is exactly right. Artificial intelligence has always referred to that thing that we expect humans to do and is now being done by an artificial intelligence machine, and so that's always gonna keep moving. And getting comfortable with black boxes is sort of the way of the world.

Sara Lord:

So the early adopters right now are really taking the weight of the initial research, the initial testing, creating that comfort in the industry and then the rest of the industry will catch up.

Tara Emory:

I would say for artificial intelligence, that's right, especially given that there's regulations and risk frameworks and a lot of attention and more technical compliance you'll have to worry about with uses of artificial intelligence. But, as Willette explained, automation and robotic process automation are much, much broader and this is where we can look for things that just seem like low level tasks that people are doing over and over again at their computers and may be able to be automated.

Sara Lord:

Well, it's exciting times for firms and attorneys who want to lead the industry and adopt these early so that they can get that extra edge. It's really great to know that there are organizations like Red Grave Data Consultancies to help them get comfortable with the data and understand what's happening and how their data is being used, so they can do this effectively and really reap the benefits. I want to thank you both again for your time and congratulate you on your forthcoming book, legal Operations in the Age of AI and Data, which again is available for pre-order from Globe Law and.

Tara Emory:

Business. Thank you, sarah, it was great to get to talk to you today.

Sara Lord:

And thanks to Tom Hage for letting me hijack this podcast to explore these issues.

Tom Hagy:

That concludes this episode of the Emerging Litigation Podcast, the co-production of HB Litigation, critical legal content, vlex, fastcase and our friends at LostG Media. I'm Tom Hage, your host, which would explain why I'm talking. Please feel free to reach out to me if you have ideas for a future episode and don't hesitate to share this with clients, colleagues, friends, animals you may have left at home, teenagers you've irresponsibly left unsupervised, and certain classifications of fruits and vegetables, and if you feel so moved, please give us a rating. Those always help. Thank you for listening.