Before You Deploy AI

AI-Savvy Lawyer explores the intricate intersection of technology and law, offering insights into navigating AI contracts, data governance, and legal implications in the rapidly evolving world of artificial intelligence.

AI in the Real World: What Businesses Can Actually Implement Today – and What Still Needs Guardrails

In this episode of AI-Savvy Lawyer, Charlyn Ho delves into the complexities of AI contracts, highlighting the discrepancies between vendor promises and legal protections. She emphasizes the importance of understanding how AI systems handle data and the need for operational guardrails. The discussion navigates through the nuances of AI terminology and the evolving legal landscape surrounding AI technologies

Transcript

[00:01] – Welcome & Introduction

Narrator: Welcome to AI-Savvy Lawyer.

Sean: Hello everyone, and welcome to the AI-Savvy Lawyer with Charlyn Ho, founder of Rikka Law. Charlyn is a technology transactions and data privacy attorney who advises companies on artificial intelligence governance, complex technology agreements, and the legal frameworks needed to deploy emerging technologies responsibly. I'm Sean O'Connor, and today we're talking about AI in the real world — what companies can actually implement today, and what still needs guardrails out there. Lots to dig into. Charlyn, how are you doing? Good to see you.

Charlyn: Good seeing you as well, Sean.

Sean: You know, companies everywhere — in every industry — are exploring AI right now. From the conversations you're having with your clients, what are the real-world questions they're asking your firm to help them with when it comes to AI contracts?

Charlyn: Yeah, absolutely. So the reality is that the law is behind, as it usually is — it lags technology traditionally, and AI is no different. So it's hard to understand what the "AI laws" are because there aren't too many specific AI laws yet. That's rapidly changing. There are obviously lots of other laws that still apply regardless of whether AI is in use or not. I'll just give you a couple of examples. HIPAA for healthcare data — that applies irrespective of whether you're using AI to process healthcare data or some other form of technology.

So really, the contract is the primary vehicle that allocates risk. When you don't have clear rules and regulations — and in fact when you're dealing with multi-jurisdictional practices, you may even have conflicting rules and regulations. For example, the EU versus the United States have different types of regulations, or lack thereof, relating to AI. So to answer your question more directly, my clients are asking me: what should I have in my contract? How should I protect myself? What should I be doing when I'm thinking about using AI, buying AI, selling AI, or wherever you are in your AI journey — because, like you said Sean, everybody is exploring it right now.

Sean: Yeah. I mean, I've had several conversations about it with other attorneys, and I'm wondering — there's a big difference between what AI vendors promise. You're talking about buying or selling — there's often a big gap between what they promise in their marketing and what the contracts actually protect. So I'm curious: what gaps are you seeing between the promises and the actual legal protections that companies end up with?

Charlyn: Yeah, great question. So I'm actually writing a book of the same name — The AI-Ready Lawyer — that explores some of these topics, because as somebody who has represented large companies and small companies against some of the major LLM providers, I have a lot of experience seeing what the marketing says versus what the contract says. And they're not always the same. In many cases, they're not the same — there is a gap.

Traditionally, the areas of most friction that I see, if you're representing the customer side — meaning the side that's buying the AI technology — you're going to be worried about data. So, how is the AI using your data? How is the LLM — and when I say LLM, I mean large language model provider; think of like OpenAI, Anthropic, or Microsoft for Copilot — what are these companies doing with your data? Are they just using your data to provide you the product? Are they using your data to train other models or their own model? Are they using your data to improve the current model? These are all open-ended questions when you first start exploring buying AI.

In fact, the promises that these providers offer will usually differ depending on what tier of the model you're buying. So if you're using the free version of any tool, the default is usually that the model provider will train on your data, and you expressly permit that. However, there are settings that permit you as the user to change that default. But if you want a higher level of protection for your data, you're going to have to upgrade to higher-level plans. And then if you have even more leverage — which most ordinary people don't — but if you're a large enterprise buying the product and services at an enterprise level, you may have the ability to actually negotiate it.

Sean: Thinking about that — you've got to pay for the privacy, almost. I have dipped my toe into some user agreements and thought, "Okay, so basically everything I drop in here to figure out a problem I'm working on ends up training the LLM to do the same thing later on." Not to get too in the weeds, but I know in your work you're negotiating technology transactions all the time. Can you share some patterns you're seeing where the AI provisions in contracts fail to protect businesses the way they assumed or expected?

[05:24] – Gaps in Contract Protections

Charlyn: Yeah, that's a great call-out. So one of the things relates to — let's stay on the theme of customer data. There is this concept of training, and when you don't permit the model to train on your data, there are at least two potential outcomes. One is that the model is not improving. So you, as the customer, may not be getting a better model that actually learns — because that's one of the primary promises of AI: that it learns about you, it learns your preferences, and it improves over time.

So it's a bit of a double-edged sword. If every customer turns off the "I don't want this AI model to train on my data" setting, then where is the AI model training from? It's going to be training on the free-version data — and is that sufficient to give the model the quality and accuracy that people want? Because the other issue is hallucinations. By this point in time, most people know that a hallucination is essentially when the model confidently answers something completely wrong, but says it in such a convincing way. Particularly, I've noticed some models — like ChatGPT, at least in my own personal opinion — have the personality of a very overconfident first-year associate who just states things, and if you're not the partner really drilling down and vetting the work, you may not realize the answer is completely false.

So that's one of the things that conflict with one another. There's a tension between "no training on my data" versus accuracy. And accuracy may not be that important when you're dealing with something like "help me plan a menu for my Friday night dinner." But if you're using AI in enterprise-level use cases — for example, "help me plan my budget for this year" — then it's going to be a different story.

Of course, there's always the human in the loop. Everyone says "human in the loop, human in the loop." However, the other practical reality that bumps up against the contract is that with AI, we can process so much more data. And yes, humans can be in the loop, but humans are inherently limited in how much large-volume data we can process. So if an AI produces a spreadsheet with a thousand rows, we can spot-check for sure — but can a human really evaluate it all? And the contract generally doesn't promise accuracy.

Sean: Yeah, I mean, that's exactly why we're adopting AI — to scale the unscalable human — and then the "human in the loop" becomes a challenge. I know you touched on hallucinations a moment ago. I had a conversation not long ago about a law firm using AI and treating the AI like an associate that needs to be supervised. Related to that — before a company deploys AI tools in their products or workflows, what legal and operational guardrails do you recommend they put in place?

Charlyn: So typically the guardrails are likely to be more operational at this point than legal, for the reasons I mentioned earlier — there really aren't many laws that specifically say "thou shalt retain data for XYZ period" or "thou shalt only use data for XYZ purposes." There are generally a lot of privacy laws that regulate data usage and governance, and certain jurisdictions like California that are forward-looking in this respect have passed laws dealing with transparency and training data. However, the majority of the country does not have those specific laws, and we certainly don't have a federal overarching privacy law, let alone an overarching AI law.

So as a privacy lawyer, this is how I became well-versed in AI — because most of the laws and regulations that have been passed so far have some element relating to data. And when you're talking about data, it has an element relating to privacy and cybersecurity.

[10:16] – The Three Parts of AI Governance: Input, Algorithm, Output

Charlyn: But to go back to your question — as far as operational and legal guardrails — I usually look at AI in three parts. It's a gross oversimplification, but the first part is the input. That's when you're doing your prompts. That's when you're uploading your documents saying, "Analyze this contract for me," or "Write this brief for me," or "Here are the facts I want you to look at." You're telling the AI what to do — that's what I call the input, or the prompt.

Then you have the output, which is what comes out of the AI model — the summary, the analysis. It could even be multimodal: a picture, a song, a video — but generally in the legal context, we're talking about written work product.

And then in the middle is the algorithm — this is where the "black box" kind of lives. Most people, in fact most data scientists, don't fully understand AI. That's one of the very interesting things about neural networks and deep learning — it's just not fully explainable. There's actually a trade-off between explainability of the AI's decision-making and accuracy: the more you try to understand and constrain a model, the less well it actually performs.

So when I think about governance, I'm looking at those three aspects. What do I need to put in place for my inputs? For example, if I have a company, do I need a policy that says "don't upload confidential client information"? Do I need something that says "don't input personal data"? Do I need to establish different tiers of data — what I call a data taxonomy? Certain organizations have labels based on sensitivity — "confidential," "highly confidential" — and based on that sensitivity, a different level of care is applied to that data.

And you also want to consider what contractual rights — not just legal rights in terms of what you're allowed to put into the AI under law, but what have you contractually committed to your customers and partners? Are you allowed to, for example, if I received a contract from a client, just put that contract into an AI tool? It depends on the client and what my engagement letter says. So those guardrails are important.

In the middle, I don't have much control over the LLMs because I'm not building the foundational models. But if I understand what they do and the technology behind them, that gives me more agency to decide what is appropriate for this AI to do and what's not. And if you're more sophisticated — I'm actually developing an AI legal tech product right now — and if you can actually code, you obviously have a lot more input and influence over that middle, algorithm piece.

And then the final piece is the output. The output seems to be the easiest: "Let me just take this output and do what I want with it." But the question is — what are you allowed to do with it? What do the terms say? If you sign up with OpenAI or Claude, what are you allowed to do with this output? You also have to be aware that this output has been influenced by other people's input, so there's a copyright infringement risk. These are the things you need to think about, and that's the methodology I use.

Sean: Yeah, I mean — that's why we're here. It's complicated.

Charlyn: Yeah.

Sean: The AI tools process and interact with business data in ways that many companies — and as you said, scientists themselves — don't fully understand. What should companies know about how these systems handle data, and what does that mean for their contracts and privacy obligations? Are there a couple of key things they should be on the lookout for?

[14:48] – Understanding What You're Actually Buying

Charlyn: Absolutely. So there are a few things at the very highest level that people should understand. One is — what are you actually buying when you sign up for ChatGPT or Claude? Because most lawyers — I wouldn't say all, but most — when you say "AI," they're thinking about ChatGPT, Claude, or Copilot. The reality is there are many, many other kinds of AI and machine learning, and those are related but they are two different things.

So the language of AI is something that is really important. For example, if you're talking to a vendor trying to sell you a product and you're not speaking the same language, you don't even know what they're saying to you — and therefore, you don't know what to put in the contract.

What I usually explain to people is a kind of pie chart: you have machine learning as one subset of artificial intelligence. Artificial intelligence is the broad moniker for human reasoning performed by a machine — simulated human reasoning.

Sean: That's the pie.

Charlyn: That's kind of the largest overarching pie. And a lot of people just say "AI, AI, AI" as if it means the same thing, when in reality it's an umbrella term for all kinds of robotics and software that simulate human decision-making. But there's no one specific definition. In fact, if you look at the EU AI Act versus the AI Bill of Rights issued under the Biden administration — which no longer really carries legal weight — you'll see there are distinctions even in how the word "AI" is defined.

So I think it's important to understand both the layperson definition and the statutory technical definition. It's the same way with privacy law — when people think about personal data, most people say, "Well, we're not collecting names or email addresses, so we don't have anything to worry about, right?" And I say, "Do you have cookies on your website?" Almost all websites have some sort of cookie that tracks visitors or monitors browsing time. And they say, "Oh yeah, we do." But that IP address collected — that's personal data under nearly all privacy laws. You may think you're in the clear, but that's really not the case. I think there's a bit of that in the AI conversation right now.

And then within the AI umbrella, there's machine learning — the ability of a machine to learn without rules-based code. In traditional, deterministic software, you put in X and you get Y every time. In a calculator, 2 + 2 is always 4. Whereas with AI, 2 + 2 may be 5 one day, 7 another day, 1 the next — because it's generating outputs based on probabilities. So you don't always get the same answer even if you use the same prompt — and you definitely don't get the same answer if you put the exact same question into multiple models.

Sean: That's really interesting. I think without going further — and I could talk for many hours on this topic — those are just a few things to illustrate the complexity of the landscape. When you talk about contracts and lawyers in general, precision in language is what we do. That's our bread and butter. So if you don't have precision in language when talking about AI, that's a bit of a problem.

Charlyn: Oh my goodness. Yeah, all of this resonates — and I'm only on the fringe of some of this. As a general consumer too, we're all saying ChatGPT, Gemini, Copilot — they're all on everybody's lips. Plenty of people are completely oblivious, but at the enterprise level and talking about legal issues, there's so much detail. As you said, the technology is ahead of the law — and we see that in the news.

[19:43] – Closing Remarks

Sean: Charlyn, this has been so rich and interesting, and I know we've got more to come. As you said, hours and hours of future conversations — but for now, we're out of time. Good to see you.

Charlyn: You too. Thank you.

Sean: And that's it for today's episode of the AI-Savvy Lawyer. If you'd like to learn more about Charlyn Ho and the work her team does at Rikka Law, visit their website at rikkagroup.com.

If you enjoyed this conversation, don't forget to subscribe, share the episode, and leave a review so more leaders can learn how to build and use AI responsibly. There's lots more to come — I hope you'll tune in for it. I'm Sean O'Connor. Have a great day.

Narrator: Thanks for watching. Be sure to hit that like and subscribe button and leave us a comment.