To AI or not to AI – that is the question

Want to hear a story? It’s about a General Counsel who stood up at a conference and talked about conducting a major contract review – and trialing an AI tool to do it.

Maybe the GC wouldn’t have gone quite so public if they didn’t have a happy ending to talk about.

But as it goes, this organization – which would have expected to pay external counsel around £200,000 for the work – instead paid £20,000. And moreover, the GC felt the final result was likely better than if human lawyers had done it.

What’s the moral of this story? Well, I suppose it’s that AI can work; and AI can save you a lot of money. If you trust it.

Trust issues

It emerged in subsequent conversations with others in the room that many legal teams in attendance had considered trying out AI in the same way. However, a big sticking point was that the vendors were failing the infosec review.

Such a review needs to be far more wide-ranging than mere cybersecurity. You need to be asking whether unacceptable risks might be run in relation to confidentiality, privacy, intellectual property, regulatory compliance and AI governance.

The jeopardy also ramps up if the contract(s) in question contain personal data, employee data, health data or regulated information.

Here are just some of the many, many questions you may want to ask.

• Has our chosen vendor been sighted or penalized by the ICO or other regulatory body for data protection violations?
• Where does our data go?
• Is our data encrypted in transit, at rest, or both?
• Who can access the data at the vendor?
• Are subcontractors involved?
• What happens to our data subsequently: is it retained, destroyed, archived?
• Is customer data sent to an external LLM provider?
• Will our data be used for model training?
• Is the vendor’s model fine-tuned on customer content?
• Can outputs leak customer information?
• What are the hallucination risks (that the AI invents case law, fabricates citations, generates plausible but incorrect legal analysis, etc)?
• Are there role-based access controls?
• Is there audit logging?
• In what jurisdictions are the vendors servers located?
• Can data subjects exercise deletion rights?
• What recognized security certifications are in place, e.g. ISO 27000 and /or SOC 2 Type II reports?
• What security architecture testing can they produce, e.g. penetration testing results, incident response policies, business continuity/disaster recovery plans?

Legal privilege and operational risks

You also need to ask yourselves some questions such as if privilege could be compromised; whether third-party processing will affect confidentiality protections; whether uploaded contracts remain legally protected; and if the vendor contractually recognizes your confidentiality obligations.

You’ll have to assess the risks to your business of relying on the AI outputs of the exercise. What if there are false positives or negatives? How are you validating the outputs? What degree of human review do you require?

You’ll have to check that the vendor is financially stable, has an acceptable ownership structure, and operates in jurisdictions you’re comfortable with.

It’s also worth checking the vendor's AI governance criteria. Do they operate to acceptable use policies, have human-in-the-loop requirements, and conduct AI ethics reviews?

It’s absolutely true that you can get a very healthy ROI from AI in legal. I’ve seen the proof of that. But at the same time, know the risks. I’ve seen proof of them too.