Correctness Over Confidence: The 'I Don't Know' Test

AI is the best salesman you will ever meet. It is fluent, confident, and fast. But a good salesman makes you check whether the product is real. The gap between a fluent answer and a correct one is where legal work breaks. Fluency can hide fake law - an invented citation (a format checker like HAQQ's Legal Citation Formatter won't tell you a case doesn't exist, only that it's styled correctly), a clause that reads clean and means nothing, a confidently wrong position.

In most fields, right 99% of the time is excellent. In law it is not enough, because the whole game is the 1% when the model is wrong and you cannot tell. You walk into a negotiation on a weak position without knowing it. The risk is not the average accuracy. It is the undetected error.

And a per-step accuracy is not a per-task accuracy. Any real piece of legal work is a chain of steps, and a chain is only as good as the product of its links - which is a much smaller number than people expect. Set the accuracy you think your tool has and see what it does over twenty steps, then add a check partway through.

"AI lawyer" is close to an oxymoron. Not because AI is weak - it can out-execute most people on speed, volume, and recall - but because a lawyer must be able to take responsibility and accountability, one of the constraints covered in Legal AI Fundamentals. You cannot outsource judgment to a thing that cannot be held accountable. That is why a human stays in the loop, and why AI is a partner, not a source of law.

Magnus Carlsen is the best chess player in history, and he will never beat Stockfish. Computers are simply better at the execution. People still play chess, still watch Carlsen, and still value the human. Law is the same - except the stakes are higher, so the human's job shifts to the part machines cannot own: judgment about what "winning" even means for this client, in this matter.

There is a simple test for a legal AI you can trust. Can it say three words: "I don't know"? Most people can't, and almost no model can either - "I don't know" sits outside its parameters, so it guesses fluently instead. The first legal AI that will say "I don't know" when it doesn't know is the one you can trust, because it would rather stop than fabricate. Look for the tool that values correctness over confidence.

A warning most people miss: AI tends to complete you. Tell it "there's a problem with clause B" and it will find a problem with clause B, because agreeing finishes your thought. Separate your suspicion from its output, or you build a private echo chamber and get confidently misled. Ask neutrally. Let it disagree.

The check does not have to be a person - it can be another model. Run several AIs over the same document and compare, the same instinct behind practitioner-led benchmarks. Direct the review where you are genuinely suspicious, not where you have already decided. Then let a human give the final judgment on what is right for this case.