LexDye Blog · LexDye AI tools · Published · LexDye, Inc.

Nobody likes being double-checked, not even AI

Summary. In a real drafting session, a legal AI first dismissed LexDye's defined-terms report as mostly noise. When asked to look again, it agreed that LexDye had caught two problems it had missed: a term used before its definition, which had survived several of the AI's own reviews, and a definition the AI had written into its own draft but never used. LexDye now tells every AI that its report is a quality-control check, not a competing opinion, and that every entry needs a decision.

LexDye finds them. Your AI reviews them. You don't have to.

Chat exchange. AI, first answer when asked about LexDye: Most of its output was noise for this document. Us: Well, but it found things you didn't. AI, second answer: You're right. My earlier assessment undersold the report. AI, then: That paragraph went through several of my reviews and I never spotted the problem. LexDye did.

What happened

A LexDye user asked a legal AI to help draft a short side letter. Between drafts, the user ran each version through the LexDye Definition Tracker and gave the AI LexDye's report.

When the drafting was done, the user asked the AI how useful the report had been. The AI answered: "Most of its output was noise for this document."

The user pushed back: LexDye had found things the AI hadn't. The AI reconsidered. "You're right," it said. "My earlier assessment undersold the report."

What LexDye caught

The AI's own summary: "LexDye isn't competing with an AI review; it supports one."

Why the first answer was "noise"

LexDye flags generously on purpose. It was built for attorney review, where a harmless flag costs a few seconds and a missed definition problem can cost much more. So on a short document, most entries in the report are harmless: names, headings, party labels, and terms that come from a related agreement.

The AI judged the report as a whole, saw mostly harmless entries, and called it noise. The two real problems were in the list, but the overall verdict buried them. Busy human reviewers do the same thing with long checklists.

What we changed

We rewrote the instructions that come with every LexDye report, whether the report is copied into a chat, downloaded, or produced by the command-line version for AI agents. The AI is now told:

Why this matters

AI reviewers are good at judgment. What an AI doesn't do reliably is check every defined and undefined term, every time, including in text it wrote itself. In LexDye's published defined-terms benchmark, an AI agent found 18 of 25 planted errors when asked to "review before signing" and 25 of 25 only when the prompt listed every check. See the benchmark write-up.

LexDye runs every definition check, every time, and hands the AI one list to review. The AI decides what matters. Neither has to do the other's job.

LexDye finds them. Your AI reviews them. You don't have to.

Limitations

This is one real drafting session on a short document, not a controlled test. The quotations are taken from the AI's answers. The AI's name, the deal, the parties, and the specific terms are left out to protect the user's confidentiality. LexDye's checks are about definitions only: capitalized terms that are never defined, terms defined more than once, terms used before their definition, definitions that are never used, and defined terms that also appear in lower case.

Try it

The LexDye Definition Tracker is free and runs entirely on your computer. Download the beta at lexdye.com/definition-tracker-beta (access code 06511). More about the tool: fact sheet.

Cite as: LexDye, Inc., "Nobody likes being double-checked, not even AI," LexDye Blog, November 21, 2026, https://www.lexdye.com/blog/ai-double-checked.