Nobody likes being double-checked, not even AI
Summary. In a real drafting session, a legal AI first dismissed LexDye's defined-terms report as mostly noise. When asked to look again, it agreed that LexDye had caught two problems it had missed: a term used before its definition, which had survived several of the AI's own reviews, and a definition the AI had written into its own draft but never used. LexDye now tells every AI that its report is a quality-control check, not a competing opinion, and that every entry needs a decision.
LexDye finds them. Your AI reviews them. You don't have to.

What happened
A LexDye user asked a legal AI to help draft a short side letter. Between drafts, the user ran each version through the LexDye Definition Tracker and gave the AI LexDye's report.
When the drafting was done, the user asked the AI how useful the report had been. The AI answered: "Most of its output was noise for this document."
The user pushed back: LexDye had found things the AI hadn't. The AI reconsidered. "You're right," it said. "My earlier assessment undersold the report."
What LexDye caught
- A term used before its definition. A paragraph relied on a defined term that the side letter only defined later. The AI said: "That paragraph went through several of my reviews and I never spotted the problem. LexDye did." When the AI went back to fix it, the rewrite did more than move a definition. It made the whole passage easier to follow.
- An unused definition in the AI's own draft. The AI had defined a term and then never used it. The AI called LexDye's flag "a check on my own drafting."
The AI's own summary: "LexDye isn't competing with an AI review; it supports one."
Why the first answer was "noise"
LexDye flags generously on purpose. It was built for attorney review, where a harmless flag costs a few seconds and a missed definition problem can cost much more. So on a short document, most entries in the report are harmless: names, headings, party labels, and terms that come from a related agreement.
The AI judged the report as a whole, saw mostly harmless entries, and called it noise. The two real problems were in the list, but the overall verdict buried them. Busy human reviewers do the same thing with long checklists.
What we changed
We rewrote the instructions that come with every LexDye report, whether the report is copied into a chat, downloaded, or produced by the command-line version for AI agents. The AI is now told:
- This is a quality-control check. The report checks the AI's own review or drafting. It is not an opinion to weigh against the AI's own.
- No summary verdicts. The AI may not call the report "noise" or "mostly false positives." Every entry gets a decision and a reason.
- Misses first. The answer opens with "Caught by LexDye": the real problems that the AI's own review or drafting missed.
- Look closer. A real flag is a reason to reread the whole passage. Where better wording would fix the flag and also make the passage clearer, the AI proposes it.
- Expect harmless entries. Common harmless types, such as names, headings, and party labels, are named in advance so the AI can dispose of them quickly.
- Definitions only. LexDye checks definitions: undefined terms, duplicate definitions, uses before definition, unused definitions, and lower-case uses. It doesn't check style, grammar, or cross-references, and the AI shouldn't fault it for that.
Why this matters
AI reviewers are good at judgment. What an AI doesn't do reliably is check every defined and undefined term, every time, including in text it wrote itself. In LexDye's published defined-terms benchmark, an AI agent found 18 of 25 planted errors when asked to "review before signing" and 25 of 25 only when the prompt listed every check. See the benchmark write-up.
LexDye runs every definition check, every time, and hands the AI one list to review. The AI decides what matters. Neither has to do the other's job.
LexDye finds them. Your AI reviews them. You don't have to.
Limitations
This is one real drafting session on a short document, not a controlled test. The quotations are taken from the AI's answers. The AI's name, the deal, the parties, and the specific terms are left out to protect the user's confidentiality. LexDye's checks are about definitions only: capitalized terms that are never defined, terms defined more than once, terms used before their definition, definitions that are never used, and defined terms that also appear in lower case.
Try it
The LexDye Definition Tracker is free and runs entirely on your computer. Download the beta at lexdye.com/definition-tracker-beta (access code 06511). More about the tool: fact sheet.