What AI Document Review Can and Cannot Verify
AI is useful in document review when the task is checking a document against itself. It becomes dangerous when the task is deciding what the document means in law or in the market.
The useful division of labour
AI-assisted review works best on tasks with a checkable answer inside the document: extracting clauses into a table, finding every place a defined term appears, comparing two versions for changed numbers, summarising a long report, or drafting a first-pass checklist. The output can be verified against the source, and the reviewer’s time goes into judgement rather than transcription.
It works worst on tasks that require facts from outside the document: whether a clause is enforceable, whether a price is market, whether a party had authority to sign, or whether a regulator would accept a control. A model can produce fluent text about any of those questions, and fluent text is not evidence.
What the document can prove
A document proves what it says, on its face, in the version you are reading. It does not prove that the facts stated are true, that the parties understood them, or that the obligations are enforceable. Reviews that treat the document as a complete account of reality miss the gap between the paper and the transaction.
Within that boundary, AI can be genuinely useful. Ask it to locate every liability cap, every notice period, every reference to a schedule, or every place the payment terms change. Those are findable facts, and a reviewer can check a sample of them against the source. The value is coverage: a model does not get tired at page eighty.
Triage what to verify first
Set likelihood and impact for an unverified finding to decide whether it needs a full read, a targeted check, or a note in the open-questions list.
Citation discipline is the control
A review output is only usable if every claim points at a location a human can open. Require page, clause, and quoted text for each finding. If the output cannot cite, treat it as a draft hypothesis rather than a finding. This single rule catches most of the failure modes, because a model that is inventing a clause usually cannot produce a matching quotation.
Check a sample of citations yourself, including the ones that look most routine. Errors cluster in tables, annexes, and cross-references, which are exactly the places where a reviewer is most likely to trust the summary. If one citation in ten is wrong, the whole output needs re-derivation rather than spot correction.
Keep the source version with the output. A finding that was true of draft three may be false in the signed version, and without a version label nobody can tell which document the review examined.
Confidentiality and tool boundaries
Every tool that receives document text is a disclosure. Before using any assistant, establish where the text is processed, whether it is retained, whether it is used for training, who can access it, and what the provider’s deletion commitment is. If those answers are not available, the tool is not appropriate for client material.
Public chat interfaces are the highest-risk option because the material leaves your control entirely. Approved enterprise environments with contractual terms, retention limits, and access controls are the baseline for sensitive work. Even then, use the minimum necessary extract rather than the whole file, and keep a record of what was sent.
This site’s workflow reflects that boundary: matters are scoped by email, handling and permitted tools are agreed before files are exchanged, and there is no public upload or automated ingestion. The review output is decision support, not regulated advice.
Where humans remain accountable
The accountable reviewer is the person whose name is on the memo, not the model that drafted it. That means the human must understand the finding well enough to defend it, know which parts were machine-generated, and be able to say what was not checked. A memo that cannot be defended without reopening the tool is not a review.
Approval authority stays with people for a reason: the decision often depends on context that is not in the documents, such as the relationship with the counterparty, the strategy behind the transaction, or a risk appetite that has not been written down. A model cannot be accountable for those judgements, and delegating them does not transfer the accountability.
A workable checklist for AI-assisted review
Before the work: define the question, choose the documents, confirm the tool is approved for this material, and agree what the output must contain, including citations. During the work: keep the source version fixed, log what was sent, and check a sample of citations. After the work: verify the findings that drive the decision, record what was not checked, and have a named person sign the memo.
If any of those steps is skipped, the output is a draft. Drafts are useful, and they should be labelled as drafts in the handoff. The failure mode is not using AI; it is presenting unverified output as if it had been reviewed.
Evaluate before you rely on it
Before using an AI-assisted step in a live matter, test it on a document whose answer you already know. Pick a contract with a clause you have reviewed by hand, run the extraction, and compare. The test should include the awkward cases: a clause split across pages, a table with merged cells, an annex referenced twice, and a defined term used inconsistently.
Measure two things: recall and citation accuracy. Recall is whether the tool found the items you know are there. Citation accuracy is whether the location it gives actually contains the text it quoted. A tool with high recall and weak citations is still usable for a first pass, but only if every output is re-anchored to the source; a tool with weak recall and confident prose is not usable at all.
Write the evaluation result down, including the model, the date, and the document type. Models change, and the evaluation that justified a workflow in March may not describe the system you are using in September. Re-run the test when the tool or the provider changes, and keep the failed examples; they are the most useful training material for the people doing the review.
Questions to ask any review provider
Ask which tools process the material, where it is stored, how long it is retained, who can access it, and whether it is used for training. Ask what the deliverable contains, whether every finding cites a source, and who is accountable for accuracy. Ask what happens if a finding turns out to be wrong, and how quickly a correction is issued.
Ask the same questions of an internal team using AI, because the risks are identical. A provider that cannot answer them clearly is telling you something about its process. So is one that promises certainty about legal outcomes; that promise is itself a warning sign.
Scope a private review by email
Send the matter, file volume, formats, deadline, and decision question. A person confirms fit, handling, and timing before files move.