Legal & contracts
In contract AI, an unsupported sentence is a liability
Grounding to a specific clause version is not a feature. It is the product.
What I have shipped here
Enterprise CLM
contract lifecycle platform in production
I designed and built an enterprise Contract Lifecycle Management platform covering authoring, review, approvals, amendments, clause libraries and template-based generation, on Angular, Node.js, PostgreSQL and CKEditor — including the approval and amendment workflows legal and business teams ran on daily.
Who I work with here
- CLM and legal-tech platforms adding review or drafting AI
- In-house legal operations teams automating contract review
- Procurement teams triaging supplier agreements at volume
Systems I see in this domain
- Clause extraction and classification
- Contract review and risk flagging
- Obligation and renewal-date extraction
- Template and clause-library assistants
- Redline generation and comparison
What makes agents hard in legal & contracts
Legal users do not want a summary. They want to know which clause says so, in which version, and they will click through to check. That inverts the usual product instinct: fluent prose is nearly worthless here, and a terse answer with an exact citation is the whole value.
Which makes grounding the product rather than a safety feature. An answer that is correct but uncited is unusable, because the reviewer has to go and find the clause anyway — at which point you have added a step rather than removed one.
The second constraint is that contract language is adversarial to naive retrieval in ways ordinary documents are not. Defined terms carry meaning from a definitions section fifty pages away. Cross-references chain: Section 7.2(b) modifies 7.1, which an amendment supersedes. Precedence clauses decide which document wins. Chunk a contract on token counts and you reliably destroy all three.
Third, the audit trail is a legal artefact. If a model-assisted review missed an indemnity cap, the question becomes what the system was shown and what a human approved. That has to be reconstructable years later, long after the model version you used has been retired.
Clause-grounding requirements
What has to be true before a legal team will rely on model output. Each item is checkable in code, which is what distinguishes a system a general counsel will sign off from a demo.
- 1Every assertion cites a specific clause identifier and the document version it came from — not a page number, not a similarity score
- 2Citations resolve: clicking one lands on the exact clause text the model was shown, retrieved from storage rather than regenerated
- 3Defined terms are resolved before or during retrieval, so a clause using "Confidential Information" carries its definition into context
- 4Cross-references are followed to a bounded depth, and the chain is recorded — Section 7.2(b) referencing 7.1 must bring 7.1 along
- 5Amendment and precedence chains are respected: the system knows which version is operative and which document controls on conflict
- 6Synthesis across documents is either forbidden or explicitly attributed per source — an unattributed merged claim is the most dangerous output shape
- 7"Not found in this contract" is a first-class answer, and the system prefers it to inference
- 8The exact retrieved context is stored immutably with the output, so a review can be reconstructed years later
- 9Model version and prompt version are recorded per document produced, and pinned — never a floating alias
- 10Redlines and diffs are generated deterministically by code, with the model proposing changes rather than rendering the comparison
- 11Human sign-off is recorded with what the reviewer was shown, not merely that they approved
- 12Numbers, dates, durations and monetary caps are extracted as structured fields and validated, never narrated from prose
The last item catches more real errors than any other. Caps, notice periods and durations narrated in prose are where models quietly transpose digits, and it is exactly the detail a reviewer skims past because the sentence reads well.
Why contract retrieval fails when ordinary RAG works
Contracts break standard retrieval in structural ways. Each of these is a solved problem once named, and each is invisible if you only measure end-to-end answer quality.
| Structure | How naive retrieval fails | What to do |
|---|---|---|
| Defined terms | "Confidential Information" retrieved without its definition; the model applies the everyday meaning | Build a definitions index per document and inject resolved definitions into the retrieved context |
| Cross-references | 7.2(b) retrieved alone; the obligation it modifies in 7.1 is absent | Parse the reference graph and expand to referenced clauses within a bounded depth |
| Amendments and restatements | A superseded clause retrieves as confidently as the operative one | Model document lineage explicitly; mark superseded text as non-retrievable or clearly flagged |
| Precedence and order clauses | MSA and SOW conflict; the model picks whichever it retrieved | Encode precedence as data and resolve conflicts in code before generation |
| Schedules, exhibits and annexes | Commercial terms live in an exhibit that was never indexed with the body | Treat the contract family as one retrieval unit with typed parts |
| Clause boundaries | Fixed-size chunking splits an indemnity mid-sentence; neither half retrieves usefully | Chunk on legal structure — clause and sub-clause — with a parent reference for expansion |
| Governing-law variation | The same clause type means different things across jurisdictions | Carry jurisdiction as retrieval metadata and in the prompt context |
Measure retrieval separately from answer quality here, or you will spend weeks tuning generation prompts while the defect is that the definitions section was never fetched.
The failure modes that dominate here
Every agent can fail in all nine ways. These are the three that cause the most damage in this domain specifically.
high · very common
The answer is wrong because retrieval returned the wrong context
The model is behaving correctly given what it was handed — and what it was handed was wrong.
high · common
Output is cut off mid-way and nothing reports an error
The model hits its output limit and stops.
medium · common
The agent ignores its instructions once the conversation gets long
Early turns follow the rules; by turn twenty the agent has drifted — wrong tone, abandoned constraints, forgotten refusals.
How I would approach it
The engagements that apply most often in this domain. Each has the full method and effort estimates.
Agent eval suites
Stop shipping prompt changes as unmeasured bets.
A suite your team owns that fails the build when agent quality drops.
The methodLLM observability & tracing
One trace ID that explains the whole run.
Any bad run can be pulled up, replayed, and explained in minutes.
The methodMulti-tenant AI isolation
A prompt is not an access-control mechanism.
Isolation that holds even when the model behaves unexpectedly.
The methodQuestions from this domain
Can an LLM be trusted to review contracts?
To triage, extract and flag — yes, with clause-level citations and a human reviewing anything material. To decide — no, and no serious legal team is asking for that.
The value is in narrowing a 60-page agreement to the eight clauses a lawyer should read, with each flag citing its source. That is a large, real saving and it does not require the model to be right unaided.
Why does standard RAG perform badly on contracts?
Because contract meaning lives in structure that chunking destroys: defined terms fifty pages away, cross-reference chains, amendment lineage, and precedence between documents.
Retrieve a clause without its definitions and referenced sub-clauses and the model reasons from incomplete text — confidently, because the fragment reads like complete language. Fixing this is retrieval engineering, not prompt engineering.
What audit trail does contract AI need?
Immutable storage of exactly what the model was shown, the model and prompt versions, the output, and what the human reviewer saw before approving — reconstructable years later, after the model version has been retired.
A "reviewed by" flag and a timestamp is not sufficient. If a missed indemnity cap becomes a dispute, the question is what the system presented and what the reviewer actually saw.
How do we stop the model inventing clause language?
Structurally rather than by instruction. Require every assertion to cite a clause ID that resolves to stored text, verify each claim against its cited clause before returning, extract numbers and dates as validated fields rather than narrating them, and make "not found in this contract" a first-class answer.
Generate redlines deterministically in code, with the model proposing changes rather than rendering the diff. Prompt instructions like "do not invent clauses" are advisory and will eventually be ignored.
Building this in legal & contracts?
Tell me what your system does and what worries you. I have shipped in this domain, so I can usually name the specific constraint you are about to hit — and that answer is free.