The challenge
Contract review was a two-day manual pass per agreement, and the bottleneck was not reading speed — it was cross-referencing clauses against precedent and policy held in a large internal library.
In legal work a fabricated citation is worse than no answer at all. A single invented case reference destroys trust in the entire tool, so citation integrity was the primary engineering constraint rather than a quality metric.
Approach
01
Treat citation as a verification problem
Generation proposes; a separate validator confirms every citation resolves to a real document and that the cited passage actually supports the claim. Unverifiable claims are stripped before the answer is returned.
02
Clause-level chunking
Chunks follow clause boundaries rather than fixed token windows, so a retrieved passage is a complete legal unit that a reviewer can read in isolation.
03
FAISS for a fixed, curated corpus
The precedent library is large but stable, which made a self-managed FAISS index the right trade-off over a hosted service — lower cost and full control over the data.
04
Reviewer stays in the loop
Output is a structured review with flagged clauses and citations, not a verdict. The paralegal validates in minutes instead of reading for two days.
Architecture
Parse
Contracts are segmented at clause level with position and heading metadata preserved.
Index
FAISS index over the curated precedent and policy library, rebuilt on library updates.
Retrieve
Clause-scoped retrieval per contract section rather than one query for the whole document.
Generate
Gemma 3 27B via LangChain produces the flagged review with candidate citations.
Validate
Citation validator resolves each reference and checks entailment against the source passage. Failures are dropped, not reworded.
Results
2 minDown from 2 full days
91%Citation accuracy
0Hallucinated citations
60%Lower review cost
6 wksDiscovery to production
What I took from it
The validator rejecting a claim is a feature, not a failure. Once we started reporting rejection rate as a health metric rather than hiding it, we could see exactly which contract types the retrieval layer was weak on.
Legal contract review built in 6 weeks. Two full days now takes two minutes at 91% citation accuracy. Their NLP expertise and delivery pace are a rare combination.
Stack
Orchestration
Models
Data
Validation