The challenge
Contract review was a 2 day manual pass per agreement, and the bottleneck was not reading speed. It was cross-referencing clauses against precedent and policy held in a large internal library.
In legal work a fabricated citation is worse than no answer at all. A single invented case reference destroys trust in the entire tool, so citation integrity was the primary engineering constraint rather than a quality metric.
Approach
01
Treat citation as a verification problem
Generation proposes; a separate validator confirms every citation resolves to a real document and that the cited passage supports the claim. Unverifiable claims are stripped before the answer is returned.
02
Clause-level chunking
Chunks follow clause boundaries rather than fixed token windows, so a retrieved passage is a complete legal unit that a reviewer can read in isolation.
03
FAISS for a fixed, curated corpus
The precedent library is large but stable, which made a self-managed FAISS index the right trade-off over a hosted service: lower cost and full control over the data.
04
Reviewer stays in the loop
Output is a structured review with flagged clauses and citations, not a verdict. The paralegal validates in minutes instead of reading for 2 days.
Architecture
Parse
Contracts are segmented at clause level with position and heading metadata preserved.
Index
FAISS index over the curated precedent and policy library, rebuilt on library updates.
Retrieve
Clause-scoped retrieval per contract section rather than one query for the whole document.
Generate
Gemma 3 27B via LangChain produces the flagged review with candidate citations.
Validate
Citation validator resolves each reference and checks entailment against the source passage. Failures are dropped, not reworded.
Results
2 minDown from 2 full days
91%Citation accuracy
0Hallucinated citations
60%Lower review cost
6 wksDiscovery to production
What I took from it
The validator rejecting a claim is a feature, not a failure. Once we started reporting rejection rate as a health metric rather than hiding it, we could see exactly which contract types the retrieval layer was weak on.
Legal contract review built in 6 weeks. 2 full days now takes 2 minutes at 91% citation accuracy. Their NLP expertise and delivery pace are a rare combination.
Stack
Orchestration
Models
Data
Validation