No contract lifecycle platform on your shortlist has published extraction accuracy measured against another CLM, so the only comparable score is the one you run yourself on your legacy contracts. For an extraction-first purchase, start the shortlist with Sirion and Icertis, keep Agiloft if your fields are unusual, treat Ironclad as the weakest fit for this specific job, and run a frontier model against your own schema as the control arm. Then decide on a 200-contract bake-off scored per field.
The workload this page normalises to: a 20,000-contract legacy repository, about a third of it scanned paper, 40 fields you care about (parties, term, renewal, notice period, liability cap, governing law, assignment, price escalators and so on), and amendment families where the live value sits in the third amendment, not the master agreement.
| Sirion | Icertis | Agiloft | Ironclad | Frontier LLM + your schema | |
|---|---|---|---|---|---|
| Published extraction accuracy | Self-run 2026 benchmark, 94.2% overall F1, against general LLMs only | No percentage published | No percentage published | No percentage published | You measure it |
| Head-to-head against a rival CLM | None | None | None | None | n/a |
| Custom field path | Extraction Agent; custom-field process unpublished | DiscoverAI with a validation spreadsheet before import | AI Trainer, starts from up to 20 documents, shows precision and recall | Custom AI properties, only inside AI clauses, 10 custom clauses by default | Edit the JSON schema |
| Legacy import evidence | Vendor benchmark | HERE: 70,000 contracts digitised | Vendor and press coverage | Smart Import, priced separately | None; you build the queue |
| Public price (checked Oct 18, 2026) | Quote only; UK G-Cloud 14 filing relayed at $95 to $128 per user per month, 100-user minimum | Quote only | Quote only | Quote only ("Get a Custom Quote") | Sonnet 5.5 at $2 / $10 per million tokens |
| Verdict for this job | Shortlist first | Shortlist first | Keep if fields are bespoke | Weakest fit | Control arm, not the system of record |
What Has Anyone Actually Published About Extraction Accuracy?
One vendor has published a number, and it measures that vendor against general-purpose models, never against another CLM. Sirion's 2026 clause-extraction benchmark reports an overall F1 of 94.2% for its extraction agent across more than 1,200 contract fields, with 96.1% on commercial terms, 93.8% on risk and compliance and 92.7% on operational clauses. That is a vendor claim from a vendor-run test. The comparison arms named on the page are GPT-4, Claude 3.5 Sonnet and Llama 3.1 70B, models several releases behind what you would use for a control arm today.
Converted to error rates, the categories are uneven. An F1 of 94.2% leaves roughly 5.8% error overall, but operational clauses sit at about 7.3% and risk and compliance at about 6.2%. On 20,000 contracts and 40 fields, a 6% field error rate is around 48,000 wrong or missing values that someone has to find.
F1 is the harmonic mean of precision (how many extracted values are right) and recall (how many of the true values were found). It is the correct metric to ask for. A single blended F1 across 1,200 fields still tells you nothing about the six fields your renewal process depends on.
Icertis publishes speed, not accuracy. Its legacy digitisation page says bulk import is "exceptionally accurate" and claims customers speed up legacy digitisation by up to 80%, with no percentage and no method. Ironclad's repository page carries a customer quote saying legacy upload is 40-50% faster with two to three times more contract data, which is a testimonial. Agiloft's AI Trainer shows precision and recall to the user on the models you train, per 2023 launch coverage, which is the closest thing in this group to a scorecard you control.
So there is no independent head-to-head. Gartner placement does not fill the gap either. The Magic Quadrant grades ability to execute and completeness of vision, which is how Sirion's own summary of its 2025 Leader placement describes it, and it publishes no extraction scores.
Where Is Each Platform Strongest for a Legacy Repository?
Sirion and Icertis are built around the post-signature repository at enterprise scale, which is the job described here. Agiloft wins on configurability. Ironclad is strongest where the contract is created, which is the other half of a CLM.
Sirion is the only vendor in this group that put a category-level accuracy figure in public, for a product it sells as its Extraction Agent. Ask it to show the custom-field process on your schema, because the benchmark covers its own field set. Who should not pick it: a legal team under roughly 100 users. The Spellbook writeup of Sirion's UK G-Cloud 14 filing (a competitor's page relaying a public government filing dated May 7, 2024) puts Contract Intelligence at $95 to $128 per user per month with a 100-user minimum, and an integration at about $13,500 a year (converted from £10,000). Those are converted, unofficial figures. Sirion sells on quote in the US.
Icertis has the strongest scale evidence. HERE Technologies digitised 70,000 legacy contracts four months ahead of schedule, by Icertis's account, and the case study reports no accuracy figure. Its import design is the most useful part for this job: in DiscoverAI, extracted attributes land in a colour-coded spreadsheet that reviewers check before bulk import, so the review queue exists before bad data reaches the repository. Who should not pick it: a team that cannot staff an implementation. Third-party estimates (from Hyperstart, a competing CLM, citing Vendr) put the median Icertis contract at $88,000 a year; Icertis publishes no price.
Agiloft is the pick when your fields are bespoke, such as industry-specific rebate mechanics or regulatory clauses no pre-trained model has seen. AI Trainer lets a non-technical user upload up to 20 example documents for a clause, tag them, train on about three quarters and test on the rest. That gives you a concrete answer to how many labelled examples a new field takes, and it is a low number. The coverage dates from the August 2023 launch, so confirm the current limits. Who should not pick it: a team that wants extraction to work out of the box with no administrator. Agiloft's pricing page lists no prices.
Ironclad loses this comparison, for this job. Its custom AI properties are detectable only inside Ironclad's AI clauses or custom AI clauses, and each customer gets 10 custom AI clauses by default. A 40-field legacy schema with unusual terms runs into that ceiling. Smart Import is priced separately ("subject to Smart Import pricing"), so the extraction you are evaluating is a separate line item on top of the platform. Who should pick it anyway: a legal-led team whose main problem is intake, drafting and approval for new contracts, with a modest legacy archive. The pricing page offers "Get a Custom Quote" and no figures.
What Does the Do-It-Yourself Baseline Cost?
A frontier model with your own schema extracts the whole 20,000-contract repository for a few hundred to a couple of thousand dollars in model fees, which is why it belongs in your bake-off as the control arm. Assume 12,000 input tokens and 2,000 output tokens per contract. At Claude Sonnet 5.5's list price of $2 per million input tokens and $10 per million output, with the 50% batch discount, that is about $240 for input and $200 for output, roughly $440. On Opus 5.5 at $4 and $20 with no batch discount it is about $1,760. Prices checked October 18, 2026.
Structured outputs constrain the response to your JSON schema, so you get valid, typed fields every time. The guarantee covers format. A schema-valid liability cap can still be the wrong number.
The model bill is the cheap part. What you would be building yourself is everything else: OCR on scanned paper (compared in our Textract vs Azure vs Gemini teardown), splitting multi-agreement PDFs, amendment linking, a review queue, an audit trail and a repository people will use. Who should not run DIY as the system of record: any team without engineers who will own it for years. Who should run it anyway: everyone, as the control arm. If a CLM's extraction cannot beat a schema you wrote in an afternoon on your own paper, you are paying for the workflow, and you should price it that way.
Why Does Legacy Migration Cost More Than Extraction?
The expensive part of a legacy migration is the paper and the contract family, and no accuracy benchmark measures either. Elevate's write-up of a 33,700-contract biotech migration lists the actual problems: non-uniform naming, inaccurate parent-child hierarchies and multiple agreements scanned into a single PDF. The fix was a separate process to link amendments to originals and split combined PDFs into individual records.
This is where vendor numbers stop transferring to you. A field extracted correctly from the master agreement is wrong if a later amendment changed it. Ask each vendor to show, on your documents, how an amendment is linked to its parent and which value the record displays when they disagree.
Scanned paper is the other cost. Every extraction model reads OCR output, so a misread digit in a liability cap is an extraction error the model cannot see. Practitioner case studies report figures like roughly 90% accuracy on a 2,000-contract migration with Kira, with no stated method for how accuracy was measured.
How Should You Run the 200-Contract Bake-Off?
Score each vendor per field, on your own contracts, against values a lawyer marked before any vendor touched them. A 200-contract set is enough to separate vendors on your common fields and too small for rare ones, so sample deliberately.
Build the set so it looks like your repository: a share of scanned paper matching your archive, at least 30 amendment families, your top counterparty paper, and your worst-formatted legacy templates. Have two reviewers mark gold values independently and adjudicate disagreements. Include contracts where a field is genuinely absent.
A September 2026 re-evaluation of a public RoBERTa model on the CUAD contract dataset found it answered 1,335 of 2,938 questions that had no annotated answer, a 45.4% false-positive rate on absent clauses, and that only 9 of CUAD's 41 categories had at least 30 positive and 30 negative contracts. A vendor that finds a "termination for convenience" clause in every contract will look great on recall. Score absent fields separately, and expect thin per-field support on rare clauses unless you oversample them.
CUAD itself (510 contracts, 41 clause categories, 13,000+ lawyer-supervised labels, CC BY 4.0) is useful for building and testing your scoring harness before your own gold set is ready. It cannot replace your paper.
Measure four things per vendor: precision and recall per field, the false-positive rate on absent fields, amendment linkage accuracy, and reviewer minutes per contract on the low-confidence queue. Reviewer minutes are what your migration budget is made of.
What Predicts Regret, and What Changes the Answer?
Regret in this category comes from three places: buying on a blended accuracy number, discovering the custom-field ceiling after signature, and underpricing the review queue. Each one is testable before you sign.
The answer changes in a few specific cases. If your legacy archive is small and most of your pain is new-contract intake, Ironclad moves up and the extraction ranking matters less. If more than half your fields are bespoke, Agiloft moves ahead of Sirion and Icertis. If your DIY control arm matches the vendors on your top 10 fields, negotiate the extraction module down hard, because you have proved you can replace it. And if a vendor will not take part in a scored bake-off on your documents, drop it from the shortlist.
This Week:
- Write the field list: the 40 fields, the source clause for each, and which ones drive renewals, liability or revenue.
- Pull 200 contracts stratified by scan quality, amendment family and counterparty paper, and start the lawyer-marked gold set.
This Month:
- Send the same 200 documents to each shortlisted vendor and run your schema through a frontier model as the control arm.
- Score precision and recall per field, absent-field false positives, amendment linkage and reviewer minutes. Compare like for like.
Before You Sign:
- Put per-field accuracy thresholds and the review-queue process into the statement of work as acceptance criteria, with the right to rerun the test after model updates.
- Get written answers on custom-field limits, import pricing and what happens to unconfirmed values when the vendor updates its model.
The Bottom Line
CLM extraction in 2026 has the same problem enterprise OCR and document-processing buyers have had for years: each vendor's figure, where one exists, was measured on documents you never see, so no two figures compare. Start with Sirion and Icertis for a large post-signature repository, keep Agiloft for bespoke fields, and hold Ironclad for intake-led buyers. Whoever wins, the deciding number should come from your 200 contracts, scored per field, with a lawyer's answer key.
Write the field list before you take the first demo.
Continue Reading
- Harvey vs CoCounsel vs Legora: Buy the Library, Not the Model
- Textract vs Azure vs Gemini: Split OCR From Extraction
- Astra for Law Scored 54% on a Set Vals Never Publishes
- Mitratech Buys BotDojo, Whose Terms Offer No Data Export
- Valsoft Bought Square 9. Who Funds the Next Model?
- Harvey Swapped Models After Agents Sank Its Margin to -50%