How is an extraction model different from an LLM?

An LLM generates text one token at a time and has to be asked to format its answer as JSON. An extraction model is built to return typed values directly, often in a single pass with no generated text, and often reports how sure it is and where in the document it found each value.

What are examples of extraction models?

As of 20 Oct 2026: Cloudflare's Clairic (hosted on Workers AI, in preview since 19 Oct 2026), NuMind's NuExtract3 (open weights, Apache 2.0) and Fastino's GLiNER2 (open weights, Apache 2.0, text only).

What is a System One model?

"System One" is a name used for models that answer a typed question in one forward pass instead of reasoning step by step in text. Extraction models fill in typed values; decision models pick from options the caller lists.

What is an extraction model?

Q: What is an extraction model?

An extraction model takes a document (text, and sometimes page images) plus a list of typed fields, and returns one value per field instead of free text. Each value usually comes with a confidence and a probability that the value isn't in the document.

An extraction model reads a document and fills in a list of typed fields you give it: a total as a number, a due date as a date, a list of line items. It returns a value for each field, not a paragraph of text. Most also return a confidence for each value and the probability that the value isn't in the document at all.

What goes in, what comes out

You send three things:

The document, as text, page images, or both.
The fields you want, each with a type (text, number, date, duration, list) and an optional one-line instruction.
Optionally a reference date, so "12 Oct" without a year resolves the way you expect.

You get back, for each field:

A typed value, or null when the document doesn't contain it.
A confidence, and a probability that the value is missing.
For some models, the character position of the value in the input, so you can highlight where it came from.
For estimated values, a median and an 80% range.

How it differs from an LLM with JSON output

An LLM writes its answer token by token, and JSON is just text it was asked to produce. It can be made reliable, but it has no built-in notion of how sure it is about each field, and it pays for every output token. An extraction model is built for the job: the output shape is fixed by the field list, the answer comes in one pass, and the confidence is part of the output. Cloudflare charges for Clairic's input tokens only.

That doesn't make extraction models better by default. Whether they are more accurate, more honest about missing fields, and better calibrated is exactly what this benchmark measures.

How it differs from document-extraction APIs

Services such as Amazon Textract or Azure Document Intelligence are pipelines with OCR, layout analysis and prebuilt document types. An extraction model is a single model you give any field list to. This benchmark covers extraction models only.

Extraction models as of 20 Oct 2026

Clairic, Cloudflare. Hosted on Workers AI as @cf/cloudflare/clairic, in preview since 19 Oct 2026. Text and up to four images, 65,536-token context, $0.10 per million input tokens. Cloudflare says it intends to release open weights.
NuExtract3, NuMind. Open weights under Apache 2.0, built on Qwen3.5-4B. Reads text and images and fills a JSON template. Hosted by NuMind's own API.
GLiNER2, Fastino. A small open encoder, Apache 2.0, text only, that extracts spans and structured records with confidence scores. Hosted by Fastino's own API.

What vendors say, and what is known independently

Cloudflare's launch post for Clairic reports a claims-processing pipeline that ran in 19.3 seconds instead of 61.7 seconds and cost $6,032 instead of $11,740 per million claims compared with an all-LLM pipeline, and a 21-day forecast that was off by 3.3% on average with 17 of 21 actual values inside the 80% interval. Cloudflare also lists limits: multi-step arithmetic is weak, forecasting is basic, and intervals can run narrow on unfamiliar data. As of 20 Oct 2026, those are vendor figures; we found no independent accuracy test. Here is how we plan to test them.