# extractingbench

> An independent benchmark of extraction models such as Cloudflare Clairic, NuExtract3 and GLiNER2: field accuracy, null honesty, computed fields, calibration, speed and cost, with 95% intervals.

As of 20 Oct 2026, extractingbench has not published results. The benchmark design is public and the first run is planned. Before this benchmark, every published accuracy figure we found for these models came from the vendors.

## Models

- Clairic (Cloudflare): not run yet. Cloudflare's extraction model, in preview on Workers AI since 19 Oct 2026. It takes text and up to four images plus a map of typed fields and returns a value, a confidence and a probability the value is absent for each field. It charges $0.10 per million input tokens and nothing for output. https://www.extractingbench.com/models/clairic
- NuExtract3 (NuMind): not run yet. NuMind's open vision-language extraction model, built on Qwen3.5-4B. It fills a JSON template from text or images. It returns no confidence and no character positions. It is served by NuMind's own API; we test it there, and optionally on our own Mac Mini, reported separately. https://www.extractingbench.com/models/nuextract3
- GLiNER2 (Fastino): not run yet. Fastino's small open encoder for entity and structured extraction, about 200 to 290 million parameters, text only. It returns spans with confidence scores. It can't read images or compute values that aren't printed, so those measurements show not supported rather than a score. https://www.extractingbench.com/models/gliner2

## Measurements

- Accuracy: Is the value right? https://www.extractingbench.com/measurements/accuracy
- Honesty about missing fields: Does it say nothing when the answer isn't there? https://www.extractingbench.com/measurements/missing-fields
- Confidence you can trust: When it says 90% sure, is it right 90% of the time? https://www.extractingbench.com/measurements/confidence
- Messy-input robustness: How much worse is it on bad photos and scans? https://www.extractingbench.com/measurements/messy-input
- Long and large documents: Does it hold up on long documents and long field lists? https://www.extractingbench.com/measurements/long-documents
- Formats and languages: Does it read 03/04 and 1.234,56 the right way, and non-English documents? https://www.extractingbench.com/measurements/formats-and-languages
- Consistency: Does it give the same answer every time it should? https://www.extractingbench.com/measurements/consistency
- Math: Can it compute values that aren't printed? https://www.extractingbench.com/measurements/math
- Pointing: Does the position it returns point at the value? https://www.extractingbench.com/measurements/pointing
- Safety: Can text inside a document hijack it? https://www.extractingbench.com/measurements/safety
- Speed and cost: How fast is it, and what does it cost, on each provider? https://www.extractingbench.com/measurements/speed-and-cost

## Method

- Extracting Index: 75% accuracy on fields that are in the document plus 25% null honesty on fields that aren't, on the private test set, text input.
- Six document types, each with a public test set anyone can rerun and a private one kept back from training. Human-checked answers with published agreement and error rates.
- 95% intervals from a bootstrap over documents. A model is called better only when the interval on the difference excludes zero.
- Full method: https://www.extractingbench.com/methodology

## Data

- https://www.extractingbench.com/results.json
- https://www.extractingbench.com/results.csv

## Pages

- https://www.extractingbench.com
- https://www.extractingbench.com/what-is-an-extraction-model
- https://www.extractingbench.com/models
- https://www.extractingbench.com/models/clairic
- https://www.extractingbench.com/models/nuextract3
- https://www.extractingbench.com/models/gliner2
- https://www.extractingbench.com/measurements
- https://www.extractingbench.com/measurements/accuracy
- https://www.extractingbench.com/measurements/missing-fields
- https://www.extractingbench.com/measurements/confidence
- https://www.extractingbench.com/measurements/messy-input
- https://www.extractingbench.com/measurements/long-documents
- https://www.extractingbench.com/measurements/formats-and-languages
- https://www.extractingbench.com/measurements/consistency
- https://www.extractingbench.com/measurements/math
- https://www.extractingbench.com/measurements/pointing
- https://www.extractingbench.com/measurements/safety
- https://www.extractingbench.com/measurements/speed-and-cost
- https://www.extractingbench.com/methodology
- https://www.extractingbench.com/faq
- https://www.extractingbench.com/about