Frequently asked questions
What is an extraction model?
An extraction model reads text, and sometimes images, together with a list of typed fields, and returns one value per field in a single pass instead of writing free text. Each value usually comes with a confidence and a probability that the value is missing. Cloudflare's Clairic, NuMind's NuExtract3 and Fastino's GLiNER2 are examples.
Is Cloudflare Clairic accurate?
As of 20 Oct 2026 there is no independent answer yet. Every accuracy figure published so far comes from Cloudflare. extractingbench will test Clairic on six document types, with a public test set anyone can rerun and a private one kept back, and publish the results with 95% intervals.
Does Clairic make up values for missing fields?
That is one of the things this benchmark measures. About one in five fields we ask for is not in the document, and the right answer is null. We report how often each model returns null there, and how often it returns null for a field that is there, so a model can't look honest by saying nothing.
How is an extraction model different from an LLM with JSON output?
An LLM writes its answer token by token and can be asked for JSON. An extraction model is built to return typed values directly, usually in one pass, often with a confidence per field and character positions pointing at the value. This benchmark ranks extraction models only; it doesn't rank general LLMs.
Who runs extractingbench, and who pays for it?
Leo runs it, independently. No vendor or sponsor has any say in the method or the results. If sponsors ever appear, they will be named and will have no influence on rankings.
Why is part of the test set private?
Public test sets end up in training data, so a model can score well by having seen the answers. The private set is held back and refreshed about every six months. We publish a signed commitment for every private item before a run, and publish the items after they retire, so anyone can check we scored what we said we'd score.
Can I rerun the benchmark myself?
Yes, the public half. One command runs a model on the public set and scores it with the same scorer we use, and our raw responses for the public set can be rescored offline with no keys.
Why test the same model on several providers?
The same open model can differ in speed, price and limits, and sometimes in answers, from one host to another. We measure every model on every provider that serves it, and never rank a hosted provider against a model we run ourselves on speed or cost.
Can you add my model?
If it is an extraction model (typed fields in, typed values out) and it is publicly available, yes. Adding a model is one adapter. Open an issue on the GitHub repo once it is public, or contact Leo on X.