Consistency
Does it give the same answer every time it should?
Same document twice: the answers should not change.
Text and image of the same document: the answers should agree.
Renamed or reordered fields with the same meaning: the answers should not change.
Answers that change on a rerun same document, same request
Clairic
Cloudflare
NuExtract3
NuMind
GLiNER2
Fastino
Share of fields whose answer changes when the same request is sent again.
Text and image agree same document as text and as an image
Clairic
Cloudflare
NuExtract3
NuMind
GLiNER2
Fastino
Share of fields where the answer from the text version matches the answer from the image version.
Answers that change when fields are renamed or reordered same meaning, different field names or order
Clairic
Cloudflare
NuExtract3
NuMind
GLiNER2
Fastino
Share of answers that change when the field list is shuffled or the fields are renamed to synonyms.
Test items and precision
- Items
- 120 documents run three times; about 400 documents as text and as images; 120 documents with renamed and reordered fields
- Precision
- About ±2 to ±3 points.