Messy-input robustness
How much worse is it on bad photos and scans?
We take documents the model reads well and blur, rotate, darken, crumple and scan them, then report the accuracy lost on the same documents.
Messy-input drop accuracy points lost on messy versions
Clairic
Cloudflare
NuExtract3
NuMind
GLiNER2
Fastino
Accuracy on clean documents minus accuracy on blurred, rotated, dark, crumpled and scanned versions of the same documents.
Test items and precision
- Items
- 120 documents in five degraded versions, 120 with noisy OCR text, 60 handwritten-style forms
- Precision
- About ±2 points pooled, about ±4 per kind of damage. Handwriting uses handwriting fonts, not real handwriting.