Document classification at production scale
Classifying a few document types is easy. Keeping thousands of types accurate across many projects, month after month, needs a review loop, regression tests and a rule library that learns from corrections.
The challenge
Large organisations classify documents for many projects at once, across thousands of types and varied formats. Manual sorting is slow and inconsistent between reviewers, and a classifier that worked at launch quietly degrades as new layouts, languages and document types appear.
Scale adds its own risks. A rule fixed for one project can break another, privacy rules may stop documents leaving the environment, and nobody can tell whether accuracy is holding without a repeatable way to measure it against known examples.
What it does
Rules with model assistance
A rule engine labels most pages with a category and confidence. A model of your choice helps with uncertain pages and with drafting rules, so cost and latency stay predictable.
Reviewer feedback loop
Reviewer decisions on uncertain documents are turned into candidate rules. Approved candidates join the rule library, so the same mistake is not corrected twice.
Cross-project regression testing
Rule changes are run against saved reference sets from many projects before release, showing which documents would change label so a fix in one place does not break another.
Misclassification analysis
Wrong and disputed labels are grouped by type and by the rule that fired, so teams find the weak spots in the library instead of reading individual cases.
Controlled rule management
A rule management interface handles creating, editing and approving rules, with reviewer validation, so changes to the library are deliberate and attributable.
Local model option
The model tier can run in your own environment, and the pipeline can run rules-only, so privacy-sensitive documents do not need to leave your control.
How it works
Classify and score
Documents are read, then rules assign a type and a confidence to each page. A model verifies only uncertain pages, and the evidence behind every label is kept.
Review the uncertain
Low-confidence results go to a review queue. Reviewers confirm or correct the label, and each correction is stored with the document it came from.
Learn into the library
Corrections become candidate rules. A model can help draft them, a reviewer validates them, and approved rules are added to the shared rule library.
Regression test and release
Before a rule change goes live it is tested across datasets from different projects. Label changes are reviewed, and drift is tracked between releases.
Under the hood
- Classification
- Python
- Rule engine
- Category and confidence scoring
- YAML catalogs
- Document reading
- OCR engine
- Language detection
- Document segmentation
- Model tier
- Provider-agnostic client
- Hosted or local models
- Retrieval over classified documents
- Operations
- Rule management interface
- Reference datasets
- Regression reports
- Docker
Inside the harness
Reliability
Regression testing across projects and a review queue for uncertain items keep classification dependable as volume, types and formats grow.
Evaluation
Reference datasets and misclassification analysis measure the rule library against known answers, so drift is seen before it reaches production.
Guardrails
Uncertain labels are never accepted silently. Low-confidence documents go to review, and rule changes need validation before they join the library.
Model routing
The model tier is optional and provider-neutral, and can be a local model or switched off, depending on what each project privacy rules allow.
Business outcome
Accuracy that holds over time. Regression tests and drift checks catch degradation early, so quality is maintained as new document types and layouts arrive rather than discovered by downstream errors.
Less manual classification. Rules and model assistance handle the confident majority, and people spend time only on the documents that need judgment.
A library that improves. Every reviewer correction can become a rule, so effort spent on review reduces future review instead of disappearing.
Safe changes across projects. Testing a rule against every project reference set means a fix for one team does not quietly damage another team's results.
Where it is used
- Document-heavy shared-service teams classifying work for many programmes from one governed rule library.
- HR and onboarding operations sorting large volumes of identity, education and employment documents across projects.
- Financial services back offices classifying application packs and statements across thousands of document types.
- Privacy-sensitive environments that classify with a local model and keep documents inside their own infrastructure.
Frequently asked questions
How is this different from the Document classification product?
That page covers how a batch is classified, split and routed. This one covers running it reliably at scale: the review loop, regression testing across datasets, and drift monitoring over time.
What is drift and how is it caught?
Drift is accuracy slipping as documents change. Saved reference sets are re-run against the current rule library, and differences in labels are reported before a release goes live.
Does a person approve new rules?
Yes. Reviewer decisions produce candidate rules, and a reviewer validates them in the rule management interface before they enter the library. A model can assist drafting, but it does not release rules alone.
Can it run without sending documents to a hosted model?
Yes. The model tier can be a local model, and the pipeline can run on rules alone. Which setup is used is decided per project against its privacy requirements.
See it on your own documents and data
Tell us what you want to automate and we will show you how it runs inside the harness.