Data curation
Data curation turns raw, inconsistent records into structured, reviewed and traceable data that AI systems can rely on, with every automated decision recorded and every doubtful one sent to a person.
We collect, clean, structure and quality-control your data, with people reviewing the uncertain cases, so it is ready for AI.
- CSV
- Excel
- JSON
- JSON Lines
- Python
- Rule-based filters
- Fuzzy matching
- Parallel batch processing
- Relational database
- Flat file export
- Containers
The challenge
Most enterprise data was never meant for AI. The same product, supplier or customer appears under a dozen spellings. Files arrive in different formats, with missing fields, duplicates and system noise mixed in with real records.
Teams clean it by hand, in spreadsheets, and the rules live in a few heads. When an AI system is fed this data, it inherits every inconsistency, and nobody can say which record was changed, by what, or why.
What it does
Collection and ingestion
We bring data in from files, folders and databases in common formats, and keep each batch separate so every record can be traced back to where it came from.
Cleaning and noise removal
Known junk, system artefacts and duplicates are filtered out by rules the customer can inspect, and each removed record is kept with the reason it was excluded.
Structuring and normalisation
Free-form, inconsistent entries are matched to a controlled master reference, so the same item is always described the same way, with consistent attributes.
Enrichment
Where a record cannot be resolved by rules, a model proposes the structured fields. Optional lookups against outside sources can fill gaps, only where the customer enables them.
Labelling and review queues
Uncertain records are routed to a reviewer queue with the proposed answer and its reasoning, so people confirm or correct instead of starting from scratch.
Quality control
Each run produces a report showing what was matched by rule, what was resolved by a model, what needs review and what was removed, so quality is visible.
How it works
Profile the data
We study your sources, formats and reference lists, and agree the target structure, the master reference and the rules for what counts as noise.
Run deterministic passes
Filtering and rule-based matching run first. Exact and close matches are accepted automatically, which keeps model usage low and results repeatable.
Resolve and review
Remaining records go to a model with a confidence check. Low-confidence answers go to people, whose corrections are fed back to improve later runs.
Deliver and monitor
Curated data is delivered as review reports and flat files for downstream systems, with an audit trail of every decision, and reruns handle new batches.
Under the hood
- Data formats
- CSV
- Excel
- JSON
- JSON Lines
- Matching and processing
- Python
- Rule-based filters
- Fuzzy matching
- Parallel batch processing
- Storage and delivery
- Relational database
- Flat file export
- Containers
- Model layer
- Pluggable model providers
- Result caching
- Optional web search enrichment
Business outcome
One consistent version of your data. The same item is described the same way everywhere, so reports, search and AI systems work from a shared and reliable reference.
People focus on the hard cases. Rules and models handle the routine records, and reviewers spend their time on the small share that genuinely needs judgement.
Every change can be explained. Each curated value links back to its source, the rule or model that produced it, and the person who approved it.
Faster path to AI adoption. Data that is structured, reviewed and documented removes a common blocker, so AI projects start from a trusted base.
Where it is used
- Normalising raw product or asset inventories against a master catalogue before analysis.
- Preparing document-derived records for AI by structuring fields and routing doubtful ones to review.
- Cleaning supplier, customer or item lists with duplicates and inconsistent spellings.
- Building labelled, reviewed datasets to evaluate or fine-tune a model for a specific job.
Frequently asked questions
Does the service require sending our data to a model?
No. Deterministic rules run first, and a model is used only for records they cannot resolve. You can run with no model at all, or choose the provider that fits your policy.
Who decides what counts as correct?
You do. We agree the master reference and rules with you at the start, and a reviewer queue lets your people confirm or correct anything the system is unsure about.
Can we see why a record was changed or removed?
Yes. Removed records are kept with a reason, and model prompts and responses are logged, so each curated value has a traceable explanation.
Is this a product or a project?
It is a service. We set up and run the curation pipeline with you, tune it to your data, and can hand over the reports and rules for your team to maintain.
See it on your own documents and data
Tell us what you want to automate and we will show you how it runs inside the harness.