The Unread Pile

Volume I · Document intelligence

The Unread Pile

The document nobody has time to read — and one person is the bottleneck.

The problem

Somebody on your team reads the same kind of document over and over: supplier contracts, incoming invoices, policy files, sustainability reports. The reading is slow, it is inconsistent between people, and when a decision is questioned months later there is no record of why that document was treated the way it was.

Who it is for

Teams with a recurring queue of documents and a person whose judgement is the bottleneck. It fits best where being wrong is expensive, so the output has to be defensible rather than merely fast.

What you end up with

A bounded workflow over one document type, with an evaluation you can read and an exception path for everything the model should not decide alone.

• A pipeline that turns a document into structured fields with the source passage retained for each one.
• An evaluation on a sample your team labelled, reporting where the system agrees with your reviewers and where it does not.

What you end up with

• An exception queue: the cases routed to a person, and the rule that decides what lands there.
• A written account of the failure modes found during evaluation.

Evidence record

The greenwashing case study is the honest version of this work: it shows a detector that tracks expert judgement closely on a continuous score while still missing most of the items it should flag, and explains why that combination makes it a triage aid rather than a filter.

Process · 01

We look at real examples of the document and the decision made from it, and agree what "correct" means before anything is built.

Discovery

A written problem statement with the decision, the document type, and the definition of a correct output.

Process · 02

One document type, one workflow. Your team labels a sample; I build against it and evaluate on held-out examples.

Bounded pilot

A working pipeline, evaluation results on your own documents, and the exception-review process.

Process · 03

The pipeline is integrated where the work actually happens, with the review step in place from day one.

Delivery

Deployed workflow, runbook, and the monitoring that shows when quality drifts.

Process · 04

Your team runs it. I document how to re-evaluate it when the documents change, because they will.

Handover

Documentation, the evaluation harness, and a walkthrough with the people who will own it.

Data provenance

• A representative sample of the documents, including the awkward ones.
• Access to a person who can say what a correct output looks like.
• Time from your reviewers to label a sample — this is usually the real constraint.
• Clarity on what may leave your infrastructure and what may not.

What is in scope

• Extraction, classification, ranking and routing over a defined document type.
• Evaluation against labels your team produced.
• The exception process and the interface a reviewer works in.

Limitations

• Unattended decisions. Everything consequential keeps a human in the loop.
• Legal, tax or audit opinions about what a document means.
• A general-purpose system that handles any document you have. Pilots are scoped to one type.

Have a problem shaped like this?

Document-intelligence pilot

A bounded workflow, evaluation results on your own documents, and an exception-review process.

Will this replace the person who reads these today?

No, and I would not build it that way. It changes what they read first and gives them the reason for the ranking. On the research project behind this service, the best detector still missed most of the items a reviewer would flag — which is exactly why the review step stays.

Does our data leave our systems?

That is a decision we make in discovery, not a default. Some approaches run entirely on your infrastructure; others use a hosted model, which means the document text goes to that provider. I will tell you which one a given approach requires before we commit to it.

How accurate is it?

Unanswerable before we have your documents and your labels. Any number quoted to you in advance of that is about someone else’s data. Establishing the number for yours is what the pilot is for.

What if the evaluation shows it does not work well enough?

Then that is the pilot’s result and you have spent a bounded amount to learn it. That outcome is why the pilot is scoped small and evaluated before anything is integrated.

Approach

• Python
• spaCy
• Hugging Face Transformers
• Sentence Transformers
• PyTorch
• scikit-learn

Have a document queue that is slowing a decision down?

Tell me which document and which decision, and I will tell you whether a pilot is worth running.