Hapax · Case study
The full parsing run from the case study above, supplier by supplier. Watch the deterministic share grow and inspect any invoice’s parse against the answer key.
The left one arrives thirty-two times a year, often enough to teach a parser. The right one arrives once, in a layout nobody will see again. Automating it would cost more than it saves. A real accounts-payable stream is mostly heads with a long tail, and the design has to respect both.
Each cell is one document, in stream order. Wine is a model read, indigo is a generated parser, and gold marks the documents that fell back after the August redesign or a planted failure. The curve underneath is the running cost of the hybrid pipeline against leaving the model to read everything, both estimates from measured tokens at published list prices.
Try this – press play and watch the wine cells give way to indigo as each supplier crosses its sixth invoice; scrub to August to catch Työkalu-Tiira’s redesign turning its column gold for six documents, then hover any cell to read that document’s own cumulative cost against the all-model line.
Työkalu-Tiira installed new invoicing software. Its parser failed loudly on the first new-layout invoice (by design, a parser that meets a layout it has not seen refuses) and the documents fell back to the model. Six accepted reads later the overseer had written a second parser, under the same rule that produced the first. November and December parsed deterministically again.
Try this – open the generated parser and read the first lines: it raises an error the moment a line does not match the layout it learned, which is exactly why the six new-layout invoices fell back to the model instead of being silently misread.
parser_s8v2.py – machine-written, regression-gated against the pipeline’s own accepted reads, never shown the answer key.
Eight suppliers sent enough invoices to teach a parser; they carry ninety per cent of the volume. The other eighteen, one or two invoices each, stay with the model. The ceiling on automation is set by the shape of the supplier base.
| Path | Documents | Fields | Wrong values | Accuracy |
|---|
The two wrong values share one explanation. One November invoice was built to lie at the source: quantity and unit price transposed, the line total identical either way.
Printed: 50 kpl à 2,00 €. Ordered: 2 kpl à 50,00 €. Every automated check passes; the parser reads faithfully; only the answer key (in production, a three-way match against the purchase order) sees it.
Every document was assigned an expected outcome before any parsing ran. The run is scored against that plan, deviations included.
| Class | Designed | Actual | Verdict |
|---|
Your accounts-payable inbox already has this shape: a handful of suppliers sending the same layout every month, and a long tail who don’t. An agent parses everything at first while an overseer watches which suppliers repeat and writes an ordinary parser for each one it comes to know. The model’s path shrinks to the one-offs and the odd fallback. Year one roughly breaks even, because the parsers have to earn out their own construction cost; the saving compounds from year two, once they are written and only the tail is left. The invoices, the answer key and the parsers are published in full. Your numbers will depend on your supplier base.
Read the long-form case study · Code and dataset · Contact Hapax