The ERP case retires a language model from a parsing job by writing ordinary code for the repeating formats. This case runs the same method on PDF invoices, one supplier at a time. The ceiling here is the supplier base: eight of 26 fictional suppliers send enough for the system to learn their layout. The parsers broke even in year one; from year two the stream runs at about a seventh of the all-model cost.
The company is invented. Pyökkipaja Oy is a Lahti contract-furniture manufacturer of about 120 employees. Its large domestic suppliers invoice through structured e-invoicing, straight into the ledger, roughly how 85% of Finnish B2B invoicing works. The rest arrives in the accounts-payable inbox as a PDF and is keyed by hand: a German fittings supplier, a Swedish fabric house, an Estonian steel shop, five smaller domestic firms, and eighteen one-off suppliers who invoice once or twice a year. That residue is a minority of the invoices and most of the manual effort.
We generated a year of that inbox: 198 invoices from 26 fictional suppliers, as PDFs in four languages, with the formatting, tax arithmetic and payment references of real B2B invoicing. An answer key records the correct value of every field, kept away from every system in the pipeline.
Eight of the 26 suppliers send frequently enough for the system to learn their layout. The other eighteen send too rarely, and automating a one-off is not worth the cost. One of the eight changes its invoicing software in August, so its layout changes with it.
Four invoices are traps designed to test the parsers: a fuel-surcharge line, a Swedish invoice with a terms attachment, and two credit notes with negative amounts. A fifth carries a silent error: it prints 50 units at €2.00 where the true order was 2 units at €50.00. The line total is €100.00 either way, so every automated check passes. Only the answer key reveals the swap.
A language model read each PDF as an image and returned a structured record. A second system watched those parses supplier by supplier. After six invoices in a consistent layout it wrote an ordinary Python parser for that supplier, which went live only once it reproduced the model’s accepted output on the invoices it had learnt from. When a parser did not recognise a document’s layout, it refused the document and the model read it instead. Nine parsers covered eight suppliers. The ninth was learnt in October, after one supplier changed its invoicing software in August and the old parser refused the new layout.
117 of the 198 invoices, 59.1%, ended up on the deterministic path. The model read 81 documents over the year, 5,427 fields, with no extraction errors. Across the full run, 13,648 of 13,650 fields were correct. The two exceptions are both from the silent-error invoice, where the quantity and unit price are swapped at the source. In production, a three-way match against purchase order and goods receipt catches that.
One of the four traps parsed correctly. The generated parser derives its columns from the document instead of assuming a fixed layout, so the fuel-surcharge line on the freight invoice was treated as ordinary data. The other three fell back to the model as designed. Four two-page invoices also fell back unplanned, because the parsers for those suppliers had only seen single-page examples.
Costs are estimates: the figures come from published prices applied to measured tokens. Year one costs about $9.05 against $9.61 for reading everything with the model. The nine parsers cost roughly what they saved in their first year. Year two needs the model for 27 documents and costs about $1.31 against the same $9.61, repeating thereafter.
The explorer below holds the year: an invoice as accounts payable receives it, the stream month by month, the parsers the system wrote, and every field checked against the answer key.
Everything on this page is invented. We generated all 198 invoices and hold a field-level answer key that the parsing pipeline never reads. Marking turned up four defects in the key and none in the parsing. Expect the same of whatever you mark a real inbox against.
The method transfers to a real invoice inbox. Let a model parse the invoices, have a second system learn each supplier’s layout and write code for it, test the code before it goes live, fall back to the model on anything new. The contracts case runs the same method on real contracts, with someone else’s expert annotations as the answer key.
The numbers will not be the same. How large the deterministic share is depends on your supplier base more than on parser quality: the ceiling here was set by how many suppliers send enough invoices for the system to learn their layout. Counting that in your own ledger is where an engagement starts.
The data is generated by a seeded Python script: the same seed always produces the same 198 invoices and answer keys, and at the pinned Chrome version the PDFs are byte-identical across runs. A validator checks coherence rules before anything runs. The pipeline then follows in order: model path, code-writing system, router, evaluation. Each of the nine parsers carries a provenance header naming the six invoices it learnt from. The accuracy and field-count figures come from the marking script; the year-two cost is extrapolated from the stream composition in the report.
The public repository holds the code, the corpus, the answer key, the nine parsers and the full writeup.
If invoices still reach your ledger as PDFs, tell us how many arrive in a month and how many suppliers they come from.