We built a 50,000-row purchase ledger, hid 87 irregularities in it, and screened it three ways. Rule-based checks caught 48. A statistical anomaly detector caught none. Three independent AI analyses triaged the flags, but the slow vendor price drift at the centre of the design was missed by every layer.
The company is invented. Saarnitukku Oy is a Vantaa wholesaler of fasteners, seals and maintenance supplies, turning over about €90 million a year. Its purchase ledger runs to roughly 50,000 rows. An e-invoice integration posts four in five; six people in accounts payable post the rest.
Nobody reads a ledger that size end to end. Automated checks will raise flags quickly enough. What is scarce is the time of someone qualified to decide whether a flag means a duplicate submission, a coding error, a habit of posting just below an approval threshold, or nothing at all. The subject is controls and process quality, not accusation.
We generated the ledger from a seeded Python model and hid 87 irregularities in it across eight categories, alongside 18 transactions designed to look suspicious but be entirely clean. An answer key records where each one sits, kept away from every screening layer.
Most of the hidden issues are things a standard audit check should catch: 12 duplicate invoice submissions (same vendor, same amount, days apart, different invoice numbers), 10 invoices at round-thousand amounts or just below the €10,000 approval threshold, 8 postings by one user on non-working days, 10 purchases split across two invoices to stay individually under the threshold, and 8 advertising invoices mis-posted to the IT-services account by a new clerk. Fifteen belong to a German seals supplier, Kärrenbach Dichtungstechnik GmbH, that exists as three separate master records with three different spellings and three different bank accounts. That is a cross-record problem that no single-transaction check can find.
The category the case is built around is a slow price drift. From July, one steel supplier, Teräskontio Oy, folds freight into the price of the goods. The memos note it in Finnish. A further rise compounds inside the bundled line, and by December invoices are about 19% above the first-half baseline. The freight carrier’s billings for that supplier’s deliveries decline by roughly the freight portion of the rise, not the compounding part. Both sides of the story are in the data. The drift starts at roughly €1,800 a month and grows as the compounding builds.
The case also includes twelve consulting invoices that cannot be caught from this data alone. They carry the correct account code, ordinary amounts, and were posted on weekdays. There is nothing in this ledger to flag them. Catching them would need a purchase order or a goods receipt, which this extract does not carry. An analysis that accuses them without evidence fails the test.
Among the 18 designed false alarms: twelve months of a landlord’s rent at a contractual €15,000, an inventory-count Saturday with a supervisor’s overtime, a coincidental equal-amount pair with distinct purchase orders, a one-off machine purchase at €78,400, and an annual insurance premium. Each has a clear explanation in the data.
Three screening approaches went through the data. Seven automated checks tested for known patterns: duplicates, round amounts, threshold splits, weekend postings, mis-posted accounts, and two others drawn from published audit practice. A statistical anomaly detector scored each transaction against the rest of the ledger and flagged the 200 most unusual. Three independent AI analyses, each blind to the others and to the answer key, then assessed the combined flags.
The rule tests raised 64 flags, 48 on genuine issues. Within each category the rules targeted, every instance was caught. Most of the other 16 flags are twelve months of the landlord’s rent at a contractual €15,000.
The statistical detector raised 200 flags. Not one was a genuine issue. It missed the drift because it measures each invoice against the vendor’s own full-year average, and a slow, steady rise lifts the average along with everything else. A 19% increase across the year reads as 2.1 standard deviations against the inflated baseline; against a clean one it would show 6.5.
The 264 flags from both layers were sorted, mostly by vendor, into 39 groups. Each group was then put to the three AI analyses, which had to call it either a genuine problem or a false alarm, with evidence that could be recomputed from the ledger, and they got 37, 35 and 38 of the 39 right. All three found the Kärrenbach name-variant problem: one supplier as three master records, same VAT number and Hamburg address, three different spellings, three different bank accounts.
Nobody found the drift. The rules were never built for it. The detector’s best-ranked affected invoice sat 818th of 50,000. The three AI analyses assessed the flagged groups, and the drift was not among them. In a real ledger, with no answer key, a drift this slow on a single vendor would likely go unnoticed.
The explorer below steps through the results for all three screening layers and all 39 assessed groups, showing what each layer flagged, what the answer key says and the evidence behind each assessment.
Everything on this page is invented. We generated the ledger, hid the irregularities, and hold the answer key. The figures on this page come from checking against it.
The method transfers. A real ledger has no answer key, so you test each screening layer against something you already trust: a reconciled control total, a known incident, or a manual sample. This extract deliberately carries no purchase orders or goods receipts, so the three-way match that would catch most routine problems is absent. A real engagement adds it.
The numbers will not be the same. A real ledger has its own anomalies in its own proportions, and how quickly a drift shows up depends on the vendor, the amounts and the baseline. The data and privacy page covers how the case studies handle data.
A standing vendor price review was then built specifically for the kind of drift the initial screening missed. For each vendor and each month, it takes a volume-weighted average unit price, compares it against a baseline built from that vendor’s own earlier months, and accumulates the deviations until they cross a threshold. The parameters were written down before the second year’s data existed: at least three invoices per month before a month counts, at least three months of history before a comparison starts, and the threshold itself. Six specific predictions about what the report should find were also written down in advance.
It ran on a second fabricated year of the same company’s ledger, 50,000 fresh rows and 200 vendors carrying three planted price drifts and two benign price rises, and flagged all five. Four sit in the top five of the 195 scoreable vendors: the August repricing first, the fast drift second, the bundled-freight drift third and the slow 1%-a-month drift fifth. The January indexation is further down at 34. The slow drift crossed the threshold in July. The two benign rises look exactly like the three real ones on the trace. They separate only on the memos behind them.
Two of the six predictions were wrong. The first expected about five vendors with nothing planted to cross the threshold, and 42 did. A baseline of only three months is volatile enough to make an ordinary month look like a deviation. The ranking survived what the count did not: the three planted drifts sit at ranks 2, 3 and 5, so in practice the instruction is to read from the top, where the genuine items cluster.
The second miss matters more. Applied to the first year’s ledger, the report cannot see Teräskontio at all, the vendor whose drift prompted it. Teräskontio bills roughly twice a month. The method needs three invoices in a month before that month counts, and Teräskontio clears that floor in none of the twelve. It has no rank and no trajectory anywhere in the output. Drift is caught wherever there is volume. A vendor that bills too rarely to build a baseline needs a different approach.
The data is generated by a seeded Python script: the same seed always produces the same ledger. A validator checks thirteen coherence rules before anything else runs. The rule tests, the statistical detector, and the grouping of flags into assessed items all run deterministically. The three AI analyses are session artefacts and would produce different results on a re-run; the variation (37, 35, 38 of 39) is part of the result. The scorecard was adjudicated by hand.
The code, the ledger, the three analyses and the full writeup are in the public repository, together with the drift report, its generator, its second ledger and its marked results.
If your purchase ledger is screened by rules that have never been tested against known answers, tell us how many flags a month they raise and how many people read them.