Hapax · Case study

Sixteen tenders, answered two ways.

Sixteen invented tenders and 510 requirements, answered two ways. Fourteen simulated people with a chatbot produced per-tender medians from 64.3% to 93.3%. A workflow run once per tender found 507 of 510 requirements, 99.4%, answered every requirement on 11 of 14 and refused both tenders the firm cannot win. The three it missed are mandatory forms buried in annexes that the human arm also missed.

Film · 1:54 Transcript
Transcript

At Visakoivu in Jyväskylä, Antti writes the tender responses, and the current batch has had him in the annexes since Monday. In the enclosure clause of one draft contract, between two forms that are properly required in the main document, a third sits with no clause number of its own. That tender is one of sixteen on the desk, about five hundred requirements between them. Some buried disqualifying clauses in the annexes, or mentioned mandatory forms only in passing. Two are tenders the firm genuinely cannot win.

We tested two ways of answering the same tenders. In the first, fourteen simulated people worked with a general chatbot, each with their own habits and level of attention. Across all sixteen tenders they produced sixty-four attempts, and twenty-two of those carried a fabricated claim, meaning the chatbot invented a qualification or a commitment the firm cannot back up. Their scores on individual tenders ranged from about sixty-four to about ninety-three per cent of requirements answered. In the second, one structured workflow ran once per tender. It extracted every requirement from the document and drafted each answer with a citation to the firm’s verified facts. Three automated checks ran on every draft before it could go out.

The workflow declined both tenders the firm cannot win. On the fourteen it bid, it answered every requirement on eleven and at or above ninety-five per cent on thirteen. No fabricated claim survived in any of the sixteen runs, against twenty-two in sixty-four individual attempts. The extraction step found five hundred and seven of the five hundred and ten requirements. The three it missed were mandatory forms mentioned once inside an annex, and the human attempts scored zero of twelve on the same class of requirement. The tenders are invented, and the simulated people are a model of how attention affects the result, not observed bidders.

If you’d like to see what we could solve for you, write to us at hapax.fi.

Read the case