A keyword filter rewards the applicants who use the role’s vocabulary, whether or not the experience is behind it. On 240 fabricated applications for one signal-processing role, we ran four screens, from that filter to a workflow that returns cited evidence against each of 12 requirements and no ranking, and marked all four against an answer key none of them could see.
A hiring team receives 240 applications for a senior signal-processing engineer at Kaikuvaara Oy, a fictional remote-sensing instruments maker in Oulu. Each carries a CV and a substantive cover letter. Reading them against 12 requirements takes more time than the team has, and a keyword-matching system ranks the wrong people first.
The workflow reads every application against the requirement profile and returns an evidence dossier per candidate. For each requirement it records what the application shows with a verbatim citation, what it claims without substantiation, what contradicts itself and what should be asked in a screening call. It produces no ranking and no verdict. The recruiter decides.
We generated 240 applications for one role, with a known answer key for every qualification claim. The applicants include 18 hidden gems, candidates who meet the requirements but each with one must-have worded in unfamiliar terms. C213, the clearest example, evidences all five must-haves and ranked 144th by keyword matching. One of the five, production Python, appears only in the cover letter as automation and analysis scripts for a particle-physics detector collaboration, and the word ‘Python’ is nowhere in the application.
At the other end, 22 applicants are keyword stuffers whose applications use the right words with little substance behind them. C228 sits at keyword rank 1. Fifteen applicants carry a planted credential inflation, eight where a degree conferral date is contradicted by a job that required the degree already finished, and seven where a claimed leadership title is contradicted by the plain title and short tenure in the same role. The remaining applicants are a mix of genuinely qualified, underqualified and borderline candidates.
Four screening arms ran on the same 240 applications. One was a keyword matcher, the kind of filter an applicant-tracking system runs. Another sent each application to a language model as a single prompt, with no requirement to cite anything. The other two ran the full extraction workflow, on a smaller model and on a larger one, to find out whether the discipline comes from the workflow or from the model.
The full workflow extracted 2,116 evidence lines with verbatim citations across the 240 applications. The answer key records 639 places where an application genuinely meets one of the five must-have requirements, and the dossiers found 618 of them. Of the 19 citations that failed verification, 17 were minor transcription differences and 2 were fabricated.
The keyword matcher ranked all 22 stuffers above the pool median. The workflow flagged claims as unsubstantiated on 19 of them. It caught 14 of 15 credential inflations; the smaller-model version caught 1 and the single-prompt version caught none. 12 of 18 hidden gems had their displaced skill surfaced as evidence against the right requirement. The misses concentrated in the cover letter. Five of the nine gems whose displaced skill appeared only there were missed, against one of the nine whose evidence was elsewhere.
Ten predictions were set before marking began. Four were met, including the evidence recall at 96.71% against a 95% bar. Six were not. Nothing was retuned after the numbers appeared.
We wrote twelve recruiter personas, each played by a language model and set by how strictly it judges thin evidence, how far it leans on keywords and how evenly it spreads a fixed reading time. Each persona screened the same 48-application subset twice, unaided and with the dossier beside the application, at 240 seconds per application both times. The subset holds four hidden gems, which gives the twelve personas 48 gem readings per arm. Unaided, they advanced the gem on 31 of them; with the dossier, on 41. The larger effects were elsewhere. The dossier raised the share of credential inflations spotted from 56% to 94%, and the share of stuffers questioned from 75% to 92%. Gems parked on hold halved, 15 to 7, which is where most of the extra advances came from.
The subset also holds two candidates built to be defensible either way. The personas marked them as borderline on 33% of unaided readings and 8% of assisted ones. The dossier turned a genuinely open call into a firm decision, the one place in the measurements where it made the personas less careful, not more. One persona overrode a false citation flag and confirmed a real fabrication, correcting the tool in both directions. Another read every flag on the title-inflators and advanced them regardless. The pattern was not uniform. The two least strict personas went from questioning none of the four stuffers to questioning all four, while the most keyword-reliant persona questioned none in either arm and the strictest was unchanged on gems. One persona advanced fewer gems with the dossier than without.
The explorer below holds all 240 applications, the workflow’s evidence dossier for each one and the answer key behind a toggle. The matched pair (C213 against C228) gets a full four-arm comparison across all 12 requirements. The coverage table carries per-candidate evidence counts in application order, sortable by any column. The screening game presents five candidates at ninety seconds each, then reveals the dossiers and the key.
Everything on this page is invented. The company, the applicants, their CVs and cover letters were generated with a known answer key for every qualification claim. A marking script checked the results against it. The work page describes how the testing works across the case studies.
The workflow transfers to a real hiring round: extraction against a requirement profile, verbatim citations, a rule against guessing when evidence is thin, consistency checks, and an aggregate table the recruiter sorts. Nothing in it knows about remote sensing or Finnish cover letters. In production, applications are personal data under the GDPR, and the AI Act classifies recruitment as high-risk, with those obligations applying from December 2027. The workflow’s assistive shape, evidence and citations for a person who decides, fits that regime. The natural deployment is a local model on the organisation’s own hardware.
The numbers will not be the same. They belong to this corpus, its mix of archetypes and one seed. The process discipline transferred to the smaller model in this run; verbatim citation quality did not. Fine-tuning a local model on the client’s own historical screening material is the genuine next step.
The corpus, all four arms' outputs, the marking script, the adjudication and the answer key are in the public repository, together with the long-form writeup this page distils.
If your hiring team receives more applications than it can read carefully, tell us the role, how many arrive per round and in what form.