A usage-based churn model reaches 0.867 AUC on this account book. An agent reading the same accounts’ support tickets takes the combination to 0.890, and surfaces €1.34M of churned ARR the usage numbers never flag.
The company is invented. Kataja Analytics sells a dashboards-and-reporting platform from Helsinki to mid-market firms, 500 accounts of 10 to 200 seats each. Its churn model runs on usage levels, trends and seat utilisation, and catches the accounts whose logins taper off before the cancellation email arrives.
It did not catch Bergvik Systems AB, a 31-seat Swedish account at €12,426 a year. Its usage never moved. What moved was the tone of four support tickets over thirteen months, each longer and more careful than the last and all closing with some version of “no need to treat this as urgent”. The account cancelled in November 2025, and nobody had time to read thirteen months of ticket history for five hundred accounts.
An agent reads each account’s tickets in order and returns a frustration trajectory, an unresolved-issue count, a short narrative and a risk score, without seeing usage data or the churn label. A logistic regression combines that read with the usage model’s score. The book is 500 accounts over 24 months, 3,777 tickets and 133 churners, scored out-of-fold against an answer key no model could see.
The usage model has ticket counts and resolution times as metadata but no ticket content, and reaches AUC 0.867. The text layer alone reaches 0.795, the two together 0.890. In a top-100 review list, a fifth of the book, usage and text each catch 59.4% of the real churners and the combination catches 66.2%, nine more of them for the same reviewer hours. €1.34M of churned ARR sits in the text layer’s list and not in the usage model’s, and €703K runs the other way.
The harder problem was the data. Making the quietly unhappy accounts invisible to metadata took four iterations, and three times the model found a path back in, through unresolved-ticket counts when satisfaction scores were removed, then through ticket volume itself when outcomes were fixed, since these accounts filed 6.65 tickets in a trailing six months against 1.04 for a healthy one. The volume that made an account worth reading was the same volume the metadata could see. The archetype went quiet only when it was cut to three to six long, courteous tickets across a whole account life, and the usage model’s mean score on it fell from 0.679 to 0.477.
Both layers have a characteristic miss. The text agent over-reads accounts that complain loudly and renew anyway; 20 of the 55 here reach its own top 100 on tone, fusion drops 19 of them, and the one that stays has unusual ticket volume. Thirty accounts churn for reasons that never touch the product, and two reach the combined top 100, about what chance returns on a fifth of the book, since no signal was planted there to find.
The explorer below holds the account book, with both layers’ scores for every account, the archetype breakdowns behind the figures above, and the ticket history each agent read.
The accounts, the usage history and all 3,777 tickets are synthetic. Deterministic Python fixed every structural signal and an agent wrote only the prose. The answer key stayed out of the feature build, the models and the fusion, and one evaluation script marked the results against it.
The structure transfers to a real account book. Use a classical model where the numbers are strong, a reading layer where they are silent, one small explainable model over the two, and count the misses per archetype. The figures do not transfer. They belong to this dataset and to the mix of account types we chose for it. A real account book offers historical outcomes in place of a planted key, noisier and later, and what mix it holds only shows on reading.
Configuration, generator, validator, feature build, both models, the fusion fit and the evaluation run in that order, on one seed throughout. Known data artefacts are documented in the code. Tenure in months is excluded from the trained model, because a fixed observation window turns it into a near-perfect churn proxy and including it lifts AUC to 0.955 for the wrong reason.
The code, the dataset, the notebook and the long-form writeup are in the public repository. The superseded datasets from the earlier attempts at the quietly unhappy archetype are not in it. They were kept rather than deleted; write to us if you want to read the trail in full.
If you sell on subscription and keep a written trail against named accounts, in tickets, CRM notes or account emails, tell us how many accounts you have and how far back the writing goes.