On 500 fabricated accounts with 3,777 support tickets and a known answer key, a usage model catches the accounts whose logins drop before they cancel. A language model reads the tickets, finds nine more genuine churners in the top hundred, and adds €1.14M in annual revenue the usage data misses.
At Kataja Analytics in Helsinki, Mika in customer success is working through his monthly review of five hundred accounts, sorted by usage.
Hedbrunn Systems is in the quiet middle of the list, and its fourth support ticket in nine months has just closed with 'no need to treat this as urgent'. That list comes from a model built to predict which accounts are about to cancel. It watches how much each account uses the product. When logins drop, the model flags the account. It did not catch Hedbrunn Systems, a thirty-one-seat Swedish account at about twelve and a half thousand euros a year. Hedbrunn’s usage never moved. What changed was the tone of four support tickets over nine months. The last one closed with 'no need to treat this as urgent'.
Nobody had time to read nine months of ticket history for five hundred accounts. A language model reads each account’s tickets in chronological order, blind to usage data and to who actually left, and scores the state of the relationship. A small model then combines that ticket reading with the usage score. In the top-one-hundred review list, the combination catches about sixty-six per cent of the genuine cancellers, up from about fifty-nine per cent on usage alone. That is nine more genuine cancellers for the same amount of work. Cancellers that only the ticket reading puts in the top hundred add up to just over a million euros in annual revenue. The ones that only the usage model catches add up to about seven hundred thousand. Thirty accounts cancelled for reasons outside the product entirely, like an acquisition, a budget cut, or a key person leaving. Nothing in the data predicted them. The ticket reading also flags accounts that complain loudly but renew anyway, and the combined model corrects most of those. Everything here is invented. The five hundred accounts and nearly four thousand tickets were generated to a design with a known answer. What transfers is the approach. Reading the tickets adds signal that usage data alone misses.
If you’d like to see what we could solve for you, write to us at hapax.fi.