Blog · Crash & Retention Audit
Why crash-free sessions mislead release decisions
Crash-free session rate is a comforting number. It is also one of the easiest ways to green-light a bad candidate. In Analytics QA for App Releases, a session that never throws can still strand a person on a pay sheet that never returns, a WebView that paints white, or a date picker that cannot submit a Buddhist calendar value.
We see this most on mid-range Android in Thailand, where memory pressure does not always crash the process. The OS keeps the app “alive.” The user leaves. Your crash vendor records a success. Your funnel records an abandonment two steps later, if you even instrumented the freeze.
Pair the vendor with a task event
In Crash & Retention Audit we insist on one usable-session definition per release: a foreground session that reaches a named task event within a timeout you publish. For a wallet that might be transfer_receipt_shown or bill_paid. For travel, trip_saved. If crash-free climbs and usable-session falls, the gate is not green.
Timeouts matter. A 30-second window punishes slow rural networks. A 10-minute window hides a frozen sheet. We usually start at two minutes for checkout-like tasks and adjust with the QA lead, not with a vendor default.
Split by build, not by week
Blended crash-free across production plus candidate is how regressions hide. The candidate may be 4% of sessions. A blended chart will look like last Tuesday. Version filters are not optional decoration; they are the QA surface.
If your crash tool cannot filter by the same build number your analytics SDK emits, stop and fix the stamp. Teaching a prettier dashboard over mismatched identifiers is theatre.
What we write on the gate memo
A typical memo line: “Crash-free 99.1% on 4.18.2 (n=2,140). Usable checkout session 71% versus 84% on 4.17.x, concentrated on Android API 34. Hold.” That sentence has survived more ship meetings than a slide titled ‘stability looks good.’
If you want the full pairing exercise, it lives inside the Release Signal Lab crash module and the shorter clinic on the courses list.