Cohort and retention analytics with AI support

A cohort report answers a question an average cannot: is the business getting better at keeping the customers it acquires, or does each new vintage decay the way the last one did? You group users by the period they joined and track a metric as each group ages. Retention analytics is the same machinery aimed at whether people stay active.

Both live or die on point-in-time correctness, and both fall apart quietly when the join dates are wrong. The reason to bother is that blended metrics move for the wrong reasons. Overall retention can climb while every single vintage is getting worse, because you are acquiring faster and the fresh, still-loyal accounts outnumber the aging ones. Simpson’s paradox is not a curiosity here. It is the default failure mode of a monthly active-user chart, and it is why a leadership deck can show a healthy line over a product that is leaking.

Get the join date right or nothing downstream matters

The cohort key is an event timestamp, and picking the wrong event poisons every curve built on it. First app open, first funded deposit, first completed KYC, and first revenue-generating transaction are four different cohort definitions that will disagree by weeks. We settle this before writing any SQL, because a retention curve computed against “signup” and one computed against “first funded account” tell opposite stories about onboarding.

Two problems tend to surface as soon as the definition is fixed.

• Entity resolution. One human opens a personal account, then a business account, then reinstalls after a device change. If those map to three cohort members, your cohort sizes are inflated and retention is understated. Resolving them to one entity is where an AI-assisted matching step earns its keep, proposing candidate merges from device, document, and payment-instrument signals that a reviewer confirms against an eval set of known duplicates.
• Backfilled and corrected events. A KYC status that flips from pending to approved three days later, a chargeback that reverses a transaction, a deposit that settles T+2. If the cohort pipeline reads the current row instead of the row as it stood that week, early retention is silently overstated. The fix is a table with valid-from and valid-to columns and a reader that filters on the as-of date. No shortcuts.

Lookahead leakage is the specific danger. Joining an account’s present-day segment, plan tier, or lifetime value onto its week-zero row lets information from the future decide how the past looked. Every attribute in a cohort table has to be the value that was true at that week, reconstructed from an event log, not looked up live.

What the AI actually does, and what it does not

The retention curve itself is rarely the hard part. A survival model, or even a well-fit geometric decay per cohort, gives you the shape. We do not reach for a large model to draw a line that a Kaplan-Meier estimator draws correctly. The model helps at the edges of the pipeline and in the reading of the output.

• Drafting cohort and segment definitions from a plain-language brief, then translating them into versioned SQL that lands in the transformation layer where it can be reviewed and tested like any other code.
• Entity resolution and deduplication, as above, with a human confirming merges and a false-positive budget so the system never silently collapses two real customers into one.
• Narrating divergence. When the March vintage retains eight points below February at week four, the model reads the underlying feature differences between the two groups and drafts the candidate explanation: acquisition channel mix shifted, a pricing test ran, onboarding added a step. The analyst confirms or kills each hypothesis. The model proposes; it does not conclude.

The line we hold is that anything feeding a board metric or a regulated disclosure carries lineage back to source events, and the AI-generated pieces are the most heavily checked, not the least. A drafted SQL definition is reviewed before it merges. A narrated explanation is a hypothesis with the query behind it attached, never a stated fact.

Tie it to finance, then read the grid

A cohort layer that does not tie to finance is a second set of books, and finance will stop trusting the dashboard the first time the numbers disagree at quarter-end. So the cohort totals reconcile to the same account and revenue ledgers the close runs on, at the same grain, every period. If the sum of a month’s cohort does not match the booked figure, the cohort table is wrong by definition. This reconciliation is a scheduled job with an audit trail, not a spot-check someone runs when a number looks off.

Cohort tables also drift in ways that are easy to miss:

• A definition change (say, first funded deposit now requires a settled rather than a pending state) reshapes historical curves. Version the definitions and stamp each rebuilt table with the definition it used, or you will compare two vintages that were never measured the same way.
• Engagement metrics decay in meaning. “Active” defined as one login per month stops being informative once the product adds background automation that logs sessions without a human present. Re-derive engagement definitions when the product changes underneath them.

Reading the grid well is a skill in itself. A retention triangle where each row is a cohort and each column is an age lets you separate two effects that a single line conflates: how a given vintage ages as it moves right, and how successive vintages compare as you move down. When week-four retention improves down the rows, onboarding is getting better. When every row sags at the same calendar column, something hit all cohorts at once, and that points at a market or product event rather than an acquisition-quality change. The AI can flag both patterns and pull the calendar events that line up with the second one. Deciding what they mean stays with the person who owns the metric.