Commercial Evidence

Proving double-digit commercial lift from mobile app adoption — never done in the company's history.

A high-growth mobile commerce SaaS platform could show app adoption numbers but couldn't prove whether the app created incremental value, or simply shifted orders from the web. The question had gone unanswered for years — prior attempts stalled on data quality and experiment design. Resolving both together produced CFO-safe evidence that became a core asset in sales decks, marketing, and on the company website.

AOV ↑ Double-digit incremental lift

Frequency ↑ Double-digit incremental lift

Retention ↑ Measurable improvement — now in GTM materials

The Situation

The business had built a mobile commerce app with real adoption — but had never been able to prove whether it drove net-new revenue or simply pulled orders away from the web. That distinction mattered enormously for sales, marketing, and product investment decisions.

Prior attempts to answer the question had stalled. Order data was incomplete and manually edited. Test orders were mixed with real transactions. Real-time pulls introduced gaps. And Marketing, Product, and Finance each operated from different metrics and experiment designs, so no single analysis was ever defensible enough to act on.

The result was an organization that believed in its product but couldn't prove its value — to external buyers, to internal stakeholders, or to the board.

What Leadership Needed

Four things had to be true before the analysis could be trusted:

A clean data foundation — Shopify order history backfilled, test transactions removed, experiment IDs propagating cleanly from source through to the warehouse
A defensible experiment design — matched cohorts, equal pre/post windows, consistent eligibility rules, no ad hoc comparisons
Metrics that executives could act on — AOV, order frequency, recency, and retention instead of "spend per day" vanity metrics
Results that could be reused — not a one-time analysis, but a repeatable framework teams could re-run by merchant, segment, or industry

The Work

The work started with the data, not the experiment. Without a clean foundation, no experiment design would produce results worth trusting.

Data Foundation

Shopify order history cleaned and backfilled → detail tables cross-checked against Shopify summary APIs Test orders systematically removed → using feature engineering, not manual filtering Real-time pull gaps resolved → reliable batch history established as the analysis base Experiment IDs and channel flags governed → propagated cleanly from source through to the semantic layer

Experiment Design

Matched cohorts designed → merchants split controlling for noise and seasonality Equal pre/post windows enforced → identical start and end dates across all groups Model-level guardrails coded → constraints to prevent invalid comparisons Consistent eligibility rules enforced → applied uniformly across all merchants

KPI Reframing

"Spend per day" replaced → with AOV, order frequency, recency, and retention Metrics selected for executive relevance → recognized by CFOs, CMOs, and CROs as commercial evidence Shared definitions locked across GTM and Finance → results meant the same thing to every stakeholder

Segmentation

Consumers segmented by first-touch channel → web-first vs. app-first cohorts Merchant-level views enabled → with the ability to slice by ICP and industry Results meaningful at both levels → aggregate and individual merchant

Repeatable Framework

Analysis codified into a parametric ThoughtSpot playbook → time periods, exclusions, and cohort definitions set as adjustable parameters Teams can re-run studies without rebuilding → by merchant, segment, or industry Confluence runbooks published → source-to-target mapping, metric definitions, tutorials, and Q&A

What Changed

The analysis proved what the business had believed but never been able to demonstrate: app exposure was linked to higher AOV, increased purchase frequency, and stronger retention — with double-digit incremental lift in the teens across all three.

The results became a public GTM proof point. Incrementality evidence now appears in sales decks, marketing materials, and on the company website as proof the app drives net-new revenue — not channel re-attribution.

Internally, the shift was equally significant. Marketing, Product, and Finance moved from operating on separate stories to a shared experiment design and shared definitions. Conversations that had been stuck on "is this real?" became "how much, and where?"

The parametric ThoughtSpot model continues to run. Teams re-run incrementality studies by merchant, segment, or industry without rebuilding the analysis from scratch.

Charmaine strikes a rare balance between strategic thinking and hands-on execution. The processes she created continue to serve as the foundation for how data flows and informs decisions across the company.

A.E. Senior Data Analyst, Product

Ready to start

Recognize the situation?

When the business has a hypothesis it believes in but can't prove, the problem is usually upstream — in the data, the experiment design, or the metrics being used to measure it. If your team is stuck in re-attribution debates or producing analysis that doesn't hold up under scrutiny, that's usually a solvable problem.

Book a Free Discovery Call

No commitment. 30 minutes. Focused on your situation.