Ratio
Overview
A Ratio metric answers the question: how much of one thing happens for every one of another thing?
You build it from two parts, a numerator and a denominator, each configured like a metric of its own. For each variant, ABsmartly adds up the numerator across all participants, adds up the denominator across all participants, and divides the first total by the second.
This makes it a good fit for rates that aren't "per user", such as clicks per page view, revenue per order or purchases per active day.
How it's calculated
Say you track page_view and click goals, and create a ratio with a Goal Count of click as the numerator and a Goal Count of page_view as the denominator.
context.track("page_view");
context.track("click");
Four users are in the same variant:
| User | page_view events | click events |
|---|---|---|
| Ana | 4 | 2 |
| Ben | 1 | 1 |
| Cara | 5 | 0 |
| Dan | 0 | 1 |
The totals are 4 clicks and 10 page views, so the ratio is 4 / 10 = 0.4 clicks per page view. In the experiment results, both Value and Mean show 0.4. Hovering over Value shows the totals (Numerator 4, Denominator 10). Hovering over Mean shows the averages per participant (Numerator 4 / 4 = 1, Denominator 10 / 4 = 2.5).
This is a ratio of totals, not an average of each user's own ratio. Averaging Ana (0.5), Ben (1) and Cara (0) would give 0.5, and Dan would have no ratio at all. With a ratio of totals, users with more page views carry more weight, and every participant can be included.
Compare that with a Goal Count of click on the same data: a Value of 4 clicks and a Mean of 4 / 4 = 1 click per participant. The Goal Count tells you how much users clicked. The ratio tells you how often a page view led to a click.
Creating the metric
When creating a metric, choose Ratio as the metric type, then:
- In Numerator Configuration, pick a Numerator type and set it up as you would that metric type: the Goal, plus the Property, Activity interval or Retention period if the type needs one, Metric property filters and, where the type allows it, Outliers limit method.
- In Denominator Configuration, do the same with a Denominator type. The denominator goal must share a datasource with the numerator goal.
- Optionally, under Unit level conditions, set Unit conditions to
Require denominator(see below). - In Format, Scale & Precision, choose how the ratio is displayed (see below).
Any metric type except Ratio and Custom SQL can be used as the numerator or denominator, and the two sides can be different types.
Both the Value and the Mean show the ratio, but a new metric starts with Value Precision set to 0 and the Mean shown as a percentage (Mean Format {}%, Scale 100). With those defaults, the example above would show a Value of 0 and a Mean of 40.000%. Set the Value precision, and the Mean format and scale, to match what your ratio measures: a percentage suits clicks per page view, but not revenue per order.
Require denominator
By default, every participant's numerator counts, even if they have no denominator. In the example, Dan's click counts although he has no page views.
With Require denominator set, a participant's numerator only counts if their denominator is not 0. Dan's click would be dropped, giving 3 / 10 = 0.3. Use it when the numerator only makes sense for users who reached the denominator, such as payments per checkout, where a payment without a tracked checkout shouldn't count.
Reading the results
In the metric results table, Value and Mean both show the ratio for the variant, and the Metric total row shows the total numerator divided by the total denominator across all variants. Unless the GST switch is on, you can hover over a variant's Value to see its numerator and denominator totals, and over its Mean to see their averages per participant.
The Relative impact compares the ratio between variants, as a continuous metric. The Impact per day column shows -- for ratio metrics.
The confidence interval takes into account that one user can contribute many page views, and that a user's numerator and denominator tend to move together.
A ratio can move because either side moved. If the ratio changes, check the Value tooltip in each variant to see whether the numerator, the denominator or both changed.
More examples
Click-through rate:Goal Countofclickdivided byGoal Countofpage_view.Average order value:Goal Propertyofpurchase(revenue) divided byGoal Countofpurchase.Items per order:Goal Propertyofpurchase(items) divided byGoal Countofpurchase.Purchases per active day:Goal Countofpurchasedivided byGoal Activity Period Countofapp_open(1 day).Payments per checkout:Goal Unique Countofpaymentdivided byGoal Unique Countofcheckout, withRequire denominator.
Good to know
- Metric property filters, Outliers limit method, Time filter and Goal relations are set separately for the numerator and the denominator, and each side uses its own settings.
- If you use CUPED, the numerator and denominator are each adjusted using their own pre-exposure value over the same lookback period. The Numerator and Denominator shown in the tooltips are the adjusted values.
- If the denominator total for a variant is 0, the ratio shows 0.