Skip to main content

Goal Activity Period Count

Overview​

A Goal Activity Period Count metric answers the question: on how many separate days (or hours, or weeks) did each user do this?

You pick a goal and an activity interval, for example 1 day. For each user, ABsmartly splits time into periods of that length and counts how many of those periods contain at least one goal event. Doing the action once or fifty times in the same period counts the same: one active period.

This makes it a good fit for questions about habits and return visits rather than volume, such as "did this change make people come back on more days?"

How it's calculated​

Say you track an app_open goal and create a metric with an activity interval of 1 day.

context.track("app_open");

Three users are exposed to the experiment:

UserExposedapp_open events after exposureActive daysValue
AnaMondayMonday (twice), WednesdayMonday, Wednesday2
BenMondayTuesday (12 times)Tuesday1
CaraTuesdaynonenone0

In the experiment results, this group would show a Value of 3 (2 + 1 + 0 active days in total) and a Mean of 3 / 3 = 1.0 active day per participant.

Compare that with a Goal Count on the same goal, which counts every event: Ana 3, Ben 12 and Cara 0, so a Value of 15 and a Mean of 5. Ben's burst of activity dominates the count, while the activity period count shows that Ana came back on more days.

Creating the metric​

When creating a metric, choose Goal Activity Period Count as the metric type, then:

  1. Select the goal you want to measure, for example app_open.
  2. Set the Activity interval: a number and a unit (Hours, Days or Weeks).
  3. Optionally add filters on the goal's properties, as with other goal-based metrics.

The activity interval can be at most 4 weeks (28 days, or 672 hours).

How periods are defined​

Periods are currently fixed time slots in UTC, the same for every user. For example, a 1 day interval means UTC calendar days, from 00:00 to 24:00 UTC. A 1 week interval currently runs from Thursday 00:00 UTC to the following Thursday, so a goal event on Wednesday and another on Thursday count as two active weeks. Longer intervals are counted from 1 January 1970 UTC, so a 2 weeks interval starts on alternate Thursdays, and an hourly interval that doesn't divide a day evenly, such as 5 hours, won't always start at midnight.

Because the slots don't start at the user's exposure, a user's first period can be partly over by the time they are exposed. With a 1-day interval, a user exposed at 23:50 UTC who triggers the goal at 23:55 and again at 00:05 has two active days, even though only ten minutes passed.

Reading the results​

In the metric results table, Value is the total number of active periods across all participants in the variant, and Mean is that total divided by the number of participants. The impact is calculated on the Mean, as with other continuous metrics such as Goal Count.

If CUPED is active for the metric, both Value and Mean are CUPED-adjusted, so they won't match the raw count of active periods and can be fractional. Configuring CUPED isn't always enough: if the pre-exposure covariate doesn't correlate well enough with the metric, or the adjustment wouldn't reduce variance, ABsmartly falls back to the unadjusted values.

Because the value counts periods rather than dividing by them, it grows the longer users stay in the experiment. A user exposed on day one has more chances to be active than a user exposed yesterday. That's fine for comparing variants, since both groups have the same mix of exposure times, but don't read the raw number as "days active per week".

More examples​

  • Active days: goal app_open, interval 1 day. How many different days users opened the app.
  • Active weeks with a purchase: goal purchase, interval 1 week. How many different weeks users bought something.
  • Active hours watching: goal video_play, interval 1 hour. How many different hours users started a video.
  • Purchases per active day: a Ratio metric with a Goal Count of purchase as the numerator and a Goal Activity Period Count of app_open (1 day) as the denominator.

Good to know​

  • Great for engagement and habit experiments: onboarding flows, notifications, reminders, streaks or loyalty features.
  • Less sensitive to bursts than Goal Count, because repeat events in the same period add nothing.
  • Filters on the goal apply before periods are counted, so Active days with a purchase in category ZYZ123 only counts days with a matching purchase.
  • If you cancel goal events, a period stops counting only when every event in it has been cancelled.
  • If you use CUPED, the pre-exposure covariate is the same count of active periods, measured over the lookback window before exposure.
  • Changing the activity interval changes what the metric means and requires a new metric version.