Skip to main content

Running Your First Experiment

Now that your SDK is set up and you have a context, you're ready to run an experiment. Here's what that looks like from your code's perspective:

  1. Check the variant to find out which experience this user should see
  2. Render the right experience based on that variant (or use variables to make it flexible)
  3. Track goals to measure whether the experiment is working
  4. Send the data to ABsmartly's collector

Once the context is ready, all treatment() calls are resolved locally with no additional network requests. Any experiment that isn't running will return 0 (Variant A, the control group).

Steps 1 and 3 are all you need for a basic experiment. Step 2 (variables) and Step 4 (manual event flushing) are useful but not required for your first test.

Step 1: Check which variant the user is in​

The treatment() method is how you find out which variant a user has been assigned to. It returns a number that maps to the variants you see in the Web Console:

NumberWeb Console nameMeaning
0Variant AControl (the baseline experience)
1Variant BFirst treatment
2Variant CSecond treatment
3Variant DThird treatment
.........

Calling treatment() also records an exposure, which tells ABsmartly that this user actually saw the experiment. This is how ABsmartly knows to include them in the statistical analysis.

Use an if/else block to branch your code based on the variant:

const treatment = context.treatment("homepage_banner_experiment");

if (treatment === 1) {
  showBanner("Check out our latest deals!");
} else {
  showBanner("Welcome back!");
}

This works well for experiments where the logic change is entirely in your code (show component A vs component B). But if you want the Web Console to control values like colours, copy, or feature flags, read on.

Step 2: Use variables for flexible experiments​

Instead of hardcoding what each variant does, you can use treatment variables. These are key-value pairs that you set in the Web Console and pull into your code at runtime.

The benefit: your product team can add new variants or tweak existing ones from the Web Console without anyone touching code.

For example, say you have an experiment to find out what colour of button gets the most clicks. In the Web Console, you'd add a variable called button.color with value "green" on Variant B. You don't need to set anything on Variant A - that's what the default value in your code is for. Then in your code, variableValue() returns the experiment value for the user's assigned variant, or the default if the experiment isn't running or the user is in the control group:

const buttonColor = context.variableValue("button.color", "blue");
document.getElementById("cta-button").style.backgroundColor = buttonColor;

Step 3: Track goals to measure success​

An experiment without goals is like running a test without checking the results. Goals are the actions you want to measure: did the user click the button? Complete a purchase? Sign up for a newsletter?

The track() method records these actions. Each goal corresponds to a goal_name defined in the Web Console. You can also pass properties to create more granular metrics.

The track() method records user actions. It takes two arguments:

  • goal_name: The name of the goal, matching one you've defined in the Web Console. Any name that doesn't match will be ignored (unless AUTO_GOAL_CREATION is enabled on your instance).
  • properties (optional): Key-value pairs for extra metrics or filtering (e.g., revenue, item category).
var properties = {
  price: 10000,
  category: "5 stars",
  free_cancellation: true,
  instance_id: 5350,
};

context.track("booking", properties);
info
  • By default, the goal_name must match exactly what you defined in the Web Console. Anything else will be ignored. However, if your instance has AUTO_GOAL_CREATION enabled, any unseen goals will be automatically created in the Web Console.
  • Properties are optional but useful for filtering and creating extra metrics.

Step 4: Make sure your data gets sent​

Most SDKs batch events and send them to ABsmartly's collector automatically. For most use cases, you don't need to do anything here.

Node.js, PHP, and Ruby users

The Javascript SDK running in Node.js, the PHP SDK, and the Ruby SDK do not auto-publish events. You must call publish() or finalize() explicitly at the end of each request, or your exposure and goal data will not be sent.

Even with auto-publishing, there are times when you need to guarantee events are sent before moving on. For example, before a page navigation in a multi-page app, or before a server sends its response. In those cases, call publish() explicitly:

Using publish() to flush events manually

Call publish() before page navigations, server responses, or any point where the process might exit before the SDK's auto-flush fires. This ensures exposure and goal data is sent to the collector before the context is lost.

// You can just publish
context.publish();

// or wait for it to finish, so if you want to
// navigate to another page without losing impressions,
// you can do that once the promise resolves.
context.publish().then(function () {
  document.location.replace("another_page");
});

Putting it all together​

Here's a realistic starting point you can copy and adapt. It covers SDK init, context creation, treatment checking, variable usage, goal tracking, and cleanup.

const absmartly = require("@absmartly/javascript-sdk");

// Initialize the SDK
const sdk = new absmartly.SDK({
  endpoint: "https://your-company.absmartly.io/v1",
  apiKey: "YOUR-API-KEY",
  environment: "production",
  application: "website",
});

// Create a context for this user
const request = {
  units: {
    user_id: "user-12345",
  },
};

const context = sdk.createContext(request);

context.ready().then(() => {
  // Check which variant the user is in
  const treatment = context.treatment("homepage_banner_experiment");

  if (treatment === 1) {
    // Variant B: show a personalized banner
    showBanner("Welcome back, we have new deals for you!");
  } else {
    // Variant A (control): show the existing banner
    showBanner("Welcome back!");
  }

  // Use a variable set in the Web Console for more flexibility
  const buttonColor = context.variableValue("button.color", "blue");
  setButtonColor(buttonColor);

  // Track when the user clicks the CTA button
  document.getElementById("cta-button").addEventListener("click", () => {
    context.track("cta_click", { page: "homepage" });
  });

  // Track a purchase with revenue data
  onPurchaseComplete((order) => {
    context.track("purchase", {
      revenue: order.total,
      item_count: order.items.length,
    });
  });
});

// When the user leaves or the app shuts down, finalize the context
// to flush any remaining events
window.addEventListener("beforeunload", () => {
  context.finalize();
});

Next steps​

You now have everything you need to run a basic experiment. When you're ready to go deeper: