# Get Started with Warehouse Native

Warehouse Native lets you run experiments directly on data that lives in your own data warehouse. Instead of sending events to ABsmartly's collectors, you point ABsmartly at your existing tables — exposures, goals, and attributes — and it takes care of the rest.

> **New to Warehouse Native?**
>
> This guide covers connecting and configuring a data source. If you're deciding between **Hybrid** and **Fully** Warehouse Native first, read [Warehouse Native Modes](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md).

Getting started takes three steps:

1. **[Connect to your data warehouse](#step-1-connect-to-your-data-warehouse)** — set up credentials and test the connection
2. **[Map your tables](#step-2-map-your-tables)** — tell ABsmartly which columns hold exposures, goals, and attributes
3. **[Configure data freshness](#step-3-configure-data-freshness)** — define how often ABsmartly should query for new data

***

## Step 1: Connect to your data warehouse

To begin, navigate to **Settings → Data Sources** and click **Add data source**.

> **The default data source**
>
> Every installation ships with a built-in **default** data source, managed by ABsmartly. In [Hybrid mode](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#hybrid-warehouse-native) you keep this default and add external data sources for your goals. In [Fully mode](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#fully-warehouse-native) your own warehouse becomes the default. Exposure events always come from the default data source, so which one is the default determines your mode.

You'll see a form with two sections: general information about the data source, and the connection credentials specific to your warehouse type.

### General information

Start by giving your data source a **name** and an optional **description**. The name will be used throughout ABsmartly to identify this connection, so pick something recognizable (e.g., "Production BigQuery" or "Analytics Snowflake").

### Choose your warehouse type

Select your warehouse from the **Database type** dropdown. ABsmartly supports five warehouse types — follow the guide for yours:

* **[BigQuery](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/bigquery.md)**
* **[Snowflake](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/snowflake.md)**
* **[ClickHouse](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/clickhouse.md)**
* **[Redshift](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/redshift.md)**
* **[Databricks](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/databricks.md)**

Each guide walks you through the specific credentials needed and how to test the connection.

Once connected, save the data source and move on to mapping your tables.

***

## Step 2: Map your tables

With your warehouse connected, the next step is to tell ABsmartly where your experiment data lives. You'll map three types of tables: **exposures**, **goals**, and optionally **attributes**.

For each table, you can either use the **Column mapping** tab to select columns from a dropdown, or switch to the **SQL query** tab to write a custom query.

> **Which tables apply to your data source**
>
> Exposures always come from the **[default data source](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#the-default-data-source)**, so the exposure mapping below matters most on whichever data source is your default. On an **external** (non-default) data source used in [Hybrid mode](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#hybrid-warehouse-native), you'll typically map only the **goals** table — exposures are brought in automatically through the [Exposures import](#step-4-configure-exposures-import-hybrid-only) setup rather than mapped here.

### Exposure table

The exposure table tracks experiment assignments — which unit was exposed to which variant of which experiment, and when.

Select your table from the **External table name** dropdown. ABsmartly will read the table's schema and populate the column dropdowns automatically.

![Mapping exposure table columns with dropdown open](/assets/images/exposure-mapping-dropdown-b785437c21c9ac29ba4a0feaa5411a6a.png)

Map each field to the corresponding column in your table:

| Field              | Description                                     | Accepted types         |
| ------------------ | ----------------------------------------------- | ---------------------- |
| **unit\_uid**      | Unique identifier for the unit                  | `STRING`, `INT64`      |
| **unit\_type\_id** | Type of the unit (e.g., `user_id`, `device_id`) | `STRING`, `INT64`      |
| **experiment\_id** | Unique identifier for the experiment            | `STRING`, `INT64`      |
| **variant**        | Variant/group assigned to the unit              | `STRING`, `INT64`      |
| **exposed\_at**    | Timestamp when the exposure occurred            | `INT64` (milliseconds) |
| **attributes**     | Attribute values as a JSON string               | `STRING`               |

> **tip**
>
> Column names don't have to match exactly — ABsmartly lets you map any column (or expression) to each field. For example, you could use `CAST(created_at AS INT64)` for the `exposed_at` field.

Once all fields are mapped, click **Test & fetch rows** to validate the mapping. ABsmartly will fetch a sample of 10 rows so you can verify the data looks right.

### Goals table

The goals table contains your conversion and metric events — the outcomes you're measuring in your experiments.

Map the following fields:

| Field              | Description                                     | Accepted types         |
| ------------------ | ----------------------------------------------- | ---------------------- |
| **goal\_id**       | Unique identifier for the goal/conversion event | `STRING`, `INT64`      |
| **unit\_uid**      | Unique identifier for the unit                  | `STRING`, `INT64`      |
| **unit\_type\_id** | Type of the unit (e.g., `user_id`, `device_id`) | `STRING`, `INT64`      |
| **achieved\_at**   | Timestamp when the goal was achieved            | `INT64` (milliseconds) |
| **properties**     | Additional goal properties as a JSON string     | `STRING`               |

As with the exposure table, click **Test & fetch rows** to validate.

### Attributes table *(optional)*

The attributes table holds per-unit attribute snapshots used for filtering or segmenting experiments. This table is optional — skip it if you don't have an attributes source for this data source.

Map the following fields:

| Field              | Description                                     | Accepted types         |
| ------------------ | ----------------------------------------------- | ---------------------- |
| **unit\_uid**      | Unique identifier for the unit                  | `STRING`, `INT64`      |
| **unit\_type\_id** | Type of the unit (e.g., `user_id`, `device_id`) | `STRING`, `INT64`      |
| **set\_at**        | Timestamp when the attribute snapshot was set   | `INT64` (milliseconds) |
| **attributes**     | Attribute values as a JSON string               | `STRING`               |

![Completed attributes table mapping](/assets/images/attributes-mapping-complete-d8d7c27087074e562125638e2d9cd298.png)

### JSON layouts table

ABsmartly uses a `json_layouts` table to power **auto-complete for JSON properties** in the metric editor. When you're defining metrics that reference goal properties or attributes, this table tells ABsmartly which JSON keys exist, their types, and where they were observed — so you get suggestions instead of typing paths from memory.

ABsmartly looks for a `json_layouts` table with six columns (`key`, `source`, `source_id`, `phase`, `value_type`, `event_at`) in your data source's default location. If it finds one with the expected schema, you'll see a green confirmation:

![JSON layouts table automatically detected](/assets/images/json-layouts-detected-bb7f213d5d45a7e5d3ccb57035fcb0ac.png)

If the table doesn't exist yet, ABsmartly will show a warning and offer two options:

* **Create the table** — ABsmartly creates a `json_layouts` table for you (save the data source first)
* **Use an existing table** — map your own table's columns to the expected schema

![JSON layouts table not found warning](/assets/images/json-layouts-table-0ccaf4581fa61e24688cf9372c071e2e.png)

If you choose to map an existing table, you'll see the column mapping interface:

![JSON layouts table column mapping](/assets/images/json-layouts-mapping-1a89eb169c9418fb3455d4f1576dc996.png)

| Field           | Description                                                                  | Accepted types    |
| --------------- | ---------------------------------------------------------------------------- | ----------------- |
| **key**         | JSON property path (e.g., `amount`, `customer.tier`)                         | `STRING`          |
| **source**      | Where the key was observed: `unit_goal_property` or `unit_attribute`         | `STRING`          |
| **source\_id**  | Goal ID (for goal properties) or attribute ID (for attributes)               | `STRING`, `INT64` |
| **phase**       | Pipeline phase: `before_enrichment` or `after_enrichment`                    | `STRING`          |
| **value\_type** | JSON value type: `null`, `string`, `number`, `boolean`, `array`, or `object` | `STRING`          |
| **event\_at**   | Timestamp of the most recent observation (milliseconds)                      | `INT64`           |

> **note**
>
> Without this table, experiments will still work — but JSON-property auto-complete will be empty for goals and attributes on this data source, requiring manual entry when configuring metrics.

> **note**
>
> On BigQuery with a **Staging dataset** set, the `json_layouts` table lives in the staging dataset instead. With **Trust relationship** authentication, JSON layouts are turned off until you set one. With **Service account key (JSON)** and no staging dataset, JSON layouts use the ABsmartly dataset. See [Create the staging dataset](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/bigquery.md#4-create-the-staging-dataset).

***

## Step 3: Configure data freshness

Finally, tell ABsmartly how often your warehouse receives new data so it can query at the right times.

![Data freshness configuration](/assets/images/data-freshness-a97d887f41c39811017cb1105d3074b2.png)

Choose an **Update frequency** that matches how often your ETL pipeline lands new data:

| Frequency                  | Best for                                        | What you configure                  |
| -------------------------- | ----------------------------------------------- | ----------------------------------- |
| **Every X minutes**        | Near real-time pipelines or streaming ingestion | Interval in minutes                 |
| **Every X hours**          | Regular batch ETL jobs (e.g., every 6 hours)    | Interval in hours                   |
| **Daily at specific time** | Once-a-day loads                                | Offset in minutes from midnight UTC |

For all frequency types, use the **Offset (minutes)** field to control when the schedule starts counting from. This is the number of minutes after midnight UTC.

ABsmartly will show a preview of the computed schedule in both UTC and your local timezone. For example:

* **Every 6 hours** with offset 0 → Updates at 00:00, 06:00, 12:00, 18:00 UTC
* **Daily** with offset 480 → Updates at 08:00 UTC (10:00 CEST)

> **tip**
>
> Match the frequency to your actual ETL schedule. If your data lands once a day at 8 AM UTC, use **Daily at specific time** with a 480-minute offset. Querying more often than data arrives just adds unnecessary load to your warehouse.

Save the data source. If you're setting up [Fully Warehouse Native](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#fully-warehouse-native) and this warehouse is your **default** data source, you're done - ABsmartly will start querying your warehouse on the configured schedule and your experiment data will flow through automatically. The built-in ABsmartly-managed default data source needs no such configuration, since it isn't a warehouse of yours. If you're connecting an **external** data source for [Hybrid mode](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#hybrid-warehouse-native), continue to Step 4.

***

## Step 4: Configure exposures import *(Hybrid only)*

This step only applies to **external** (non-default) data sources — the setup used in [Hybrid Warehouse Native](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#hybrid-warehouse-native). If your warehouse is the default data source, you can skip it.

In Hybrid mode, a goal can live in an external warehouse, but exposures come from the default data source (managed by ABsmartly). To compute metrics for such a goal, ABsmartly needs to make those exposures available to your external warehouse. It does this through an object store that you provide - you point ABsmartly at a storage location it can use, and your warehouse reads the exposures from there.

The **Exposures staging** section configures that storage location:

| Field                | Description                                                                                                                                                                                                                                                                                                                                                    |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Object store URL** | The bucket/prefix ABsmartly uses to stage exposures for your warehouse. The accepted scheme depends on your warehouse (for example, `gs://` for BigQuery, `s3://` for Redshift, and `s3://`, `gs://`, or `azure://` for Snowflake).                                                                                                                            |
| **Credentials**      | The access needed to read and write that location — an access key/secret for ClickHouse, a stage name for Snowflake, an IAM role ARN for Redshift. Databricks needs no extra credentials. For BigQuery, pick the method in the section's own **Authentication** dropdown: **Access key (HMAC)** (the default, an access key/secret) or **Trust relationship**. |

With **Trust relationship** on BigQuery, the access key and secret fields are replaced by ABsmartly's service account email: grant that service account **Storage Object User** (`roles/storage.objectUser`) on the staging bucket. The dropdown is independent of the data source's own **Authentication** setting, but BigQuery always reads the staged files with the data source's credentials. That service account needs access to the bucket too: **Storage Object User** for ABsmartly's service account under Trust relationship (see [Grant access to the exposures staging bucket](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/bigquery.md#5-grant-access-to-the-exposures-staging-bucket-hybrid-only)), or **Storage Object Viewer** (`roles/storage.objectViewer`) for your own service account under Service account key (JSON).

With **Access key (HMAC)**, the `storage.restrictAuthTypes` organization policy on the bucket's project must allow HMAC authentication.

On BigQuery, Hybrid exposures import requires a [staging dataset](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/connect/bigquery.md#4-create-the-staging-dataset) with either **Service account key (JSON)** or **Trust relationship** as the data source's authentication, regardless of the exposures staging **Authentication** choice. The import creates its temporary external table there. ABsmartly won't save a data source with an exposures import, or run **Test exposures import**, until you set a **Staging dataset**. A data source saved earlier without a staging dataset keeps failing Hybrid metric queries until you add one, and its next save requires it. Put the bucket in the same location as the staging and source datasets, or, for a `US` or `EU` multi-region dataset, in a single region inside that multi-region (for example `europe-west1` for `EU`).

> **note**
>
> Staged files are cached and reused for a short window, then cleaned up automatically — no manual cleanup is required. For defense in depth, you can also add a bucket lifecycle rule (e.g. "delete objects older than 1 day") as a backstop.

Save the data source once the exposures import is configured. ABsmartly will then query your warehouse on the configured schedule, evaluating your externally-sourced goals against the exposures automatically.

In [Fully Warehouse Native](https://docs.absmartly.com/docs/web-console-docs/warehouse-native/modes.md#fully-warehouse-native), your warehouse is the default and already holds the exposures, so this step doesn't apply.
