Google Analytics BigQuery Export

The Google Analytics BigQuery Export connector is a beta feature according to Algolia’s Terms of Service (“Beta Services”).

If you’re already using Google Analytics 4 (GA4) on your website, you can use Algolia’s BigQuery Export connector to capture Insights events for analytics and AI features.

Google Analytics lets you export events to BigQuery from where Algolia’s connector can import them. This guide shows how to set up the connector.

Set up Google Analytics BigQuery Export

Follow Google’s Set up BigQuery Export guide for Google Analytics 4 to link your Google Analytics property to BigQuery.

When setting up the export in GA4, choose the streaming export, rather than the daily one. This lets Algolia import events throughout the day instead of after a delay.

Once you’ve made the BigQuery connection, Google syncs your GA4 data with BigQuery. If your GA property is new, you need to collect some data first before you can see anything in BigQuery.

Create a Google Cloud service account and key

Algolia’s GA4 BigQuery Export connector needs permission to read from your BigQuery tables so that it can extract the data and import it into your events dataset.

To read your BigQuery tables, the connector requires a Google service account key.

First, create the Google service account:

1. Follow the guide Create service accounts to create a service account.

   1. Enter a name and ID for the service account.

   2. When prompted for IAM roles to grant the service account specify the BigQuery Job User role.

2. Go to the BigQuery console.

3. From the actions menu of the dataset containing the BigQuery Export select Share > Manage Permissions.

4. Click Add Principal and in the New principals text box select the name of the service account.

5. In the Select a role drop-down menu select the BigQuery > BigQuery Data Viewer role.

Next, create a key attached to this service account:

1. In the Google Cloud console, go to Service accounts.

2. Select the service account that you created in the previous step. Under Actions, select Manage keys:

3. Select Add key > Create new key.

4. Choose JSON for the Key type.

5. Press Create.

Make a note of the location on your computer where the newly created key file is saved.

You now have a Google service account with the required permissions and a key you can provide to the Algolia connector.

Create GA4 BigQuery Export connection in Algolia

To link your BigQuery data to Algolia, follow these steps:

1. Click this link: Google Analytics 4 BigQuery Export to create a new connector.

Configure your data source

The connection’s source tells Algolia how to read from the BigQuery tables that Google Analytics syncs to.

1. Under Google Service Account, select Create a new Google Service Account authentication.

   1. Upload a key file for a Google service account with these roles: BigQuery Data Viewer and BigQuery Job User.

   2. Enter a name for this account.

   3. Click Create authentication.

2. Under Enter your BigQuery table details, enter your project ID and dataset ID from your BigQuery project.

3. Select a table type:

   • Streaming if you configured your GA4 BigQuery link for streaming exports (recommended).

   • Daily, if you configured your GA4 BigQuery link to export daily, or if you use both streaming and daily exports.

   If both daily and streaming exports are active, the streaming tables expire at the end of each day, potentially before they’re imported.

4. Enter a name for this data source.

5. Click Save and continue.

Configure your destination

The connection’s destination tells Algolia where to send source data. For Google Analytics events data the destination is the Insights API.

1. Press Create a new destination.

2. Enter an index name for Algolia events imported by this connection.

3. To create Algolia credentials for the connector, click Create one for me.

4. Click Save and continue.

Configure how to map your data

The mapping section tells Algolia how to convert GA events to Algolia events.

Algolia needs only a subset of the events that are typically captured in a GA implementation. The table below indicates what type of events each Algolia feature needs to function. Create a mapping for any events in this table that are present in your GA.

The mapping UI suggests mappings based on events found in your GA BigQuery Export that conform to the set of recommended GA ecommerce events. For other GA events you can define a custom mapping.

Many of Algolia’s AI and analytics features rely on events being associated to a search with a query ID. For information how to add query IDs to your GA implementation, see Attribute events to searches.

Only create mappings for event types that you are not already sending to Algolia from other ingestion methods. Otherwise those events may be recorded twice. For example, if you already send clicked items after searches from a frontend library such as InstantSearch, don’t map these for ingestion through the Google Analytics Connector. Create a mapping only for events that aren’t already captured, for example clicked items without queryID or purchased items with and without queryID.

Algolia only syncs events that have a mapping.

How do I map my conversion events?

If you are an ecommerce customer, please map your GA add to cart to Algolia addToCart and your GA purchase to Algolia purchase and you are done. Other conversion events are not required. However, if you do track additional conversion types (for example, add to wishlist), please map those to Algolia conversion.

How do I use GA user tokens in my search requests?

Google Analytics’ persistent, anonymous user token is present in imported GA events and should be provided with search requests to Algolia to enable analytics and query personalization.

The Google Analytics user token can be retrieved by using this JS snippet and providing it when performing searches.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
const algoliasearch = require('algoliasearch');
const client = algoliasearch('YourApplicationID', 'YourWriteAPIKey');
const index = client.initIndex('test_index');
const userToken = getGoogleAnalyticsUserIdFromBrowserCookie('_ga');

// Use the GA user token when performing searches.
index.search('test', {
    userToken: userToken,
});

// The logged in user should be preferred if available.
const loggedInUser = '...';

index.search('test', {
    userToken: loggedInUser || userToken,
});

Create your task

Review your connection before creating it.

Click Create task to start syncing your GA events with Algolia.

Check your GA4 BigQuery Export connection in Algolia

After creating your connection, use Algolia’s Connectors Debugger and Events Debugger to see the progress of the sync.

Connectors debugger

Open the Connector Debugger to ensure that your events get imported from Google Analytics.

If your events aren’t imported, the following section includes some troubleshooting steps.

Check for mapping errors

You might need to edit your mapping if you don’t use recommended GA fields. For example, if your GA event uses something other than items and you don’t change that in your mapping, then your events won’t sync and you’ll see an error such as Algolia API error [422] {"status":422,"message":"Event should specify either some ObjectIDs or some Filters"}.

Events debugger

Open the Events Debugger to ensure that your GA events get mapped and imported.

Check for successful requests which show that your events have been imported.

For a more thorough check, select an event from the debugger’s table and make sure the JSON in the details panel is correct.

Attribute events to searches

Each Algolia search is associated with an identifier called the query ID which is used to attribute click and conversion events to searches. This query ID isn’t tracked by default by GA but one can link GA events to their originating search by keeping track of the current query ID and recording it as a custom event attribute in GA.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// This will be turned into a click event associated with a search.
gtag('event', 'select_item', {
    // We use algolia_query_id to not conflict with GA's query_id.
    algolia_query_id: 'tracked query id',
    items: [{
      item_id: 'object1',
      item_name: 'red shoes',
    }],
});

// For purchases, different items can be associated with different searches.
gtag('event', 'purchase', {
    items: [{
        item_id: 'object1',
        algolia_query_id: 'object1 query',
    }, {
        item_id: 'object2',
        algolia_query_id: 'object2 query',
    }]
});