BigQuery

This document will walk you through step-by-step instructions to get your BigQuery connector up and running.

1. Connect to your Warehouse

Complete the following steps:

1. Navigate to Project Settings, then select Warehouse Sources.

2. Click on + Add Connection and select BigQuery.

3. You should see a new page to create your BigQuery connector. In the first view, fill out the required field GCP Project ID.

   • Run the commands below in Cloud shell & click Create Source

   gcloud projects add-iam-policy-binding <your_gcp_project_id> \
   --member serviceAccount:project-2946576@mixpanel-warehouse-1.iam.gserviceaccount.com \
   --role roles/bigquery.dataViewer
   gcloud projects add-iam-policy-binding <your_gcp_project_id> \
   --member serviceAccount:project-2946576@mixpanel-warehouse-1.iam.gserviceaccount.com \
   --role roles/bigquery.jobUser

4. In the second view, you should see that the credentials are validated and a source is added.

2. Format Data for Mixpanel Ingestion

To import data from a warehouse directly, make sure the tables are formatted correctly.

Each column in the table will be mapped as a property on the event.

Read more about best practices for tracking Events and Properties in Mixpanel here (opens in a new tab).

The following columns are required:

• Event Time
• Distinct ID
• Insert ID
  • This is an optional field, but we recommend it as a best practice in case duplicate events need to be reconciled.
• Insert Time
  • We expect this to be a monotonically increasing TIMESTAMP_NTZ. We use this as a water mark to figure out which events are new and ingest them.
• Event Name
  • You can specify the column that has this value.
  • Alternatively, you can specify a static value during creation.

3. Send Events

Once you have created a warehouse source, follow the below steps to send events into Mixpanel.

1. Go to Project Settings > Warehouse Data.
2. Click on + Add Warehouse Event.
3. Select the BigQuery as source from the dropdown.
4. Select the Dataset.
5. Select the Table.
6. Specify the Event Name:
   • You can specify the column which has the event name.
   • OR You can specify the event name for all the events.
7. Specify the Event Time:
   • This is the time the event occurred.
8. Select the column to be used for Insert ID.
9. Identity Management:
   • If your project has Simplified ID Merge, the warehouse connector will prompt you to map the $device_id and $user_id properties to columns in your table during setup. For more information on how we map those fields to the resulting distinct id, refer to our documentation here (opens in a new tab).
   • If your project has Original ID Merge, the warehouse connector will prompt you to map the Distinct ID column. This column represents the entity to which the events belong; it could be a user or a device. If you want to map two entities together you have to additionally call either $identify, $merge, $alias functions, along with their corresponding properties. For more information, refer to our documentation here (opens in a new tab).
10. Check the preview to ensure that everything looks correct, and then create the event.