ETL Pipelines integration is available for the Scale plan.

​
Event Structure

Each event is published with these Cloud Pub/Sub message attributes:

TypeKeyValue
Attributeevent-typeVital event type, e.g., daily.data.activity.created
Attributeblob-typejson : UTF8 JSON document
Attributeblob-codecgzip: The blob is compressed by gzip. null or absent: The blob is uncompressed.
Attributeidempotency-keyThe event delivery ID; only unique amongst all events from the same user and of the same event type.
Ordering Key-Vital User ID (If you have enabled Event Ordering)

Payload smaller than 1KiB would not be compressed. You must check blob_codec to identify if decompression of the message payload blob is necessary.

At this time, Vital publishes exclusively JSON blobs (blob-type == json).

Having said that, you are strongly encouraged to check for and drop any events with unrecognized blob-type and blob-codec.

​
Pub/Sub event ordering

If you wish to receive event for each Vital user in their publication order, Vital supports Pub/Sub Event Ordering, and uses the Vital User ID as the event ordering key.

Vital publishes all events through the us-central1 and europe-west1 PubSub regional endpoints.

Pub/Sub Exactly Once Delivery is discouraged to be used together with Event Ordering. Our own trial suggested that the combination would struggle to move high volume of events.

​
IAM Permission

Grant the relevant IAM Principals on your topic with these pre-defined roles:

  • Pub/Sub Publisher (roles/pubsub.publisher)
  • Pub/Sub Viewer (roles/pubsub.viewer)

Or more specifically these permissions if you wish to create a custom IAM role with a minimized grant:

  • pubsub.topics.get
  • pubsub.topics.publish
EnvironmentRegionService Account
SandboxUScustomer-topics-us@vital-sandbox.iam.gserviceaccount.com
SandboxEuropecustomer-topics-eu@vital-sandbox.iam.gserviceaccount.com
ProductionUScustomer-topics-us@vital-prod-307415.iam.gserviceaccount.com
ProductionEuropecustomer-topics-eu@vital-prod-307415.iam.gserviceaccount.com

​
Configuration

Org Management API is available for the Scale plan.

You can manage your RabbitMQ destination through the Org Management API Team ETL Pipeline endpoints.

A basic configuration would look as such:

{
  "team_id": "TEAM_ID",
  "preferences": {
    "enabled": ["cloud_pubsub"],
    "preferred": "cloud_pubsub",
  },
  "cloud_pubsub": {
      "project": "my-gcp-project-123456",
      "topic": "event-ingestion",
      "message_ordering": true
  },
  "push_historical_data": true
}
OverviewRabbitMQ