Skip to main content

Dynamic Data Sources for Development/Production Environment

Coming Soon

This feature is under development and will be coming soon!

info

Holistics support this thanks to our ability to define dynamic data sources.

Introduction

In a typical development workflow, you'll have several environments: Production, Development, and Staging, each using different data sources.

This document will show you how to set up this workflow in Holistics.

Dynamic Development/Production environment

How it works

At a high level, the data source will dynamically switch based on which branch users are on.

Holistics provides a global dynamic variable, H.git.current_branch, which returns the name of the current branch.

You can then use conditional statements (if-else) to dynamically switch the database or schema name based on the branch.

1. Connect databases

Note that the pre-requisite of the flow is that you need to connect to all of your databases first (development, staging, production). Please refer to this link for more information.

2. Create a databases.aml file to map the datasource based on the current branch

// H.git.current_branch is a dynamic variable, injected by Holistics
const current_branch = H.git.current_branch

// This is for dynamic database use case by branch
const dynamic_db = if (H.git.current_branch == 'master') {
'production'
} else if (H.git.current_branch == 'staging') {
'staging'
} else {
'develop'
}

// This is for dynamic schema use case by branch
const dynamic_schema = if (H.git.current_branch == 'master') {
'prod'
} else if (H.git.current_branch == 'staging') {
'stg'
} else {
'dev'
}
Note

The file database.aml will be stored in git, so please don’t store sensitive information credentials there

3. Use the dynamic_db and dynamic_schema in datasets/models

Model users {
label: 'Users'
description: 'Info about Users'
data_source_name: dynamic_db
table_name: '${dynamic_schema}.revenue'
}

Dataset ecommerce {
label: 'Ecommerce'
description: 'Dynamic Data Source Ecom Dataset'
data_source_name: dynamic_db
models: [users, orders]
relationships: [orders.user_id > users.id, true]
}

For example:

  • If you change to develop branch => your database will be develop
  • If you change to staging branch => your database will be staging
  • If you deploy => your project on master branch will use the database production

Available Variables

While not exhaustive, here are some common system variables you can use:

  • H.git.current_branch → this will return the name of the current branch
  • H.git.is_production → this will return true when in Reporting tab, or in Development tab with production mode enabled.

Let us know what you think about this document :)