Dynamic Data Sources for Development/Production Environment
This feature is under development and will be coming soon!
Holistics support this thanks to our ability to define dynamic data sources.
Introduction
In a typical development workflow, you'll have several environments: Production, Development, and Staging, each using different data sources.
This document will show you how to set up this workflow in Holistics.
How it works
At a high level, the data source will dynamically switch based on which branch users are on.
Holistics provides a global dynamic variable, H.git.current_branch
, which returns the name of the current branch.
You can then use conditional statements (if-else) to dynamically switch the database or schema name based on the branch.
1. Connect databases
Note that the pre-requisite of the flow is that you need to connect to all of your databases first (development, staging, production). Please refer to this link for more information.
2. Create a databases.aml
file to map the datasource based on the current branch
// H.git.current_branch is a dynamic variable, injected by Holistics
const current_branch = H.git.current_branch
// This is for dynamic database use case by branch
const dynamic_db = if (H.git.current_branch == 'master') {
'production'
} else if (H.git.current_branch == 'staging') {
'staging'
} else {
'develop'
}
// This is for dynamic schema use case by branch
const dynamic_schema = if (H.git.current_branch == 'master') {
'prod'
} else if (H.git.current_branch == 'staging') {
'stg'
} else {
'dev'
}
The file database.aml
will be stored in git, so please don’t store sensitive information credentials there
3. Use the dynamic_db and dynamic_schema in datasets/models
Model users {
label: 'Users'
description: 'Info about Users'
data_source_name: dynamic_db
table_name: '${dynamic_schema}.revenue'
}
Dataset ecommerce {
label: 'Ecommerce'
description: 'Dynamic Data Source Ecom Dataset'
data_source_name: dynamic_db
models: [users, orders]
relationships: [orders.user_id > users.id, true]
}
4. Switching branch will switch the data source name of the related models and datasets
For example:
- If you change to
develop
branch => your database will bedevelop
- If you change to
staging
branch => your database will bestaging
- If you deploy => your project on
master
branch will use the databaseproduction
Available Variables
While not exhaustive, here are some common system variables you can use:
H.git.current_branch
→ this will return the name of the current branchH.git.is_production
→ this will returntrue
when in Reporting tab, or in Development tab with production mode enabled.