Skip to main content

Getting started with Holistics As-code (Holistics 4.0)

Knowledge Checkpoint

A grasp of these concepts will help you understand this documentation better:

Introduction​

In Holistics 4.0, the major update is the capability to implement software engineering best practices in the analytics workflow, which also known as As-code.

This documentation aims to guide you through the process of getting started with As-code, using tutorials for both the code-based interface and the GUI.

Please note that As-code features are only available by default on Holistics 4.0. If you are on Holistics 3.0 and would like to try them out, please submit a request to us at [email protected]

Holistics 4.0 Demo Environment​

If you only want to try out Holistics 4.0 demo environment, please click here or visit: https://demo4.holistics.io/demo

Connect to Data Warehouse​

The first thing to do is to connect Holistics to your SQL database (usually data warehouse). This allows your analysts to run and validate the project setup and confirm that the expected results are returned.

If you set up Holistics the first time, simply follow the in-app guide.

If you're adding additional SQL connection, go to Settings ⇒ Data Sources ⇒ New Data Source and fill in your connection details.

In order to check which database Holistics supports natively and how to connect, you can refer to our document about Connect Database.

Enable Development mode and start working on your project​

Each project has two modes (development and production). Development mode allows for changes without affecting end-users, while Production mode is the finalized version of datasets for end-users.

For more information about Project Mode, you can refer to Development Mode, Production Mode and Deploy

Develop Data Model​

info

For more information, please refer to Model Syntax doc

There are two types of data model in version 4.0: Table Model and Query Model.

Table Model​

To make an existing SQL table available in Holistics modeling layer.

Step 1: Add a model object in the model file and name it

Model users { }

Step 2: Inside the curly brackets { }, define model's metadata

type: 'table'
label: 'Users'
description: ''
data_source_name: 'demodb'
owner: '[email protected]'
table_name: '"ecommerce"."users"'

Step 3: Specify which dimensions and measures you want to include in the model.

dimension id {
  label: 'Id'
  type: 'number'
}
dimension sign_up_date {
  label: 'Sign Up Date'
  type: 'date'
}

Example:

Model users {
  type: 'table'
  label: 'Users'
  description: ''
  data_source_name: 'demodb'
  owner: '[email protected]'
  table_name: '"ecommerce"."users"'

  dimension id {
    label: 'Id'
    type: 'number'
  }
  dimension sign_up_date {
    label: 'Sign Up Date'
    type: 'date'
  }
  dimension sign_up_at {
    label: 'Sign Up At'
    type: 'datetime'
  }
  dimension first_name {
    label: 'First Name'
    type: 'text'
  }
  dimension last_name {
    label: 'Last Name'
    type: 'text'
  }
}

Query Model​

To centralize SQL logic and data transformations from multiple tables or models into a single, reusable model, enabling the reusability of business logic across reports and analyses. This helps reduce duplicate SQL queries and improve query performance.

Step 1: Import all the files that store the models you want to include in your model's query. Please skip this step if you are using AML 2.0

import 'path/to/model_a_file' {model_a}
import 'path/to/model_a_file' {model_a}

Step 2: Add a model object in the model file and name it

Model location{ }

Then inside the curly brackets { } of model object

Step 3: Define model's metadata

type: 'query'
label: 'Location'
description: ''
data_source_name: 'demodb'
owner: '[email protected]'

Step 4: Insert model name which will be then used inside the query

models: [
  ecommerce_cities,
  ecommerce_countries
]

Step 5: define the query of the model

//These queries select data from two tables, "ecommerce_cities" and "ecommerce_countries",
//and combine them to show the name of each city and its corresponding country.
query: @sql
      SELECT {{ #ci.name }} as city_name,
             {{ #co.name }} as country_name
      FROM {{ #ecommerce_cities as ci }}
      LEFT JOIN {{ #ecommerce_countries as co }}
      ON {{ #co.code }} = {{ #ci.country_code }}
:::

Step 6: Specify which dimensions and measures you want to include in the model.

dimension city_name {
  label: 'City Name'
  type: 'text'
}
dimension country_name {
  label: 'Country Name'
  type: 'text'
}

Example:

//Skip this step if you are using AML 2.0
import '../path/to/cities.model.aml' {cities}
import '../path/to/countries.model.aml' {countries}

Model location {
  type: 'query'
  label: 'Location'
  description: ''
  data_source_name: 'demodb'
  owner: '[email protected]'

  models: [
    ecommerce_cities,
    ecommerce_countries
  ]
  query: @sql
          SELECT {{ #ci.name }} as city_name,
              {{ #co.name }} as country_name
          FROM {{ #ecommerce_cities as ci }}
          LEFT JOIN {{ #ecommerce_countries as co }} ON {{ #co.code }} = {{ #ci.country_code }}
  ;;
  dimension city_name {
    label: 'City Name'
    type: 'text'
  }
  dimension country_name {
    label: 'Country Name'
    type: 'text'
  }
}

Develop Dataset​

info

For more information, please refer to Dataset Syntax doc

A dataset object is defined in a dataset file. You can create a dataset file by adding a new file with the extension .dataset.aml (its full form is dataset_name.dataset.aml).

A dataset component includes its metadata, models used in the dataset, and the relationship among these models.

Step 1: Import all the files that store the models you want to include in the dataset.

Skip this step if you are using AML 2.0

import 'path/to/model_a' {model_a}`

Step 2: Add dataset object in the dataset file and name it

Dataset dataset_name { }

Then inside the curly brackets { } of dataset object

Step 3: Define Dataset metadata

label: "General Dataset"
description: "Short Descritpion"
data_source_name: 'your_data_source'

Step 4: Include all the models in the dataset

models: [model_a, model_b, model_c]

Step 5: Define the relationship among those models added in step 4.

relationships: [
  // define many-to-one relationship
  relationship(model_a.field_name > model_c.field_name, true),

  //define one-to-one relationship
  relationship(model_a.field_name - model_c.field_name, true)
]

Example:

//Skip the import step if you use AML 2.0
import '../path/to/users.model.aml' { users }
import '../path/to/cities.model.aml' {cities}
import '../path/to/countries.model.aml' {countries}

Dataset demo_dataset {
  label: 'Demo Dataset'
  description: ''
  data_source_name: 'demodb'

  models: [
    ecommerce_users,
    ecommerce_countries,
    ecommerce_cities,
  ]
  relationships: [
    relationship(users.city_id > cities.id, true),
    relationship(cities.country_code - countries.code, true)
 ]
}

Save changes and Deploy to Production​

Save changes​

During your development workflow, please remember to Save changes upon making changes to your data models and dataset files.

Deploy to Production​

Finally, to have your prepared data available for exploration and insights in Reporting tab, follow these steps:

Step 1: Choose "Commit changes & Push"

Step 2: Write a commit message, then press the "Commit and push" button

Step 3: Click "Deploy to Production"

info

If you have not connected to an external Git repository, you can skip step 1 and 2. You only need to Save changes and then select Deploy to Production. Learn how to connect to an external Git repository here

Explore the dataset in Reporting tab​

After your dataset has been successfully deployed to our Reporting tab, you can now begin exploring and analyzing your data.

To learn about what you can do further with your dataset, refer to:

Let us know what you think about this document :)