Holistics Docs - End-to-End Business Intelligence Platform

The Holistics Documentation

Welcome to the Holistics Documentation. You'll find comprehensive guides and documentation to help you start working with Holistics as quickly as possible, as well as support if you get stuck. Let's jump right in!

Documentation    Get Started

Connect Database

Suggest Edits

Do You Have Data Warehouse

I have Data Warehouse

You can connect directly to Holistics

I don't have Data Warehouse

If you don't have your own Data Warehouse, there are 2 approaches:

  1. Connect Directly to your production database
  2. Spin up a Data Warehouse on Google Cloud by following the tutorial below https://www.holistics.io/guides/create-cloud-db/

Connect to Local Database

Sorry, we don't support local database at this moment.

Connect to your Database

To get started with Holistics, first you need to connect your database to Holistics. Go to Data Sources page, click on "New Data Source", and fill in necessary information about your database.

0. Connection Requirements

These are the checklist of a successful connection to Holistics

  1. Allows Holistics to connect to your DB server: White-list IP addresses, or Setting Up SSH Tunnel
  2. Create database user for Holistics

1. Allow Holistics To Connect To DB

If your database is not accessible in the public network, either behind a firewall or private network, you need to consider:

  • Whitelisting Holistics IP Addresses, or
  • Setting Up SSH Tunneling

White-list IP Addresses

You can restrict access to your data from the network layer by white-listing the IP addresses of Holistics from your firewall. Doing so ensures that only specific, trusted hosts can have access to your database.

You can find the 2 IP addresses to whitelist in your Data Source Connection Form (click New Data Source to open the form).
Please consult with your sysadmin, DBA to help you with the white-listing.

Amazon RDS

If your database is hosted on Amazon AWS, but not on a VPC, you can set up a security group and white-list
Holistics' IP addresses. See the UI below (remember to change the IP address to holistics IP, with /32 as suffix). E.g:
12.12.12.12/32

Setting Up SSH Tunneling

Setting up reverse tunnel is great if you want to open a dedicated connection from your bastion server to our tunnel server.

All connections from Holistics will then be through the tunnel for a more secure data transfer. All connections to your DB will look as if it originates from your bastion server.

Allow External Connections

Some databases by default don't allow external connections to connect to the DB. If that's the case please read more on how to do it here.

2. Create Holistics Database User

You can use your existing database user, however we recommend creating a new database user account to connect to Holistics.

Please read more about how to Grant Database User Permission here.

Supported Databases and How to Connect

  • PostgreSQL
  • MySQL
  • Microsoft SQL Server
  • Amazon Redshift
  • Amazon AWS Athena Setup
  • Google BigQuery
  • CitusDB
  • Greenplum
  • VoltDB
  • PrestoDB
  • Presto on Qubole (QDS)
  • MongoDB (for data sync only)
  • Oracle
  • Pipedrive

Connection Timeout Settings

Query Timeout

Sometimes your query takes a very long time to run and consumes too much resource. With this setting, you can set up time out limit to abort any query that takes more than specificed number of milliseconds.

Just open the data source setting and enter the limit time in milliseconds:

How It Works

  • When your query reach the number of milliseconds, an error will be raised, and depend on your database the message will be
    • In PostgreSQL: ERROR: canceling statement due to statement timeout
    • In RedShift: ERROR: Query cancelled on user's request
  • Currently, Holistics only supports this feature for PostgreSQL and RedShift.

Configure DB to Allow External Connections

Some databases by default don't allow external connections to connect to the DB. If that's the case please follow the below steps:

If you're using SSH tunneling and your bastion server is the same as your DB host, then you shouldn't need the below, since the DB connection will look like a local connection.

PostgreSQL

Open postgresql.conf (usually in /etc/postgresql/9.x/main/) and enable the following:

listen_addresses = '*'

Also, add the following lines to pg_hba.conf:

# TYPE  DATABASE   USER   ADDRESS              METHOD
host    all        all    128.199.219.128/32   md5
host    all        all    188.166.254.67/32    md5

And restart your database to take effect:

sudo service postgresql restart

MySQL

Open my.conf and comment out bind-address line

# bind-address = 127.0.0.1

Then restart MySQL for the change to take effect

sudo /etc/init.d/mysql restart
# or
sudo service mysql restart

Connect Database