AMQL

info

AMQL is still in active development. If you're interested in trying out, please reach out to your Holistics account manager.

What is AMQL?

AMQL (Analytics Modeling & Querying Language) is an integrated set of 2 analytics-as-code languages designed by Holistics to enable data analysts to define analytics logic in code:

• AML: a declarative language used to describe data semantic model and analytics objects such as database tables, their relationship, and visualization
• AQL: a query language that leverages data semantic model defined in AML to query SQL databases in a higher abstraction manner, especially composing and reusing metric-based queries

AMQL is designed for data engineers, data analysts, analyst engineers, or anyone who works with analytics queries and want better reusability, composability, development productivity than currently possible with existing tools.

AMQL is a part of Holistics Lite offering, our open source self-service BI tool.

AMQL also comes with a VSCode extension and open source parser, type checker and interpreter.

Why we built AMQL?

We strongly believe in the vision of analytics as-code as the future of BI and analytics. However, in our opinion, there is not an adequate existing analytics-as-code language.

Firstly, we believe there is a need for a declarative language to define semantic model, similar to how Ansible and Terraform revolutionized the way infrastructure is defined in code. When we look at existing declarative languages, there are languages like XML, JSON or YAML but each of them has their own limitations. The most suitable language is LookML but it is proprietary and only works with Looker.

Secondly, we also believe there is a need for a new query language that can leverage the semantic model defined in the first language, built to be concise, expressive while also suitable for adhoc analysis. We believe SQL is too low level as a language and solution like adding templating language (like dbt’s Jinja use) on top is just a band-aid.

Thus, in order to realize this vision, we created AMQL to be a complete analytics-as-code language.

Key design principles

Declarative and query-based language design

• Similar to how Ansible and Terraform changed the way infrastructure is defined in code, we strongly believe in the superiority of declarative based language for expressing analytics semantic models
• But defining semantic models by itself is not enough, we also have the need to have a query language

Intuitive and easy to learn syntax

• AML's declarative syntax is carefully designed to be familiar with data analysts who work with JSON and YAML data before, while avoiding ambiguity that plagues languages like YAML.
• AQL's query syntax is designed to be familiar with analysts who has knowledge of SQL, while at the same time fixing the flaws of SQL such as lack of composability and extensibility

Correctness

• Holistics understands that trust in analytical logic is crucial for accurate decision-making. That's why AMQL includes a powerful type system that provides real-time feedback to analysts to help them avoid errors early.
• In the future, AMQL will also include a built-in testing system that allows analytics logic to be properly tested before deploying to production.

Developer experience

• It is 2023 and data analysts should expect their analytics logic code to have excellent developer experience like their programming language counterparts with features such as:
  • Smart syntax highlighting
  • Auto-completion and suggestion
  • Automated refactoring
  • Real time feedback
• AMQL is designed from the ground up to be able to support these features smoothly and effectively
• AML Studio and Holistics Lite’s VSCode extension provides dual code-UI mode, which not only softens the learning curve but also makes the experience more intuitive

Reusability

• Using reusable code makes it significantly easier and quicker to create new analytics logic. Additionally, defining analytics logic in reusable code simplifies maintenance and updates, ensuring that reports and dashboards more likely to remain up-to-date.
• Unlike pure data serialization formats such as XML, JSON or YAML, AMQL provides ability to define constants, functions, modules right inside the language. These mechanisms enable analysts to factor and abstract out repeated logic and reuse them across datasets and dashboards.
• Unlike SQL, AQL allows you to save query fragments for later reuse. These fragments can then be combined to create new variations of existing queries when the need arises.
• Holistics will soon provide a packaging system and a central repository that allows users to import reusable code published by others. This will enable cross-organization reusability of analytics logic.

Performance

• Ideally, the language should get out of the way of the analysts when they develop analytics logic, thus the performance of its toolchain is of utmost importance.
• AMQL internal architecture is designed to be incremental, requiring minimal amount of computation needed to handle small changes that are inherent in typical, day-to-day development process.
• At the same time, the processing performance of full projects is benchmarked and heavily optimized to minimize deployment time.

Extensibility

• AMQL's parser, type checker, and interpreter are going to be open sourced, enabling developers to create custom tools on top of them.
• Unlike LookML which only supports Looker’s object, developer can declare their own types, effectively create their own domain specific objects