Skip to content

Dremio

This document provides the specific information relevant to creating Dremio connections. To understand the overall connection and registration flow see Connections Overview

Create a Dremio connection

Create a new Dremio connection from the Connections Page.

Dremio connection example

You will need to input these properties:

  • Host: Your Dremio host.
  • Port: Default is 31010.
  • Driver file path: You need to upload the Dremio jdbc driver to your cloud storage, and link the path to the driver here. Ensure Okera has access to this file path.
  • Username file: Link to your Dremio username secret file. For more info see Providing Secure Credentials.
  • Password file: Link to your Dremio username secret file. For more info see Providing Secure Credentials.

Programmatic Example

CREATE DATACONNECTION dremio_connection CXNPROPERTIES
(
  'connection_type'='JDBC',
  'driver'='dremio',
  'host'='my-host.com',
  'port'='31010',
  'user'='awsps:///my-username',
  'password'='awsps:///my-password',
  'connection_properties'='{
      "driver.jar.path":"s3://path/to/dremio-jdbc-driver-12.0.0.jar", 
    }'
)

Querying data in Dremio

Okera's Gateway connector pushes down full queries (including joins and aggregations) to Dremio, while enforcing the complete access policy, as well as audit log entries. Note this connector is designed for data read/SELECT queries and not INSERT operations or DDL operations on the underlying Dremio database.

Users, such as data analysts, can connect their favorite SQL tool (e.g. DBeaver, Tableau, Looker) via Okera’s ODBC/JDBC endpoint, and their queries will be transparently sent to Dremio, after being authorized and audited by Okera (and if the user does not have permission to access the data they are trying to access, the query will be rejected). With this connector you get the benefit of Dremio's native performance scale and Okera's complete policy and auditing capabilities.

Okera's JDBC gateway connector currently supports push-down for these BI tools:

  • DBeaver
  • Tableau
  • Looker
  • PowerBI

Note

The entire query must only access data in Dremio and in a single connection. Any queries joining other data (e.g. S3) to Dremio will not be pushed down. Some predicates will still be pushed down by Okera’s scan, but not the full set.

Enabling Query Pushdown

To enable the transparent query pushdown, add dremio:direct to the list of enabled engines in the OKERA_CTE_REWRITE_ENABLED_ENGINES configuration setting, for example:

OKERA_CTE_REWRITE_ENABLED_ENGINES: dremio:direct

Note

Multiple engines can be enabled (beyond the ones enabled by default) by providing them as comma-separated values in the OKERA_CTE_REWRITE_ENABLED_ENGINES configuration setting.