Dremio Data Source Connections¶

This page describes how to connect to a Dremio data warehouse, how to query data in Dremio, and how to enable query pushdown for Dremio.

Create a Dremio Connection¶

Create a new Dremio connection from the Connections Page.
Supply values for these fields:
- Host: Your Dremio host.
- Port: Default is 31010.
- Driver file path: You need to upload the Dremio jdbc driver to your cloud storage, and link the path to the driver here. Ensure Okera has access to this file path.
- Username file: Link to your Dremio username secret file. For more info see Providing Secure Credentials.
- Password file: Link to your Dremio username secret file. For more info see Providing Secure Credentials.

Programmatic Connection Example¶

Here is a programmatic example of a Dremio connection.

CREATE DATACONNECTION dremio_connection CXNPROPERTIES
(
  'connection_type'='JDBC',
  'driver'='dremio',
  'host'='my-host.com',
  'port'='31010',
  'user'='awsps:///my-username',
  'password'='awsps:///my-password',
  'connection_properties'='{
      "driver.jar.path":"s3://path/to/dremio-jdbc-driver-12.0.0.jar", 
    }'
)

Query Data in Dremio¶

Okera's gateway connector pushes down full queries (including joins and aggregations) to Dremio, while enforcing the complete access policy, as well as audit log entries.

Note: This connector is designed for data read/SELECT queries and not INSERT operations or DDL operations on the underlying Dremio database.

Users, such as data analysts, can connect their favorite SQL tool using Okera’s ODBC/JDBC endpoint. Queries are then transparently sent to Dremio, after being authorized and audited by Okera (if the user does not have permission to access the data they are trying to access, the query will be rejected). With this connector you get the benefit of Dremio's native performance scale and Okera's complete policy and auditing capabilities.

Okera's JDBC gateway connector currently supports push-down for these BI tools:

DBeaver
Tableau
Looker
PowerBI

Note: The entire query must only access data in Dremio and in a single connection. Any queries joining other data (e.g. Amazon S3) to Dremio are not pushed down. Some predicates are still pushed down by Okera’s scan, but not the full set.

Enable Query Pushdown¶

To enable transparent query pushdown, add dremio:direct to the list of enabled engines in the OKERA_CTE_REWRITE_ENABLED_ENGINES configuration setting. Multiple engines can be enabled (in addition to the ones enabled by default) by providing them as comma-separated values in this configuration setting.

For example:

OKERA_CTE_REWRITE_ENABLED_ENGINES: dremio:direct,redshift,postgresql