Skip to content

Okera Version 2.13 Release Notes

This topic provides Release Notes for all 2.13 versions of Okera.

2.13.1 (11/15/2022)

OkeraEnsemble System Token Generation Changes in Default Mode

With this release, the system token required by OkeraEnsemble in default (non-nScale) mode is not automatically generated from a private key.

When deployed in nScale mode, the OkeraEnsemble access proxy requires a JWT token to authenticate to the Okera cluster. This token can be provided using either the JWT_PRIVATE_KEY or SYSTEM_TOKEN configuration parameters. When the JWT_PRIVATE_KEY configuration parameter is specified, the OkeraEnsemble access proxy automatically generates its own JWT token with the provided private key. When the SYSTEM_TOKEN configuration parameter is specified, it defines the location of the system JWT token file. If both are specified, the JWT_PRIVATE_KEY takes precedence and is used, by default, to generate the required JWT token..

However, when OkeraEnsemble is deployed in default (non-nScale) mode on the Okera cluster, the access proxy now defaults to using the token defined by the SYSTEM_TOKEN configuration parameter. It no longer will generate the token from a private key.

BigQuery Updates

The Okera UDFs for BigQuery environments were updated in this release. If you are running BigQuery, be sure to run the big-query SQL script provided by Okera to update the Okera UDFs in your BigQuery environment. This script is provided in the okera-release-uswest/udfs bucket.

Security Vulnerabilities (CVEs/CWEs) Addressed

Bug Fixes and Improvements

  • Fixed an out-of-resources error that occurred with the OkeraEnsemble access proxy.

2.13.0 (11/2/2022)

Snowflake Policy Synchronization Updates

The following Snowflake policy synchronization updates have been made in this release.

  • You can now set the Snowflake policy synchronization interval by individual connection, overriding the interval set for the cluster by the property POLICY_SYNC_INTERVAL configuration property. A new advanced Snowflake connection property, okera.policy_sync.interval has been added. Valid values are specified as a combination of a number and a one or two-letter code that represent the units. Valid unit codes are ns (nanoseconds), us (microseconds), ms (milliseconds), s (seconds), m (minutes), and h (hours). For example, 1h is one hour and 5000ms is 5000 milliseconds. For example, okera.policy_sync.interval=5m sets the interval to five minutes.

    Note: If the setting for POLICY_SYNC_INTERVAL is greater than the setting for okera.policy_sync.interval on a connection, the POLICY_SYNC_INTERVAL setting is used.

    In addition, if a custom synchronization interval has been set for a Snowflake connection, you can determine what interval was set by locating the okera.policy_sync.interval setting in the Advanced Properties list on the Connection Details tab for the connection.

  • Deleting a Snowflake connection now also cleans up the table permissions that were last synced, in addition to the Snowflake roles and other objects created by the connection.

  • Roles created during Snowflake/Okera policy synchronization are now granted USAGE privilege on the warehouse specified in the Snowflake connection.

  • Specifying the warehouse is now required for a Snowflake connection.

See Snowflake Data Source Connections.

Tag Templates ( Preview Feature)

As a preview feature in this release, you can now import best-practice tags from tag templates provided by Okera. Several tag templates are provided. These templates can help you determine how to classify your data and reduce the time it takes you set up your classification system.

You can elect to import all the tags in a tag template or select specific tags for import. You can rename the tags and you can select or add a different namespace into which they should be imported.

For complete information, see Import Tags From a Namespace Template.

UI Changes

The following UI changes were made in this release.

  • The appearance and use of date and time pickers changed on the Insights page and on the Permissions dialog.

  • You can now edit a connection on the details page for the connection. A new option is now available when you review the connection details page for a connection. Select this option to view the Edit connection dialog.

  • Added a option on the connection details page that allows you to view and copy the SQL statements used to create an individual connection.

  • The names of the following screen buttons changed:

    • The Create new database button changed to Create database.

    • The Create new permission button changed to Create permission.

    • The Register selected (n) button changed to Register selected datasets.

    • The Create new role button changed to Create role.

    • The Create new tag button changed to Create tag.

  • The names of the following dialog buttons changed:

    • The Add permission button changed to Create.

    • The Register selected button changed to Register.

    • The Add button (on a tags dialog) changed to Create.

  • The following dialog titles have changed:

    • The Create new database dialog title changed to Create database.

    • The Add permission to database dialog title changed to Create permission.

    • The Add permission to dataset dialog title changed to Create permission.

    • The Add permission to dataset dialog title changed to Create permission.

    • The Create new tag dialog title changed to Create tag.

Okera Test Drive (Trial) Updates

Okera Test Drive (trial) tenants now connect to a trial-specific Snowflake account and perform policy synchronization with a service user and role, separate from the admin user and role.

Updates for BigQuery Environments

In this release Okera updated all variants of its tokenize and (n)fp_random UDFs so they can be successfully used with BigQuery data. Only the data types of TIME and BYTE are not currently supported. Referential integrity with BigQuery pushdown is also now maintained where applicable. See Tokenization.

Support for Referencing S3 Objects in Okera Configuration Parameters in nScale EMR Deployments

With this release, you can now reference objects stored in S3 as Okera configuration parameters for odas-emr-bootstrap. Okera pulls the objects referenced in the configuration parameters and mounts them in the nScale container, making the S3 paths available to Okera for processing. For example, this is helpful when configuring the SSL certificate and key required to start the OkeraEnsemble EMR access proxy in TLS/SSL mode:

--external-objects-to-container SSL_CERTIFICATE_FILE=s3://bucket/certificate-object, SSL_KEY_FILE=s3://bucket/key-object

If SSL_CERTIFICATE_FILE specifies the path to the SSL certificates file in S3 and SSL_KEY_FILE specifies the path to the SSL key file in S3, these paths can be used by the OkeraEnsemble access proxy for any necessary TLS/SSL processing.

API Updates

  • A new endpoint /api/v2/tags/<tagName>/datasets-tagged has been introduced that returns the number of unique datasets assigned the tag.

  • A new CREATE method is added to the api/v2/tags endpoint. This method allows you to create a tag using the API.

  • A new DELETE method is added to the api/v2/tags endpoint. This method allows you to delete a tag using the API.

For information about any Okera API endpoint, see the Okera API documentation, available after you log into the Web UI by appending /api/v2-docs/api/ after the web UI port number (8083). For example: https://my.okera.installation:8083/api/v2-docs/api/.

Security Vulnerabilities (CVEs/CWEs) Addressed

Bug Fixes

The following bugs were fixed in this release:

  • Updated the Okera Spark3 connector to address an incompatibility resulting from a Databricks library change in Okera-supported Databricks runtime versions 9.1 LTS and later.
  • Fixed a bug where tagging rules could not be deleted.
  • Fixed an issue where some sensitive environment variables were not always redacted in logs.
  • You no longer need to specify the transport protocol (http:// or https://) in the bootstrap script option --rest-server-hostports of odas-emr-bootstrap.