Okera Version 2.13 Release Notes¶
This topic provides Release Notes for all 2.13 versions of Okera.
2.13.1 (11/15/2022)¶
OkeraEnsemble System Token Generation Changes in Default Mode¶
With this release, the system token required by OkeraEnsemble in default (non-nScale) mode is not automatically generated from a private key.
When deployed in nScale mode, the OkeraEnsemble access proxy requires a JWT token to authenticate to the Okera cluster. This token can be provided using either the JWT_PRIVATE_KEY or SYSTEM_TOKEN configuration parameters. When the JWT_PRIVATE_KEY configuration parameter is specified, the OkeraEnsemble access proxy automatically generates its own JWT token with the provided private key. When the SYSTEM_TOKEN configuration parameter is specified, it defines the location of the system JWT token file. If both are specified, the JWT_PRIVATE_KEY takes precedence and is used, by default, to generate the required JWT token..
However, when OkeraEnsemble is deployed in default (non-nScale) mode on the Okera cluster, the access proxy now defaults to using the token defined by the SYSTEM_TOKEN configuration parameter. It no longer will generate the token from a private key.
BigQuery Updates¶
The Okera UDFs for BigQuery environments were updated in this release. If you are running BigQuery, be sure to run the big-query
SQL script provided by Okera to update the Okera UDFs in your BigQuery environment. This script is provided in the okera-release-uswest/udfs
bucket.
Security Vulnerabilities (CVEs/CWEs) Addressed¶
- CVE-2020-16156 Ubuntu 18.04 - Perl (Improper Verification of Cryptographic Signature)
- CVE-2021-43618 Ubuntu 18.04 - gmp (Integer Overflow or Wraparound)
- CVE-2021-46848 Alpine Curl - Out-of-bounds Read
- CVE-2022-21589 Ubuntu 18.04 - MySQL Server Vulnerability
- CVE-2022-21592 Ubuntu 18.04 - MySQL Server Vulnerability
- CVE-2022-21608 Ubuntu 18.04 - MySQL Server Vulnerability
- CVE-2022-21617 Ubuntu 18.04 - MySQL Server Vulnerability
- CVE-2022-32221 Ubuntu 18.04 - Curl
- CVE-2022-39253 Ubuntu 18.04 - git (Link Following)
- CVE-2022-39260 Ubuntu 18.04 - git (Out-of-bounds Write)
- CVE-2022-42915 Alpine Curl - Double Free
- CVE-2022-42916 Alpine Curl - Cleartext Transmission of Sensitive Information
Okera uses Snyk and GitHub Advanced Security for security vulnerability scanning.
Bug Fixes and Improvements¶
- Fixed an out-of-resources error that occurred with the OkeraEnsemble access proxy.
2.13.0 (11/2/2022)¶
Snowflake Policy Synchronization Updates¶
The following Snowflake policy synchronization updates have been made in this release.
-
You can now set the Snowflake policy synchronization interval by individual connection, overriding the interval set for the cluster by the property
POLICY_SYNC_INTERVAL
configuration property. A new advanced Snowflake connection property,okera.policy_sync.interval
has been added. Valid values are specified as a combination of a number and a one or two-letter code that represent the units. Valid unit codes arens
(nanoseconds),us
(microseconds),ms
(milliseconds),s
(seconds),m
(minutes), andh
(hours). For example,1h
is one hour and5000ms
is 5000 milliseconds. For example,okera.policy_sync.interval=5m
sets the interval to five minutes.Note: If the setting for
POLICY_SYNC_INTERVAL
is greater than the setting forokera.policy_sync.interval
on a connection, thePOLICY_SYNC_INTERVAL
setting is used.In addition, if a custom synchronization interval has been set for a Snowflake connection, you can determine what interval was set by locating the
okera.policy_sync.interval
setting in the Advanced Properties list on the Connection Details tab for the connection. -
Deleting a Snowflake connection now also cleans up the table permissions that were last synced, in addition to the Snowflake roles and other objects created by the connection.
-
Roles created during Snowflake/Okera policy synchronization are now granted USAGE privilege on the warehouse specified in the Snowflake connection.
-
Specifying the warehouse is now required for a Snowflake connection.
See Snowflake Data Source Connections.
Tag Templates ( Preview Feature)¶
As a preview feature in this release, you can now import best-practice tags from tag templates provided by Okera. Several tag templates are provided. These templates can help you determine how to classify your data and reduce the time it takes you set up your classification system.
You can elect to import all the tags in a tag template or select specific tags for import. You can rename the tags and you can select or add a different namespace into which they should be imported.
For complete information, see Import Tags From a Namespace Template.
UI Changes¶
The following UI changes were made in this release.
-
The appearance and use of date and time pickers changed on the Insights page and on the Permissions dialog.
-
You can now edit a connection on the details page for the connection. A new
option is now available when you review the connection details page for a connection. Select this option to view the Edit connection dialog.
-
Added a
option on the connection details page that allows you to view and copy the SQL statements used to create an individual connection.
-
The names of the following screen buttons changed:
-
The Create new database button changed to Create database.
-
The Create new permission button changed to Create permission.
-
The Register selected (n) button changed to Register selected datasets.
-
The Create new role button changed to Create role.
-
The Create new tag button changed to Create tag.
-
-
The names of the following dialog buttons changed:
-
The Add permission button changed to Create.
-
The Register selected button changed to Register.
-
The Add button (on a tags dialog) changed to Create.
-
-
The following dialog titles have changed:
-
The Create new database dialog title changed to Create database.
-
The Add permission to database dialog title changed to Create permission.
-
The Add permission to dataset dialog title changed to Create permission.
-
The Add permission to dataset dialog title changed to Create permission.
-
The Create new tag dialog title changed to Create tag.
-
Okera Test Drive (Trial) Updates¶
Okera Test Drive (trial) tenants now connect to a trial-specific Snowflake account and perform policy synchronization with a service user and role, separate from the admin user and role.
Updates for BigQuery Environments¶
In this release Okera updated all variants of its tokenize
and (n)fp_random
UDFs so they can be successfully used with BigQuery data. Only the data types of TIME and BYTE are not currently supported. Referential integrity with BigQuery pushdown is also now maintained where applicable. See Tokenization.
Support for Referencing Amazon S3 Objects in Okera Configuration Parameters in nScale Amazon EMR Deployments¶
With this release, you can now reference objects stored in Amazon S3 as Okera configuration parameters for odas-emr-bootstrap
. Okera pulls the objects referenced in the configuration parameters and mounts them in the nScale container, making the Amazon S3 paths available to Okera for processing. For example, this is helpful when configuring the SSL certificate and key required to start the OkeraEnsemble Amazon EMR access proxy in TLS/SSL mode:
--external-objects-to-container SSL_CERTIFICATE_FILE=s3://bucket/certificate-object, SSL_KEY_FILE=s3://bucket/key-object
If SSL_CERTIFICATE_FILE
specifies the path to the SSL certificates file in Amazon S3 and SSL_KEY_FILE
specifies the path to the SSL key file in Amazon S3, these paths can be used by the OkeraEnsemble access proxy for any necessary TLS/SSL processing.
API Updates¶
-
A new endpoint
/api/v2/tags/<tagName>/datasets-tagged
has been introduced that returns the number of unique datasets assigned the tag. -
A new
CREATE
method is added to theapi/v2/tags
endpoint. This method allows you to create a tag using the API. -
A new
DELETE
method is added to theapi/v2/tags
endpoint. This method allows you to delete a tag using the API.
For information about any Okera API endpoint, see the Okera API documentation, available after you log into the Web UI by appending /api/v2-docs/api/
after the web UI port number (8083). For example: https://my.okera.installation:8083/api/v2-docs/api/
.
Security Vulnerabilities (CVEs/CWEs) Addressed¶
- CVE-2021-4209 Null Pointer Dereference
- CVE-2022-2509 Double Free
- CVE-2022-2526 Use After Free
- CVE-2022-36944 Remote Code Execute (RCE)
- CVE-2022-37434 Out-of-Bounds Write
- CVE-2022-40664 Improper Authentication
- CVE-2022-41828 Use of Function with Inconsistent Implementations
- CVE-2022-42003 Deserialization of Untrusted Data
- CVE-2022-42004 Deserialization of Untrusted Data
Okera uses Snyk and GitHub Advanced Security for security vulnerability scanning.
Bug Fixes¶
The following bugs were fixed in this release:
- Updated the Okera Spark3 connector to address an incompatibility resulting from a Databricks library change in Okera-supported Databricks runtime versions 9.1 LTS and later.
- Fixed a bug in which the autotagger’s Row test maximum setting was not processed correctly for large tables when BigQuery was used.
- Fixed a bug where tagging rules could not be deleted.
- Fixed an issue where some sensitive environment variables were not always redacted in logs.
- You no longer need to specify the transport protocol (
http://
orhttps://
) in the bootstrap script option--rest-server-hostports
ofodas-emr-bootstrap
.
- The default for the Databricks environment variable
OKERA_PREFER_ACTIVE_SESSION_FOR_ID
is nowtrue
, so the workaround described in Problem 2: Partitioned Temporary Tables Generated Cannot Be Accessed is no longer necessary.
- This release introduces the ability to enable grants for a nonexistent database or table object. To enable this functionality, set a new configuration parameter
SKIP_OBJECT_CHECKS_ON_GRANT
totrue
. Valid values aretrue
andfalse
. The default isfalse
.