Skip to content

Okera Version 2.15 Release Notes

This topic provides Release Notes for all 2.15 versions of Okera.

2.15.3 (4/26/2023)

Bug Fixes and Improvements

  • Fixes NPE encountered when a user executes a query with a compound IN/AND statement on decimal typed columns.

  • Added support for Okera built-in functions in internal view definitions. Updated SQL statement evaluation for the internal views. Fixed logic for processing subqueries with WHERE clauses.

2.15.2 (2/28/2023)

Bug Fixes and Improvements

  • Role Template part of the Create Role UI is now scrollable.

  • Added ability to collapse the editor in the workspace.

  • Added ability to preview results in a separate modal dialog.

  • Added support for loading auto-tagging configurations from Google Cloud Storage.

  • Added a Helm chart configuration option DISABLE_WORKSPACE_DOWNLOAD_BUTTON to disable query result download button from being displayed.

  • Support added for Google Cloud Storage(GS) files when running in nScale mode.

  • Fixed an issue in which user role information would be displayed multiple times. It is now displayed once for each role.

Spark 3.2.1 Support

  • Added support for Apache's release of Spark 3.2.1. Support for EMR's release of Spark 3.2.1 (EMR 6.6) to follow later.

2.15.1 (2/10/2023)

Bug Fixes and Improvements

  • Fixes an issue where Okera could not connect to Azure SQL Managed instances.

2.15.0 (1/25/2023)

Role Templates

With this release, Okera introduces role templates to assist you in creating Okera roles that map to roles used by your authentication software. Okera provides templates for data stewards, tag managers, compliance auditors, and data custodians. When you create a role using a role template, Okera automatically assigns groups and permissions for the new role based on your template specifications.

At this time, role templates can only be used by Okera system or catalog admins. All other users will see the traditional workflow when creating roles.

Note: You are not required to use an Okera role template to create roles and can still create roles manually.

For more information, see Role Templates.

UI Updates

The following general updates were made to the UI in this release:

  • The transform type AES Decryption was added to the permission builder in this release. You can now use this transformation type when you create an access condition in a permission. See Transform Data and Privacy and Security Functions.

  • All tag names are now referenced in the format <namespace>.<tag-name>. In prior releases, a colon was used sometimes instead of a period between the namespace name and the tag name. In addition, all tag drop-down menus now look the same and are grouped by namespace.

  • Error messages produced when using tag templates now reference the actual tag name.

  • Updates were made to the UI login page. The Login button is now disabled until both a username and password have been specified. In addition, error text when invalid credentials are specified now appears in white text.

  • Some cosmetic changes were made to the UI that corrected some irregular spacing and alignment on the left-side menu.

  • The UI icons were updated and improved in this release.

    Note: The Okera documentation may not fully reflect these icon changes at this time.

Unstructured Data URI UI Updates

The following updates were made to unstructured URIs in the UI in this release.

  • The list of unstructured data URIs on the Files page in the UI is now paginated, with a maximum of 15 URIs showing on a page. You can scroll through the pages using the Previous and Next buttons at the bottom of the page.

  • The detailed role information for an unstructured data URI now includes roles for which permissions were granted for any parent folders of the URI as well as any permissions granted to the catalog that apply to the URI. In past releases, the list of roles showed all roles with any permissions on the catalog. Okera now shows only roles with permissions applicable to the URI itself.

  • The tags and roles assigned to an unstructured data URI can now be removed in the UI. When a URI is assigned no roles or tags in Okera, it is no longer registered in Okera. For more information, see Delete Unstructured Data URIs.

Snowflake Policy Synchronization Updates

In this release, Okera fine-tuned its synchronization implementation to ensure users cannot see databases in a crawler to which they should not have access.

OkeraEnsemble Updates

The following changes have been made for OkeraEnsemble in this release.

  • Okera has updated how you should deploy OkeraEnsemble nScale mode support with Amazon EMR 5 and Amazon EMR 6. Differences in the two Amazon EMR versions require that OkeraEnsemble nScale mode be deployed differently, based on the version of Amazon EMR you are using.

    When deploying OkeraEnsemble nScale in an Amazon EMR 5 environment, set the core-site.xml flag called fs.s3a.s3.client.factory.impl to org.apache.hadoop.fs.s3a.OkeraS3ClientFactory. When deploying OkeraEnsemble in an Amazon EMR 6 environment, set the core-site.xml flag called fs.s3a.s3.client.factory.impl to com.okera.recordservice.hadoop.OkeraS3ClientFactory.

    For more information, see OkeraEnsemble nScale Mode Deployment in Amazon EMR Environments.

  • OkeraEnsemble now supports RSA256 as a JWT algorithm. In past releases, only RSA512 was supported, although Okera itself has always supported both RSA256 and RSA512. The algorithm type used in your environment should be set using the JWT_ALGORITHM configuration parameter.

BigQuery Updates

The following updates have been made for BigQuery connections in this release:

  • You can now inject the Okera connection query ID into BigQuery history and in the Okera audit logs. This ID can be used to correlate the BigQuery project history with the logging in Okera audit logs.

    To support this functionality, a new connection configuration parameter inject.query-id has been added. Valid values are true (enable okera ID injection) and false (do not enable okera ID injection). When enabled for a connection, the ID is injected as a comment in the Okera-generated SQL sent to the connection and appears in BigQuery history. For most connections, the default for inject.query-id is false, but for BigQuery connections, the default is true. See Inject the Okera Connection Query ID Into BigQuery History.

  • You can now register cross-project BigQuery tables from the same Okera connection. For example, using a single connection that references one BigQuery project, you can create a second Okera crawler to crawl the same connection using a second BigQuery project. This new functionality ensures that defining multiple BigQuery connections in Okera is no longer necessary, allowing Dataproc cross-project join queries to complete successfully. It also enables cross-project joins using Presto pushdown, which moves the compute actions to the BigQuery engine and away from the Okera Enforcement Fleet (workers). Finally, it reduces your BigQuery chargeback complexity because all queries get consolidated into a single Okera connection.

Databricks Internal View Support Changes

With this release you can grant access to Databricks internal views without the need to grant access to the underlying tables.

Note: Internal views in this release don't support Okera Built-In Functions. This feature will be supported in future release.

Okera SQL Updates

This release introduces the following new SQL commands in Okera SQL:

Command Description
ALTER URI '<uri>' ADD|DROP ATTRIBUTE '<namespace>'.'<tag>'['<namespace>'.'<tag>', ...] Adds or drops attributes (tags) for URIs if you are assigned to a role with the ability to assign tags to URIs (using the GRANT ALTER ON URI... command) and to Okera objects (using the GRANT ADD_ATTRIBUTE ON CATALOG TO ROLE... command).
DESCRIBE URI <uri> Returns a list of tags and metadata associated with the specified URI. For example, DESCRIBE URI 's3://okera-demo/sample/sub-folder/file.csv'; returns the private tag if it was previously assigned with the ALTER URI command.
GRANT ALTER ON URI <top_level_uri> TO ROLE <role_name> Grants a role permission to assign tags to a URI and any of the URIs contained within the URI folder. If you specify an asterisk (*) as a wildcard for the URI name, the role is granted permission to assign tags to any URI.
SHOW GRANTED URIS Returns a list of all URIs referenced in URI permissions. You can use the LIKE option to obtain a filtered subset of URIs.
SHOW TAGGED URIS Returns a list of all tagged URIs. You can use the LIKE option to obtain a filtered subset of URIs.
SHOW URIS Returns a list of all URIs referenced in Okera. This is a union of the output from the SHOW TAGGED URIS and SHOW GRANTED URIS commands. You can use the LIKE option to obtain a filtered subset of URIs.

In addition, the GRANT SELECT ON URI <uri> SQL command can now be qualified to restrict access to URIs that have been assigned (or not assigned) specified tags. This SQL command now supports the HAVING ATTRIBUTE IN (<namespace.tag>) and HAVING ATTRIBUTE NOT IN (<namespace.tag>) qualifiers. In addition, policy properties can be set for the grant (permission) using the POLICYPROPERTIES qualifier.

API Updates

A dedicated API endpoint has been added for Okera's rewrites of BigQuery, Hive, Impala, Presto, and Snowflake queries, /api/v2/query/rewrite. POST is the only method provided with this endpoint. There are three parameters (one required):

  • The query parameter is a required string parameter and specifies the SQL query to be authorized and rewritten.

  • The cteRewrite parameter is an optional boolean parameter. It indicates whether common table expressions should be used in the rewrite. Valid values are true and false. The default is false.

  • The dialect parameter is an optional string parameter that specifies the dialect for the SQL query (in double quotes). Valid values are "BIG_QUERY", "HIVE", "IMPALA", "PRESTO", and "SNOWFLAKE". The default is "PRESTO".

For information about any Okera API endpoint, see the Okera API documentation, available after you log into the Web UI by appending /api/v2-docs/api/ after the web UI port number (8083). For example: https://my.okera.installation:8083/api/v2-docs/api/.

Okera Reserved Keyword Updates

The following reserved words were added for Okera in this release. If Okera object definition names, such as tag names, use any Okera reserved words, they must be escaped using backtick characters (`). For example, phi.`date`.

  • GRANTED
  • TAGGED
  • URI
  • URIS

A complete list of Okera reserved keywords can be found in Okera Reserved Keywords.

Security Vulnerabilities (CVEs/CWEs) Addressed

Okera uses Snyk and GitHub Advanced Security for security vulnerability scanning.

Bug Fixes and Improvements

  • Fixed an issue in which null transforms were incorrectly applied to VARCHAR columns for Athena query pushdown queries.

  • Fixed a bug that occurred when using the mask_ccn privacy and security function in Athena environments.

  • Fixed a SQL parsing error on the Insights page.

  • Optimized the performance of Okera's getPartitions() API endpoint, resulting in lower latency and load on the catalog database.
  • Improved the performance of SHOW CREATE TABLE statements.
  • Fixed a bug that caused null pointer exceptions after an upgrade from Okera 2.11.x. This bug caused problems logging into the UI as a non-admin user.
  • Fixed page errors that occurred when there were conflicts creating permissions.
  • Fixed a bug where the Copy Access Token option in the UI generated a non-integer expression value, causing connections to the API to fail.
  • Fixed a bug where a crawler ignored the default schema specified for an Athena connection.
  • Fixed a bug where you could not test a connection in the connection list.
  • Updated the error text for creating tags and tag templates.
  • Fixed a bug in which the Snowflake connection sync status was marked as failed when it had, in fact, succeeded.