Okera Version 2.15 Release Notes¶
This topic provides Release Notes for all 2.15 versions of Okera.
Bug Fixes and Improvements¶
Fixes NPE encountered when a user executes a query with a compound IN/AND statement on decimal typed columns.
Added support for Okera built-in functions in internal view definitions. Updated SQL statement evaluation for the internal views. Fixed logic for processing subqueries with
Bug Fixes and Improvements¶
Role Template part of the Create Role UI is now scrollable.
Added ability to collapse the editor in the workspace.
Added ability to preview results in a separate modal dialog.
Added support for loading auto-tagging configurations from Google Cloud Storage.
Added a Helm chart configuration option
DISABLE_WORKSPACE_DOWNLOAD_BUTTONto disable query result download button from being displayed.
Support added for Google Cloud Storage(GS) files when running in nScale mode.
Fixed an issue in which user role information would be displayed multiple times. It is now displayed once for each role.
Spark 3.2.1 Support¶
- Added support for Apache's release of Spark 3.2.1. Support for EMR's release of Spark 3.2.1 (EMR 6.6) to follow later.
Bug Fixes and Improvements¶
- Fixes an issue where Okera could not connect to Azure SQL Managed instances.
With this release, Okera introduces role templates to assist you in creating Okera roles that map to roles used by your authentication software. Okera provides templates for data stewards, tag managers, compliance auditors, and data custodians. When you create a role using a role template, Okera automatically assigns groups and permissions for the new role based on your template specifications.
At this time, role templates can only be used by Okera system or catalog admins. All other users will see the traditional workflow when creating roles.
Note: You are not required to use an Okera role template to create roles and can still create roles manually.
For more information, see Role Templates.
The following general updates were made to the UI in this release:
The transform type
AES Decryptionwas added to the permission builder in this release. You can now use this transformation type when you create an access condition in a permission. See Transform Data and Privacy and Security Functions.
All tag names are now referenced in the format
<namespace>.<tag-name>. In prior releases, a colon was used sometimes instead of a period between the namespace name and the tag name. In addition, all tag drop-down menus now look the same and are grouped by namespace.
Error messages produced when using tag templates now reference the actual tag name.
Updates were made to the UI login page. The Login button is now disabled until both a username and password have been specified. In addition, error text when invalid credentials are specified now appears in white text.
Some cosmetic changes were made to the UI that corrected some irregular spacing and alignment on the left-side menu.
The UI icons were updated and improved in this release.
Note: The Okera documentation may not fully reflect these icon changes at this time.
Unstructured Data URI UI Updates¶
The following updates were made to unstructured URIs in the UI in this release.
The list of unstructured data URIs on the Files page in the UI is now paginated, with a maximum of 15 URIs showing on a page. You can scroll through the pages using the Previous and Next buttons at the bottom of the page.
The detailed role information for an unstructured data URI now includes roles for which permissions were granted for any parent folders of the URI as well as any permissions granted to the catalog that apply to the URI. In past releases, the list of roles showed all roles with any permissions on the catalog. Okera now shows only roles with permissions applicable to the URI itself.
The tags and roles assigned to an unstructured data URI can now be removed in the UI. When a URI is assigned no roles or tags in Okera, it is no longer registered in Okera. For more information, see Delete Unstructured Data URIs.
Snowflake Policy Synchronization Updates¶
In this release, Okera fine-tuned its synchronization implementation to ensure users cannot see databases in a crawler to which they should not have access.
The following changes have been made for OkeraEnsemble in this release.
Okera has updated how you should deploy OkeraEnsemble nScale mode support with Amazon EMR 5 and Amazon EMR 6. Differences in the two Amazon EMR versions require that OkeraEnsemble nScale mode be deployed differently, based on the version of Amazon EMR you are using.
When deploying OkeraEnsemble nScale in an Amazon EMR 5 environment, set the
org.apache.hadoop.fs.s3a.OkeraS3ClientFactory. When deploying OkeraEnsemble in an Amazon EMR 6 environment, set the
For more information, see OkeraEnsemble nScale Mode Deployment in Amazon EMR Environments.
OkeraEnsemble now supports
RSA256as a JWT algorithm. In past releases, only
RSA512was supported, although Okera itself has always supported both
RSA512. The algorithm type used in your environment should be set using the
The following updates have been made for BigQuery connections in this release:
You can now inject the Okera connection query ID into BigQuery history and in the Okera audit logs. This ID can be used to correlate the BigQuery project history with the logging in Okera audit logs.
To support this functionality, a new connection configuration parameter
inject.query-idhas been added. Valid values are
true(enable okera ID injection) and
false(do not enable okera ID injection). When enabled for a connection, the ID is injected as a comment in the Okera-generated SQL sent to the connection and appears in BigQuery history. For most connections, the default for
false, but for BigQuery connections, the default is
true. See Inject the Okera Connection Query ID Into BigQuery History.
You can now register cross-project BigQuery tables from the same Okera connection. For example, using a single connection that references one BigQuery project, you can create a second Okera crawler to crawl the same connection using a second BigQuery project. This new functionality ensures that defining multiple BigQuery connections in Okera is no longer necessary, allowing Dataproc cross-project join queries to complete successfully. It also enables cross-project joins using Presto pushdown, which moves the compute actions to the BigQuery engine and away from the Okera Enforcement Fleet (workers). Finally, it reduces your BigQuery chargeback complexity because all queries get consolidated into a single Okera connection.
Databricks Internal View Support Changes¶
With this release you can grant access to Databricks internal views without the need to grant access to the underlying tables.
Note: Internal views in this release don't support Okera Built-In Functions. This feature will be supported in future release.
Okera SQL Updates¶
This release introduces the following new SQL commands in Okera SQL:
||Adds or drops attributes (tags) for URIs if you are assigned to a role with the ability to assign tags to URIs (using the
||Returns a list of tags and metadata associated with the specified URI. For example,
||Grants a role permission to assign tags to a URI and any of the URIs contained within the URI folder. If you specify an asterisk (*) as a wildcard for the URI name, the role is granted permission to assign tags to any URI.|
||Returns a list of all URIs referenced in URI permissions. You can use the
||Returns a list of all tagged URIs. You can use the
||Returns a list of all URIs referenced in Okera. This is a union of the output from the
In addition, the
GRANT SELECT ON URI <uri> SQL command can now be qualified to restrict access to URIs that have been assigned (or not assigned) specified tags. This SQL command now supports the
HAVING ATTRIBUTE IN (<namespace.tag>) and
HAVING ATTRIBUTE NOT IN (<namespace.tag>) qualifiers. In addition, policy properties can be set for the grant (permission) using the
A dedicated API endpoint has been added for Okera's rewrites of BigQuery, Hive, Impala, Presto, and Snowflake queries,
POST is the only method provided with this endpoint. There are three parameters (one required):
queryparameter is a required string parameter and specifies the SQL query to be authorized and rewritten.
cteRewriteparameter is an optional boolean parameter. It indicates whether common table expressions should be used in the rewrite. Valid values are
false. The default is
dialectparameter is an optional string parameter that specifies the dialect for the SQL query (in double quotes). Valid values are
"SNOWFLAKE". The default is
For information about any Okera API endpoint, see the Okera API documentation, available after you log into the Web UI by appending
/api/v2-docs/api/ after the web UI port number (8083). For example:
Okera Reserved Keyword Updates¶
The following reserved words were added for Okera in this release. If Okera object definition names, such as tag names, use any Okera reserved words, they must be escaped using backtick characters (`). For example,
A complete list of Okera reserved keywords can be found in Okera Reserved Keywords.
Security Vulnerabilities (CVEs/CWEs) Addressed¶
- Alpine-13661 Alpine314: Alpine-13661
- CVE-2018-25032 Alpine314: Out-of-bounds Write
- CVE-2021-46828 Alpine314: Allocation of Resources Without Limits or Throttling
- CVE-2022-0778 Alpine314: Loop with Unreachable Exit Condition ('Infinite Loop')
- CVE-2022-1097 Alpine314: OpenJDK
- CVE-2022-1271 Alpine314: Improper Input Validation
- CVE-2022-2097 Alpine314: Inadequate Encryption Strength
- CVE-2022-2309 Alpine314: NULL Pointer Dereference
- CVE-2022-3510 Denial of Service (DoS)
- CVE-2022-21540 Alpine315: OpenJDK
- CVE-2022-21541 Alpine315: OpenJDK
- CVE-2022-21549 Alpine315: OpenJDK
- CVE-2022-21619 Alpine315: OpenJDK
- CVE-2022-21624 Alpine315: OpenJDK
- CVE-2022-21626 Alpine315: OpenJDK
- CVE-2022-21628 Alpine315: OpenJDK
- CVE-2022-21698 Denial of Service (DoS)
- CVE-2022-22576 Alpine314: Improper Authentication
- CVE-2022-25647 Alpine315: Deserialization of Untrusted Data
- CVE-2022-27404 Alpine314: Out-of-bounds Write
- CVE-2022-27405 Alpine314: Out-of-bounds Read
- CVE-2022-27406 Alpine314: Out-of-bounds Read
- CVE-2022-27774 Alpine314: Insufficiently Protected Credentials
- CVE-2022-27775 Alpine314: Curl
- CVE-2022-27776 Alpine314: Insufficiently Protected Credentials
- CVE-2022-27781 Alpine314: Loop with Unreachable Exit Condition ('Infinite Loop')
- CVE-2022-27782 Alpine314: Improper Certificate Validation
- CVE-2022-28391 Alpine314: BusyBox
- CVE-2022-29458 Alpine314: Out-of-bounds Read
- CVE-2022-29824 Alpine314: Integer Overflow or Wraparound
- CVE-2022-32205 Alpine314: Allocation of Resources Without Limits or Throttling
- CVE-2022-32206 Alpine314: Allocation of Resources Without Limits or Throttling
- CVE-2022-32207 Alpine314: Incorrect Default Permissions
- CVE-2022-32208 Alpine314: Out-of-bounds Write
- CVE-2022-32221 Alpine315: Curl
- CVE-2022-34169 Alpine315: Incorrect Conversion between Numeric Types
- CVE-2022-35252 Alpine314: Curl
- CVE-2022-37434 Alpine314: Out-of-bounds Write
- CVE-2022-39399 Alpine315: OpenJDK
- CVE-2022-40303 Alpine314: Integer Overflow or Wraparound
- CVE-2022-40304 Alpine314: XML External Entity (XXE) Injection
- CVE-2022-40674 Alpine314: Use After Free
- CVE-2022-41946 Information Exposure
- CVE-2022-42898 Integer Overflow or Wraparound
- CVE-2022-42915 Alpine315: Double Free
- CVE-2022-42916 Alpine315: Cleartext Transmission of Sensitive Information
- CVE-2022-43680 Alpine314: Use After Free
- CVE-2022-45061 Resource Exhaustion
Okera uses Snyk and GitHub Advanced Security for security vulnerability scanning.
Bug Fixes and Improvements¶
Fixed an issue in which null transforms were incorrectly applied to VARCHAR columns for Athena query pushdown queries.
Fixed a bug that occurred when using the
mask_ccnprivacy and security function in Athena environments.
Fixed a SQL parsing error on the Insights page.
- Optimized the performance of Okera's
getPartitions()API endpoint, resulting in lower latency and load on the catalog database.
- Improved the performance of SHOW CREATE TABLE statements.
- Fixed a bug that caused null pointer exceptions after an upgrade from Okera 2.11.x. This bug caused problems logging into the UI as a non-admin user.
- Fixed page errors that occurred when there were conflicts creating permissions.
- Fixed a bug where the Copy Access Token option in the UI generated a non-integer expression value, causing connections to the API to fail.
- Fixed a bug where a crawler ignored the default schema specified for an Athena connection.
- Fixed a bug where you could not test a connection in the connection list.
- Updated the error text for creating tags and tag templates.
- Fixed a bug in which the Snowflake connection sync status was marked as failed when it had, in fact, succeeded.