RS_ARGS Options¶
Some configuration options can be specified in RS_ARGS. Some of them are described in the following table. Specify options separated by a space. When specified, each option begins with a double-dash (--
).
Option |
Values | Description |
---|---|---|
abort_on_error |
true false |
In 2.1.x and later versions, many data correctness issues fail queries, as opposed to silently ignoring them (for example, when converting data into NULL). To revert the behavior, add --abort_on_error to RS_ARGS. |
allow_nl_in_csv |
true false |
Enables CSVs with embedded newlines within records that are enclosed within the quote separator. To enable such CSVs, specify --allow_nl_in_csv=true . |
audit_request_query |
true false |
Enables or disables Spark query logging. See Enable Spark Query Logging for Databricks. |
authorize_query_client_max_sessions |
integers from 1024 or higher | Specifies the maximum number of query client sessions that Okera should maintain. The default for this parameter is 1024 client sessions. |
batch_check |
integer | Specifies the amount of cluster memory used for a query. When querying tables that have columns with very large values (e.g. 100KB), specify batch_check=64 (or another low number). |
defer_signed_urls_initial_num_tasks |
integer | Specifies the number of nScale tasks for which deferred URI signing is performed. The default is 64. Okera defers signing for either the number of tasks specified by defer_signed_urls_initial_num_tasks or for the percentage of tasks specified by defer_signed_urls_initial_percent_tasks , whichever is lower. |
defer_signed_urls_initial_percent_tasks |
percentage | Specifies the percentage of nScale tasks for which deferred signing should be performed. The default is 25%. Okera defers signing for either the number of tasks specified by defer_signed_urls_initial_num_tasks or for the percentage of tasks specified by defer_signed_urls_initial_percent_tasks , whichever is lower. |
encryption_key_path |
path | Specifies the path to the file containing the encryption and decryption key for nScale. See Configuration File Encryption Settings for nScale. There are no constraints on the size of this file (except that it cannot be empty). Okera uses SHA256 on the contents to get a 32-byte key (256-bit). |
idle_query_timeout |
integer | The time, in seconds, that a query may be idle ( no processing work is done and no updates are received from the client) before it is cancelled. If set to 0 , idle queries never expire. The query option QUERY_TIMEOUT_S overrides this setting, but, if set, --idle_query_timeout represents the maximum allowable timeout. |
idle_session_timeout |
integer | The time, in seconds, that a session may be idle for before it is closed (and all running queries cancelled). If 0, idle sessions are never expired. |
ssl_enable |
true false |
Indicates whether SSL is enabled. |
ssl_private_key |
path | Specifies the path to the SSL private key. |
ssl_server_certificate |
path | Specifies the path to the SSL certificate. |
zstd_default_compression_level |
integer | Specifies the default zstd compression level. |
For example:
RS_ARGS: --ssl_enable=true --ssl_private_key=/path/in/pod/to/key --ssl_server_certificate=/path/in/pod/to/cert --idle_session_timeout=0 --idle_query_timeout=0