Skip to content

Operational Log Configuration

Operational logs can be configured to be stored in cloud storage such as S3 or ADLS Gen2.

Configuration Settings

The following configuration settings are used to configure operational log storage:

  • WATCHER_AUDIT_LOG_DST_DIR
  • WATCHER_LOG_DST_DIR
  • WATCHER_S3_REGION
  • WATCHER_S3_ENCRYPT
  • WATCHER_AUDIT_UPLOAD_INTERVAL_SEC

Configuring Operational Log Storage on S3

To store operational logs on S3, specify an S3 path where these logs will be stored. This location should be in the Okera S3 storage you created in the Prerequisites section.

An example configuration will have:

WATCHER_LOG_DST_DIR: "s3://company/okera/ops/"
WATCHER_AUDIT_LOG_DST_DIR: "s3://company/okera/audit/"
WATCHER_S3_REGION: "us-east-1"
WATCHER_S3_ENCRYPT: "true"
WATCHER_AUDIT_UPLOAD_INTERVAL_SEC: "15"

Configuring Operational Log Storage on ADLS Gen2

To store operational logs on ADLS Gen2, specify an ADLS path where the logs will be stored. This location should be in the Okera ADLS storage you created in the Prerequisites section.

An example configuration will have:

WATCHER_LOG_DST_DIR: abfss://<file_system>@<account_name>.dfs.core.windows.net/mypath/logs

Configuring Operational Log Storage on Google Cloud Storage

To store operational logs on Google Cloud Storage (GCS), specify a GCS path where these logs will be stored. This location should be in the Okera Google Cloud Storage you created in the Prerequisites section.

An example configuration will have:

WATCHER_LOG_DST_DIR: gs://okera-config/gcp.demo.okera.com/logs

Configuring Upload Intervals

By default, Okera will upload operational logs every 5 minutes, but this can be modified by setting the WATCHER_LOG_UPLOAD_SEC value to a different number of seconds.

For example, to configure Okera to upload the operational logs every 15 seconds:

WATCHER_LOG_UPLOAD_SEC: 15