Operational Log Storage

Operational logs can be configured to be stored in cloud storage such as S3 or ADLS Gen1.

Configuration Settings

The following configuration settings are used to configure operational log storage:

  • WATCHER_LOG_DST_DIR
  • WATCHER_S3_REGION
  • WATCHER_S3_ENCRYPT
  • WATCHER_LOG_UPLOAD_SEC

Configuring Operational Log Storage on S3

To store operational logs on S3, you will need to have an S3 path where these logs will be stored. This should be done in the ODAS S3 Storage you created in the Prerequisites section.

An example configuration will have:

WATCHER_LOG_DST_DIR: s3://company/okera/logs
WATCHER_S3_REGION: us-east-1
WATCHER_S3_ENCRYPT: true

Configuring Operational Log Storage on ADLS Gen1

To store operational logs on ADLS Gen1, you will need to have an ADLS path where these logs will be stored. This should be done in the ODAS ADLS Storage you created in the Prerequisites section.

An example configuration will have:

WATCHER_LOG_DST_DIR: adl://<company>.azuredatalakestorage.net/odas/logs

Configuring Upload Intervals

By default, ODAS will upload operational logs every 5 minutes, but this can be modified by setting the WATCHER_LOG_UPLOAD_SEC value to a different number of seconds.

For example, to configure ODAS to upload the operational logs every 15 seconds:

WATCHER_LOG_UPLOAD_SEC: 15