2. Okera Install - Setting Up the Deployment Manager

These steps configure the DeploymentManager which can manage all of your Okera clusters. This set of steps is typically required once, to configure Okera to your environment.

Acquire and Extract the DM Tarball

Follow these steps to get the Deployment Manager tarball on the DM host.

  1. Download and extract the Deployment Manager tarball.
    • The recommended download location is /opt/okera, but it can be downloaded to anywhere.
    sudo mkdir -p /opt/okera && cd /opt/okera
  2. Update ownership of the destination directory.

    echo `whoami` | xargs -I '{}' sudo chown -R '{}' /opt/okera
  3. Get the tarball from S3.

    curl -O https://s3.amazonaws.com/okera-release-useast/1.5.0/deployment-manager-1.5.0.tar.gz
  4. Extract the bits.

    tar xzf deployment-manager-1.5.0.tar.gz && rm deployment-manager-1.5.0.tar.gz && ln -sfn deployment-manager-1.5.0 deployment-manager

Acquire the Okera CLI

Perform this for any machine on which you want to run the Okera CLI tool. Download the shell binary for either Linux or OS X (depending on which OS is running the CLI).

  • Linux
 curl -O https://s3.amazonaws.com/okera-release-useast/1.5.0/cli/linux/ocadm && chmod +x ocadm
  • OS X
 curl -O https://s3.amazonaws.com/okera-release-useast/1.5.0/cli/darwin/ocadm && chmod +x ocadm

Establish DM Logging and Install Paths

Configure the logging and local install directories. These directories should be paths on the local file system. The install directory should be restored if this machine is moved. By default, the directories are /var/log/okera and /etc/okera. The directories can be changed by setting DEPLOYMENT_MANAGER_LOG_DIR and DEPLOYMENT_MANAGER_INSTALL_DIR in the environment.

Example: Creating the Logging and Install Directories

sudo mkdir -p /var/log/okera
sudo mkdir -p /etc/okera
sudo chmod 700 /etc/okera

The Deployment Manager user needs exclusive access to those directories. If the directories are created with a different user than the Deployment Manager user, run:

echo `whoami` | xargs -I '{}' sudo chown -R '{}' /var/log/okera
echo `whoami` | xargs -I '{}' sudo chown -R '{}' /etc/okera

Configure the Deployment Manager

You must configure the Deployment Manager before it can be run. The configuration is done by using environment variables and before starting up the server. It is recommended that you copy the template configuration and update it. The steps below assume that the standard install paths were used.

  1. Copy the Deployment Manager environment.

    cp /opt/okera/deployment-manager/conf/env-template.sh /etc/okera/env.sh
  2. Open and edit env.sh, modifying it as necessary.

The config script from /etc/okera/env.sh is automatically loaded when starting the Deployment Manager. If this is the path used, it is not necessary to source the script. If any other path is used, env.sh must be sourced before starting the Deployment Manager.

  1. Start (or restart) the deployment manager by running:

DM Configuration Options

Below are common available Okera configuration options. They can be set as environment variables or in /etc/okera/env.sh, as explained above. They are loaded when the Deployment Manager starts.

For additional Deployment Manager configuration options, see Advanced Installation: DM Configs.


This directory is the OKERA_S3_STAGING_DIR for logs and install files. It can exist anywhere in S3 that makes sense for your organization. Okera creates in this directory a number of required subdirectories.

Example: Setting the S3 staging directory

export OKERA_S3_STAGING_DIR=s3://<your_okera_dir>

Relational Database Credentials

The settings OKERA_DB_URL, OKERA_DB_USERNAME, and OKERA_DB_PASSWORD are the end point and credentials for the DM’s RDB backend. The OKERA_DB_URL should be host:port (typically port 3306) of the running MySQL/Aurora instance.

Example: Setting the database location and credentials

export OKERA_DB_URL=okera.xyzzzz.rds.amazonaws.com:3306
export OKERA_DB_USERNAME=okera
export OKERA_DB_PASSWORD=<password>


ODAS clusters by default start up with one user which has admin on the system. The admin users can create and manage roles, grant roles to other users and read all datasets. This default user is the system user - the user that ODAS services use to authenticate itself and depends on how authentication was configured.

With Kerberos, the admin user is the first part of the kerberos principal.

When using JSON Web Tokens (JST), the admin user is the subject in the OKERA_SYSTEM_TOKEN.

With no authentication, the admin user is ‘root’. To specify other admin users, edit the comma-separated list of admin users and groups.

Example: Setting the list of admin users and groups.

export OKERA_CATALOG_ADMINS=admin,username1,admin-group

Admins users can grant permissions to other users/groups including the ability to grant to other users. However, only admin users can create new roles.


This configuration specifies the ports on which cluster services run. These ports must be available for all users connecting to Okera for metadata and data. The ports are contained in a comma-separated list of service:portname:port number. If the ports are not specified, Okera randomly selects available ports on which to expose these services.

Below are the ports that are required for clients.

  • cerebro_planner:planner
  • cerebro_worker:worker
  • cdas_rest_server:api
  • cdas_rest_server:webui
  • cerebro_catalog:hms
  • cerebro_catalog:sentry
  • kubernetes_dashboard:admin_ui
  • grafana:ui

Tip: There are also two variables that can be used to set the built-in web UIs of the Planner and Worker processes. These are cerebro_planner:webui and cerebro_worker:webui respectively. They are usually only needed for cluster administrators.

An example configuration (showing the default ports):

Kerberos Configuration

To enable Kerberos, specify these configurations:


    Principal for non-REST Okera services.


    Principal for REST Okera services. By convention, this configuration should start with HTTP/ and is highly recommend for maintaining compatibility with existing tools.


    Path to the keytab file containing both principals. This path should be accessible from the Deployment Manager, either on the local file system or in S3.


    List of comma-separated cross realms from which to accept connections. This configuration does not need to be specified if only connections from the OKERA_KERBEROS_PRINCIPAL realm are allowed to connect.

Example: Setting up Kerberos principal variables

export OKERA_KERBEROS_PRINCIPAL="okera/cname.example.com@REALM.com"
export OKERA_KERBEROS_HTTP_PRINCIPAL="HTTP/cname.example.com@REALM.com"
export OKERA_KERBEROS_KEYTAB_FILE="/etc/okera.keytab"

For more information on how to set up a Kerberized cluster, see Kerberos Cluster Setup.

JSON Web Tokens (JWT) Configuration

To enable authentication using JSON Web Tokens (JWT), specify these configurations:


    Path to the public key to decrypt tokens. It must be accessible on the Deployment Manager machine, but can be on the local machine or in S3.


    Algorithm used to decrypt tokens. Must be set to RSA256 or RSA512.


    URL used to authenticate JWT tokens. A POST call is issued to this URL, specifying the token to authenticate. This cannot be set if the OKERA_JWT_PUBLIC_KEY is set.


    Path to a file that contains the token (on a single line) that Okera services uses to authenticate itself. The subject for this token acts as the Okera system user.

Example: Setting the JWT token variables with a local path

export OKERA_JWT_PUBLIC_KEY="/etc/jwt.512.pub"

Example: Setting the JWT token variables for a REST call and a token to pass

export OKERA_JWT_AUTHENTICATION_SERVER_URL="http://sso/verify-token"
export OKERA_SYSTEM_TOKEN="/etc/okera.token"

For more about authentication with JWTs, see ODAS Authentication.

Required Group Membership

If needed, it is possible to configure a required group for all non-admin ODAS users. We anticipate that most installations will not need this.

System Token Rotation

If you use a system token to authenticate communication between Okera services, it’s good practice to rotate the token in case the ODAS cluster remains in service beyond the token’s expiration date. Use the OKERA_SYSTEM_TOKEN_REFRESH_PATH property to specify an S3 path from which ODAS services can read the token. The default refresh interval is once an hour, which you can modify via the OKERA_SYSTEM_TOKEN_REFRESH_INTERVAL_MINUTES property. The token located at the S3 path specified by OKERA_SYSTEM_TOKEN_REFRESH_PATH will be used to overwrite the active token each time the refresh process executes. If the OKERA_SYSTEM_TOKEN_REFRESH_PATH property is configured, the OKERA_AWS_DEFAULT_REGION property must also be configured. The process for updating the token located at OKERA_SYSTEM_TOKEN_REFRESH_PATH is left for ODAS administrators to design and implement.

HTTPS Configuration

To enable HTTPS, specify the SSL certificate and private key that Okera uses. Setting the certificate and key enables HTTPS on the REST server and Web UI.

Note: Enabling HTTPS also has an effect on the optional Kubernetes Dashboard. It usually uses a self-signed certificate, but with SSL enabled, the specified SSL certificate is used instead.


    Path to the certificate file.


    Path to the file with the private key.

  • OKERA_SSL_FQDN (optional)

    Fully qualified domain name for the cluster. If set, this must be a valid “Subject Alternate Name” in the certificate. This can be the fully qualified domain name for any node in this cluster (typically the CNAME for the REST service). This is required for some clients, which often do not allow navigating IP addresses with SSL is enabled, such as newer versions of Chrome.

Example: Setting the HTTPS environment variables

export OKERA_SSL_CERTIFICATE_FILE=/etc/okera.cert
export OKERA_SSL_KEY_FILE=/etc/okera.key
export OKERA_SSL_FQDN=rest-server.okera.com
LDAP Configuration

For information on how to set LDAP Basic Auth related environment variables, see LDAP Basic Auth.

For more details on interactions with authenticated Okera, see:

OAuth Configuration

Okera can be deployed with OAuth enabled for easier Web UI login. For more information, see

The Okera Web Portal can be configured to use shared cookies for authentication. For this to work, Okera needs to be configured with the shared cookie name and the shared cookie domain. The following environment variables need to be set either on the Deployment Manager or on the cdas_rest_server service.


    The name of the cookie that contains the login token.


    The full name of the (sub)domain on which OKERA_SHARED_COOKIE_NAME is shared. This value will be passed verbatim to the browser’s cookie domain.

Example: Setting the Shared Cookie values

export OKERA_SHARED_COOKIE_NAME=my_company_token
export OKERA_SHARED_COOKIE_DOMAIN=subdomain1.example.com

Given the above example, the Okera Web Portal will attempt to read the cookie named my_company_token for a value assumed to be the token. The Okera Web Portal must be hosted on subdomain1.example.com or a subdomain of it in order to read that value. Additionally, if the Okera Web Portal generates a new token, it will write it to the my_company_token cookie, sharing it across subdomain1.example.com.

Start the Deployment Manager

After setting the environment variables and sourcing them, start the Deployment Manager.

Example: Deployment Manager startup


This should output System up and running and the server hostport.

Note: This step is not expected to take more than 30 seconds. It usually indicates a configuration issue if the Deployment Manager does not come up in a reasonable amount of time.

The Deployment Manager runs as a daemon Its logs can be viewed by entering:

less /var/log/okera/deployment-manager.log

Any configuration issues should be visible at the end of the log.

Deployment Manager Config Validator

It is possible to run the Deployment Manager config validation without restarting the Deployment Manager process. This is convenient to test new configs or to quickly diagnose problematic configs.

To run the Deployment Manager config validation without restarting the Deployment Manager, run the deployment-manager script with the verify-configs-only argument.

Example: Validating a DM config without starting the DM

/opt/okera/deployment-manager/bin/deployment-manager verify-configs-only
  Config validation successful.

Configure ocadm

Configure ocadm to connect with a running Deployment Manager.

Run the following:

/opt/okera/ocadm configure --server <DM_host:port>
/opt/okera/ocadm status

Running the status command should return the result: Service available. Status: Running.


Now you’re ready to create a cluster.