Deploy Okera on AKS

This document will guide you through installing Okera on an AKS cluster. You will walk through the following steps:

1. Verifying you have access to your AKS cluster.
2. Downloading and unpacking okctl.
3. Deploying a base unconfigured Okera cluster.
4. Updating the configuration of your cluster.

Prerequisites

1. A functional AKS cluster that adheres to the Okera prerequisites.
2. A Linux node that can execute kubectl commands against the AKS cluster - we will call this the deployer node.

Verify AKS Access

On the deployer node, you can verify you have AKS access by running:

$ kubectl get nodes -owide
NAME                       STATUS   ROLES   AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-agentpool-35993971-0   Ready    agent   13d   v1.13.9   10.240.0.35   <none>        Ubuntu 16.04.6 LTS   4.15.0-1052-azure   docker://3.0.6
aks-agentpool-35993971-1   Ready    agent   13d   v1.13.9   10.240.0.4    <none>        Ubuntu 16.04.6 LTS   4.15.0-1052-azure   docker://3.0.6
aks-agentpool-35993971-2   Ready    agent   13d   v1.13.9   10.240.0.66   <none>        Ubuntu 16.04.6 LTS   4.15.0-1052-azure   docker://3.0.6

You should see all the nodes that you have in your AKS cluster listed.

Choosing a Region

Okera provides the installation files in three locations, US West, US East, and EU West. You should modify the links shown below choosing the region closest to you by replacing the S3 base URL, while leaving the rest of the URL path as given in the examples on this page.

The base links for the available regions are:

Region	Base URL
US West	https://okera-release-uswest.s3-us-west-2.amazonaws.com
US East	https://okera-release-useast.s3.amazonaws.com
EU West	https://okera-release-euwest.s3.eu-west-2.amazonaws.com
Asia Pacific (Sydney)	https://okera-release-apsoutheast.s3.apsoutheast-2.amazonaws.com

For example, for the download link of the Okera command-line tool (explained in the next section) in the US East region, combine the S3 base URL for the region with the path of the installation resource, including the Okera version number:

https://okera-release-useast.s3.amazonaws.com/2.10.0/gravity/okctl.tar

|---------------- Base URL -----------------||---- Resource Path ----|

Download okctl

On the deployer node, we will be downloading okctl and the Okera Kubernetes resource files. Run the following commands to create a directory and then download and unpack okctl in it:

$ mkdir okera && cd okera
$ curl -O https://okera-release-uswest.s3.amazonaws.com/2.10.0/gravity/okctl.tar
$ tar xvf okctl.tar

We can verify we have a functional okctl binary by running:

$ ./okctl version
2.1.0
$ ./okctl --help
usage: okctl [<flags>] <command> [<args> ...]

the Okera Controller

Flags:
    --help                   Show context-sensitive help (also try --help-long and --help-man).
-v, --debug                  Show debug output (WARNING: this will output sensitive information to your shell)
    --kubeconfig=KUBECONFIG  Kubernetes config to use
    --cluster=CLUSTER        Kubernetes cluster to use as defined in KUBECONFIG
    --context=CONTEXT        Kubernetes context to use as defined in KUBECONFIG
    --namespace=NAMESPACE    Kubernetes namespace to use
-o, --output=OUTPUT          Format to print output in
    --auth-dir=".auth"       Location for auth related files

Commands:
help [<command>...]
    Show help.

...

Below instructions will assume that you are executing commands from within the folder we created above.

Deploying Okera

To deploy Okera, run:

$ ./okctl deploy --arch azure
2019/08/26 16:26:08 Deploying 'resources/kubernetes/azure/00-roles.yaml'...
2019/08/26 16:26:09 Deployed successfully
2019/08/26 16:26:09 Deploying 'resources/kubernetes/azure/01-secrets.yaml'...
2019/08/26 16:26:10 Deployed successfully
2019/08/26 16:26:10 Deploying 'resources/kubernetes/azure/02-config.yaml'...
2019/08/26 16:26:11 Deployed successfully
2019/08/26 16:26:11 Deploying 'resources/kubernetes/azure/03-services.yaml'...
2019/08/26 16:26:12 Deployed successfully
2019/08/26 16:26:12 Deploying 'resources/kubernetes/azure/04-canary.yaml'...
2019/08/26 16:26:13 Deployed successfully
2019/08/26 16:26:13 Deploying 'resources/kubernetes/azure/05-zookeeper.yaml'...
2019/08/26 16:26:14 Deployed successfully
2019/08/26 16:26:14 Deploying 'resources/kubernetes/azure/06-catalog.yaml'...
2019/08/26 16:26:15 Deployed successfully
2019/08/26 16:26:15 Deploying 'resources/kubernetes/azure/07-planner.yaml'...
2019/08/26 16:26:16 Deployed successfully
2019/08/26 16:26:16 Deploying 'resources/kubernetes/azure/08-worker.yaml'...
2019/08/26 16:26:17 Deployed successfully
2019/08/26 16:26:17 Deploying 'resources/kubernetes/azure/09-rest-server.yaml'...
2019/08/26 16:26:17 Deployed successfully
2019/08/26 16:26:17 Deploying 'resources/kubernetes/azure/10-presto.yaml'...
2019/08/26 16:26:18 Deployed successfully

This deploys a base version of Okera on your AKS cluster that is not configured - no authentication is set up and all state is being persisted locally.

Notes: By default, Okera clusters are accessible only within your configured Virtual Network/Subnet. To change this, edit resources/kubernetes/aks/03-services.yaml and set service.beta.kubernetes.io/azure-load-balancer-internal to "false".

The files in resources/kubernetes/azure are self-contained Kubernetes resource files. You can deploy them yourself using your preferred Kubernetes tooling, including using kubectl apply -f.

The cerebro-planner, cerebro-worker, cdas-rest-server and presto-coordinator services use only LoadBalancer types (type: LoadBalancer). Consequently Helm Charts must also use LoadBalancer types. The Helm Chart settings can be specified in the values.yaml file associated with the chart. The Okera settings are in the service yaml file: resources/kubernetes/aks/03-services.yaml.

Configuring Your Okera Cluster

Okera clusters use a YAML configuration file, which you can learn more about here.

You can update the configuration of your cluster using okctl update.

To deploy the Quickstart configuration, which will add authentication and SSL:

$ ./okctl update --config configs/config-quickstart.yaml

You can copy this file (or the other example configuration files in the configs/ directory) and modify them for your deployment, and then apply them using okctl update.

Configuring Azure Credentials

By default, your Okera cluster will not be able to access any ADLS resources as it does not have your Azure AD Application credentials.

To add these credentials, add the following configuration values to the config section of your configuration file:

ADLS_CLIENT_ID: <ADLS Client ID>
ADLS_CLIENT_PASSWORD: <ADLS Client Password>
ADLS_REFRESH_URL: <ADLS Refresh URL>
ADLS_TENANT_ID: <ADLS Tenant ID>

Note: These are the values of the Azure AD Application you created in the Prerequisites section.