Skip to content

Deploy Okera on EC2

This document will guide you through installing Okera on EC2 machines. You will walk through the following steps:

  1. Provision an EC2 instance for Okera.
  2. Deploy a single-node base unconfigured cluster.
  3. Updating the configuration of your cluster.
  4. Joining more instances to your cluster manually.
  5. Creating a launch template and auto-scaling group to scale your cluster.

Prerequisites

  • Required: Security groups

  • Required: IAM role

  • Optional: S3 location to store logs (required for multi-node clusters)

  • Optional: RDS instance

Provisioning an EC2 Instance

In this step we will provision an EC2 instance to install our Okera cluster on. Our instance will have the following configuration:

  • Amazon Linux 2 64-bit AMI.

  • t3.2xlarge instance type.

  • Use the security group configured in Prerequisites.

  • Have the IAM role configured in Prerequisites.

  • Have 120GB EBS volume attached.

To create the instance, navigate to the EC2 Launch Instance wizard:

  1. Select Amazon Linux 2 AMI (HVM), SSD Volume Type AMI and click Select.
  2. Select the t3.2xlarge instance type and select Configure Instance Details.
  3. Select your desired VPC and Subnet (and any other normal EC2 configuration you use).
  4. Select the IAM role created for the Okera cluster as per the prerequisites section and select Add Storage.
  5. Configure the volume to be 120GB in size of type gp2 and select Add Tags.
  6. Add any tags you need for this instance and select Configure Security Group.
  7. Select the security group created for the Okera cluster (see Prerequisites) and select Review and Launch.
  8. Ensure all the settings are correct and select Launch.

Once the instance is created, SSH into the instance. The instructions below should be performed in this SSH session. All commands can be run as the default ec2-user unless they explicitly use sudo.

Select a Region

Okera provides installation files in three locations, US West, US East, and EU West. Modify the links shown below, selecting the region closest to you by replacing the S3 base URL, while leaving the rest of the URL path as shown in the examples in this topic.

The base links for the available regions are:

Region Base URL
US West https://okera-release-uswest.s3-us-west-2.amazonaws.com
US East https://okera-release-useast.s3.amazonaws.com
EU West https://okera-release-euwest.s3.eu-west-2.amazonaws.com
Asia Pacific (Sydney) https://okera-release-apsoutheast.s3.apsoutheast-2.amazonaws.com

For example, to determine the download link of the install script (explained in the next section) in the US East region, combine the S3 base URL for the region with the path of the installation resource, including the Okera version number:

https://okera-release-useast.s3.amazonaws.com/2.10.0/gravity/install.sh

|---------------- Base URL -----------------||---- Resource Path ----|

Deploy a Single-Node Unconfigured Cluster

After you have SSHed onto the machine, run the following command to download and run the Okera installer:

$ curl https://okera-release-uswest.s3.amazonaws.com/2.10.0/gravity/install.sh | sh -

The output should look similar to this:

Created /home/ec2-user/okera-2.10.0
Preparing for Okera v2.10.0 Installation...
Downloading https://okera-release-uswest.s3.amazonaws.com/2.10.0/gravity/odas.tar to /home/ec2-user/okera-2.10.0/odas.tar...
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 6539M  100 6539M    0     0  41.0M      0  0:02:39  0:02:39 --:--:-- 40.9M
Untarring /home/ec2-user/okera-2.10.0/odas.tar...
Preparing for cluster bootstrap...

...
<various validation passed or skipped messages>
...

2022/05/05 06:09:20 Created 'cluster-config.yaml'
2022/05/05 06:09:20 Created 'create.cmd'
2022/05/05 06:09:20 Created 'join.cmd'
2022/05/05 06:09:20 Created 'join.asg.cmd'
2022/05/05 06:09:20 If you are creating a new cluster, you can execute:
2022/05/05 06:09:20
2022/05/05 06:09:20   cd /home/ec2-user/okera-2.10.0 && sudo ./gravity install --token=kymPnYuvEW --advertise-addr=10.1.10.168 --cloud-provider=generic --config=cluster-config.yaml --pod-network-cidr="172.23.0.0/16" --service-cidr="172.34.0.0/16"
2022/05/05 06:09:20
2022/05/05 06:09:20 If you later want to join a new node to this cluster, you can execute the following on that node:
2022/05/05 06:09:20
2022/05/05 06:09:20   sudo ./gravity join --token=kymPnYuvEW --role=worker 10.1.10.168 --cloud-provider=generic
2022/05/05 06:09:20
2022/05/05 06:09:20 You can reference 'create.cmd', 'join.cmd' and 'join.asg.cmd' for future use

Note: The install.sh script simply downloads odas.tar from the same location, unpacks it and runs the installation commands. You can review the script commands and run them manually if you prefer.

This prepares this node to install Okera and provides instructions for adding more nodes to this cluster (which we will use in later sections).

As instructed by the output, you can now install the cluster. Note that the following command is copied from the earlier output and will look different for your installation:

$ cd /home/ec2-user/okera-2.10.0 && sudo ./gravity install --token=kymPnYuvEW \
    --advertise-addr=10.1.10.168 --cloud-provider=generic --config=cluster-config.yaml \
    --pod-network-cidr="172.23.0.0/16" --service-cidr="172.34.0.0/16"

Note: If you did not save the original command, you can retrieve it by going to the installation directory and finding it in the create.cmd file. You can also simply execute $ ./create.cmd to start the cluster creation process.

The output should be similar to this (slightly abbreviated):

Thu May  5 06:10:25 UTC Starting installer

To abort the installation and clean up the system,
press Ctrl+C two times in a row.

If the you get disconnected from the terminal, you can reconnect to the installer
agent by issuing 'gravity resume' command.

If the installation fails, use 'gravity plan' to inspect the state and
'gravity resume' to continue the operation.
See https://gravitational.com/gravity/docs/cluster/#managing-an-ongoing-operation for details.

Thu May  5 06:10:25 UTC Connecting to installer
Thu May  5 06:10:39 UTC Connected to installer
Thu May  5 06:10:40 UTC Successfully added "master" node on 10.1.10.168
Thu May  5 06:10:40 UTC All agents have connected!
Thu May  5 06:10:41 UTC Operation has been created
Thu May  5 06:10:42 UTC Executing "/checks" locally
Thu May  5 06:10:42 UTC Execute preflight checks
Thu May  5 06:10:42 UTC Running pre-flight checks
Thu May  5 06:10:45 UTC Executing "/configure" locally
Thu May  5 06:10:45 UTC Configuring cluster packages
Thu May  5 06:10:45 UTC Configure packages for all nodes
Thu May  5 06:10:49 UTC Executing "/bootstrap/ip-10-1-10-168.us-west-2.compute.internal" locally
Thu May  5 06:10:49 UTC Bootstrap master node ip-10-1-10-168.us-west-2.compute.internal
Thu May  5 06:10:50 UTC Configuring system directories
Thu May  5 06:10:52 UTC Configuring application-specific volumes
Thu May  5 06:10:54 UTC Executing "/pull/ip-10-1-10-168.us-west-2.compute.internal" locally
Thu May  5 06:10:54 UTC Pulling user application
Thu May  5 06:10:54 UTC Pull packages on master node ip-10-1-10-168.us-west-2.compute.internal
Thu May  5 06:11:04 UTC         Still pulling user application (10 seconds elapsed)
...
Thu May  5 06:11:36 UTC Pulling configured packages
Thu May  5 06:11:44 UTC Unpacking pulled packages
Thu May  5 06:11:54 UTC         Still unpacking pulled packages (10 seconds elapsed)
Thu May  5 06:12:03 UTC Install system software on master nodes
Thu May  5 06:12:04 UTC Executing "/masters/ip-10-1-10-168.us-west-2.compute.internal/teleport" locally
Thu May  5 06:12:04 UTC Installing system service teleport:3.2.7
Thu May  5 06:12:04 UTC Install system package teleport:3.2.7 on master node ip-10-1-10-168.us-west-2.compute.internal
Thu May  5 06:12:06 UTC Executing "/masters/ip-10-1-10-168.us-west-2.compute.internal/planet" locally
Thu May  5 06:12:06 UTC Install system package odas-planet:6.1.3-2.1.0-planet on master node ip-10-1-10-168.us-west-2.compute.internal
Thu May  5 06:12:06 UTC Installing system service odas-planet:6.1.3-2.1.0-planet
Thu May  5 06:12:16 UTC         Still installing system service odas-planet:6.1.3-2.1.0-planet (10 seconds elapsed)
Thu May  5 06:12:25 UTC Executing "/wait" locally
Thu May  5 06:12:25 UTC Wait for Kubernetes to become available
Thu May  5 06:12:35 UTC         Still executing "/wait" locally (10 seconds elapsed)
Thu May  5 06:12:39 UTC Executing "/rbac" locally
Thu May  5 06:12:40 UTC Creating Kubernetes RBAC resources
Thu May  5 06:12:40 UTC Bootstrap Kubernetes roles and PSPs
Thu May  5 06:12:42 UTC Executing "/coredns" locally
Thu May  5 06:12:42 UTC Configure CoreDNS
Thu May  5 06:12:43 UTC Configuring CoreDNS
Thu May  5 06:12:43 UTC Executing "/system-resources" locally
Thu May  5 06:12:43 UTC Create system Kubernetes resources
Thu May  5 06:12:44 UTC Configuring system Kubernetes resources
Thu May  5 06:12:44 UTC Executing "/user-resources" locally
Thu May  5 06:12:45 UTC Creating user-supplied Kubernetes resources
Thu May  5 06:12:45 UTC Create user-supplied Kubernetes resources
Thu May  5 06:12:46 UTC Executing "/export/ip-10-1-10-168.us-west-2.compute.internal" locally
Thu May  5 06:12:46 UTC Populate Docker registry on master node ip-10-1-10-168.us-west-2.compute.internal
Thu May  5 06:12:47 UTC Unpacking application rbac-app:6.1.4
Thu May  5 06:12:47 UTC Exporting application rbac-app:6.1.4 to local registry
Thu May  5 06:12:47 UTC Unpacking application dns-app:0.3.0
Thu May  5 06:12:48 UTC Exporting application dns-app:0.3.0 to local registry
Thu May  5 06:12:49 UTC Unpacking application bandwagon:6.0.1
Thu May  5 06:12:50 UTC Exporting application bandwagon:6.0.1 to local registry
Thu May  5 06:12:51 UTC Unpacking application logging-app:6.0.2
Thu May  5 06:12:52 UTC Exporting application logging-app:6.0.2 to local registry
Thu May  5 06:12:55 UTC Unpacking application monitoring-app:6.0.4
Thu May  5 06:12:57 UTC Exporting application monitoring-app:6.0.4 to local registry
Thu May  5 06:13:04 UTC Unpacking application tiller-app:6.0.0
Thu May  5 06:13:04 UTC Exporting application tiller-app:6.0.0 to local registry
Thu May  5 06:13:05 UTC Unpacking application site:6.1.4
Thu May  5 06:13:06 UTC Exporting application site:6.1.4 to local registry
Thu May  5 06:13:07 UTC Unpacking application odas:1.0.0-2.1.0
Thu May  5 06:13:17 UTC         Still unpacking application odas:1.0.0-2.1.0 (10 seconds elapsed)
...
Thu May  5 06:14:19 UTC Exporting application odas:1.0.0-2.1.0 to local registry
Thu May  5 06:14:29 UTC         Still exporting application odas:1.0.0-2.1.0 to local registry (10 seconds elapsed)
...    
Thu May  5 06:15:18 UTC Executing "/health" locally
Thu May  5 06:15:18 UTC Waiting for the planet to start
Thu May  5 06:15:18 UTC Wait for cluster to pass health checks
Thu May  5 06:15:19 UTC Executing "/runtime/dns-app" locally
Thu May  5 06:15:19 UTC Install system application dns-app:0.3.0
Thu May  5 06:15:20 UTC Executing install hook for dns-app:0.3.0
Thu May  5 06:15:30 UTC         Still executing install hook for dns-app:0.3.0 (10 seconds elapsed)
Thu May  5 06:15:34 UTC Executing "/runtime/logging-app" locally
Thu May  5 06:15:34 UTC Executing install hook for logging-app:6.0.2
Thu May  5 06:15:34 UTC Install system application logging-app:6.0.2
Thu May  5 06:15:40 UTC Executing "/runtime/monitoring-app" locally
Thu May  5 06:15:40 UTC Executing install hook for monitoring-app:6.0.4
Thu May  5 06:15:40 UTC Install system application monitoring-app:6.0.4
Thu May  5 06:15:50 UTC         Still executing install hook for monitoring-app:6.0.4 (10 seconds elapsed)
Thu May  5 06:15:56 UTC Executing "/runtime/tiller-app" locally
Thu May  5 06:15:56 UTC Install system application tiller-app:6.0.0
Thu May  5 06:15:57 UTC Executing install hook for tiller-app:6.0.0
Thu May  5 06:16:07 UTC         Still executing install hook for tiller-app:6.0.0 (10 seconds elapsed)
Thu May  5 06:16:11 UTC Executing "/runtime/site" locally
Thu May  5 06:16:11 UTC Executing install hook for site:6.1.4
Thu May  5 06:16:11 UTC Install system application site:6.1.4
Thu May  5 06:16:21 UTC         Still executing install hook for site:6.1.4 (10 seconds elapsed)
...
Thu May  5 06:17:40 UTC Executing postInstall hook for site:6.1.4
Thu May  5 06:17:50 UTC         Still executing postInstall hook for site:6.1.4 (10 seconds elapsed)
Thu May  5 06:18:00 UTC         Still executing postInstall hook for site:6.1.4 (20 seconds elapsed)
Thu May  5 06:18:01 UTC Executing "/runtime/kubernetes" locally
Thu May  5 06:18:01 UTC Install system application kubernetes:6.1.4
Thu May  5 06:18:02 UTC Executing "/app/odas" locally
Thu May  5 06:18:02 UTC Install application odas:1.0.0-2.1.0
Thu May  5 06:18:03 UTC Executing install hook for odas:1.0.0-2.1.0
Thu May  5 06:18:13 UTC         Still executing install hook for odas:1.0.0-2.1.0 (10 seconds elapsed)
...
Thu May  5 06:18:47 UTC Executing "/connect-installer" locally
Thu May  5 06:18:47 UTC Connect to installer
Thu May  5 06:18:48 UTC Connecting to installer
Thu May  5 06:18:50 UTC Executing "/election" locally
Thu May  5 06:18:50 UTC Enable leader elections
Thu May  5 06:18:50 UTC Enable cluster leader elections
Thu May  5 06:18:51 UTC Executing "/gravity-resources" locally
Thu May  5 06:18:51 UTC Create user-supplied Gravity resources
Thu May  5 06:18:52 UTC Creating user-supplied cluster resources
Thu May  5 06:18:53 UTC Executing operation finished in 8 minudastes
Thu May  5 06:18:54 UTC Operation has completed
Thu May  5 06:18:54 UTC Installation succeeded in 8m13.47619093s
Thu May  5 06:18:57 UTC
Cluster endpoints:
    * Authentication gateway:
        - 10.1.10.168:32009
    * Cluster management URL:
        - https://10.1.10.168:32009

Application endpoints:
    * odas:1.0.0-2.1.0:
        - Okera Presto:
            - https://10.1.10.168:14050
        - Okera Planner:
            - tcp://10.1.10.168:12050
        - Okera Worker:
            - tcp://10.1.10.168:13050
        - Okera REST:
            - http://10.1.10.168:8089
            - http://10.1.10.168:8083

Congratulations!
The cluster is up and running. Please take a look at "cluster management" section:
https://gravitational.com/gravity/docs/cluster/

When the command completes (usually around 10-15 minutes), your cluster performs its final setup steps and should be ready within two to three minutes. To verify your cluster is ready, run:

$ ./okctl status
ready

After the cluster is ready, use the okctl endpoints command to find the URL for the webui:

$ ./okctl endpoints
2022/05/05 09:51:25 cdas-rest-server:api (type: NodePort)
2022/05/05 09:51:25   10.1.10.168:8089
2022/05/05 09:51:25 cdas-rest-server:webui (type: NodePort)
2022/05/05 09:51:25   10.1.10.168:8083
2022/05/05 09:51:25 cerebro-planner:planner (type: NodePort)
2022/05/05 09:51:25   10.1.10.168:12050
2022/05/05 09:51:25 cerebro-worker:worker (type: NodePort)
2022/05/05 09:51:25   10.1.10.168:13050
2022/05/05 09:51:25 presto-coordinator:api (type: NodePort)
2022/05/05 09:51:25   10.1.10.168:14050

Open your browser at the address specified under cdas-rest-sever:webui, in this example http://10.1.10.168:8083. Since this is an unconfigured cluster, there is no authentication and you can type in root to log in to the web UI as an administrator.

Configure Your Okera Cluster

Okera clusters use a YAML configuration file. You can update the configuration of your cluster using okctl update.

To deploy the quickstart configuration, which will add authentication and SSL, run:

$ ./okctl update --config configs/config-quickstart.yaml

You can copy this file (or the other example configuration files in the configs/ directory) and modify them for your deployment, and then apply them using okctl update.

Manually Grow Your Okera Cluster

You can join more nodes to your cluster by provisioning additional EC2 instances (follow the same EC2 instructions). After the instance is created, run the following command to download the minimal installer:

$ curl -O https://okera-release-uswest.s3.amazonaws.com/2.10.0/gravity/gravity

After it is downloaded, use the command printed during the original cluster step to join the node to the cluster:

$ chmod +x gravity
$ sudo ./gravity join --token=kymPnYuvEW --role=worker 10.1.10.168 --cloud-provider=generic

Note: If you did not save the original command, you can always retrieve it by going to the installation directory and finding it in the join.cmd file.

The output should be similar to this:

Thu May  5 09:58:19 UTC Starting agent

To abort the agent and clean up the system,
press Ctrl+C two times in a row.

If the you get disconnected from the terminal, you can reconnect to the installer
agent by issuing 'gravity resume' command.
See https://gravitational.com/gravity/docs/cluster/#managing-an-ongoing-operation for details.

Thu May  5 09:58:19 UTC Connecting to agent
Thu May  5 09:58:20 UTC Connected to agent
Thu May  5 09:58:20 UTC Connecting to cluster
Thu May  5 09:58:20 UTC Connected to existing cluster at 10.1.10.168
Thu May  5 09:58:21 UTC Executing "/configure" locally
Thu May  5 09:58:21 UTC Configuring cluster packages
Thu May  5 09:58:25 UTC Executing "/bootstrap" locally
Thu May  5 09:58:26 UTC Configuring system directories
Thu May  5 09:58:27 UTC Configuring application-specific volumes
Thu May  5 09:58:27 UTC Executing "/pull" locally
Thu May  5 09:58:27 UTC Pulling user application
Thu May  5 09:58:37 UTC         Still pulling user application (10 seconds elapsed)
...
Thu May  5 09:59:26 UTC Pulling configured packages
Thu May  5 09:59:32 UTC Unpacking pulled packages
Thu May  5 09:59:42 UTC         Still unpacking pulled packages (10 seconds elapsed)
Thu May  5 09:59:50 UTC Executing "/system/teleport" locally
Thu May  5 09:59:50 UTC Installing system service teleport:3.2.7
Thu May  5 09:59:51 UTC Executing "/system/planet" locally
Thu May  5 09:59:51 UTC Installing system service odas-planet:6.1.3-2.1.0-planet
Thu May  5 10:00:01 UTC         Still installing system service odas-planet:6.1.3-2.1.0-planet (10 seconds elapsed)
Thu May  5 10:00:10 UTC Executing "/wait/planet" locally
Thu May  5 10:00:10 UTC Waiting for the planet to start
Thu May  5 10:00:20 UTC         Still waiting for the planet to start (10 seconds elapsed)
...
Thu May  5 10:00:50 UTC Executing "/wait/k8s" locally
Thu May  5 10:00:50 UTC Waiting for the Kubernetes node to register
Thu May  5 10:00:50 UTC Executing "/elect" locally
Thu May  5 10:00:50 UTC Enabling leader elections
Thu May  5 10:00:51 UTC Operation completed

Grow Your Okera Cluster Using Auto-Scaling Groups

In addition to manually scaling your cluster, you can configure an auto-scaling group to automatically scale your cluster up and down.

You will first create a launch configuration using the AWS Launch Configuration Wizard:

  1. Select the Amazon Linux 2 AMI (HVM), SSD Volume Type AMI and click Select.
  2. Select the t3.2xlarge instance type and se;ect Configure Details.
  3. Give your launch configuration a name (e.g. okera-launch-config).
  4. Expand Advanced Details, and in the User Data section, put the contents of the join.asg.cmd file from your cluster install directory.
  5. Select the IAM role created for the Okera cluster (in Prerequisites) and select Add Storage.
  6. Configure the volume to be 120GB in size of type gp2 and select Configure Security Group.
  7. Select the security group created for the Okera cluster (see Prerequisites) and select Review.
  8. Ensure all the settings are correct and select Create Launch Configuration.

After your launch configuration is created, create an auto-scaling group (ASG) that leverages it, navigating to the Auto-Scaling Group Wizard:

  1. Select your previously created launch configuration (e.g. okera-launch-config).
  2. Give your ASG a name, e.g. okera-asg and your initial desired number of instances (1 is fine to start with).
  3. Choose which VPC and subnet to create this ASG in. This should be the same as your original cluster node. Select Configure Scaling Policies.
  4. Select the default option Keep this group at its initial size and select Configure Notifications.
  5. Don't create any notifications, and select Configure Tags.
  6. Add any tags you need for this instance and select Review.
  7. Ensure all the settings are correct and select Create Auto Scaling Group.

The number of initial instances you requested are created and automatically joined to your Okera cluster. You can grow and shrink the cluster as your scaling needs change. You cal also apply scaling policies (e.g. based on CPU utilization) to automatically scale your cluster up or down.