3. Okera Install - Setting up a ODAS Cluster
- Configure Node Settings
- Prep the Cluster
- Create a Cluster
- Restarting a Failed Cluster after Addressing Issues
- Starting Multiple Okera Catalogs Sharing the Same Metadata
To create and launch an ODAS cluster, the Deployment Manager must be running.
Configure Node Settings
Cluster launch is broken up into two parts: actions required on the Deployment Manager and actions required on each launched EC2 instance.
Actions required on the Deployment Manager are referred to as the launch script or cluster launch script. One of those two scripts are required, and when called, should provision either a new EC2 instance or a new autoscaling group, respectively. The configured script is run from the DM master and should specify all of the required configurations described in the Cluster machine section in the prerequisites (for example, VPC, security groups, IAM Roles, etc). This script is the best place to tag machines or do other setup required by your organization. Details about launch scripts and cluster launch scripts can be found in Cluster Launch Plugin API.
Actions required on each launched EC2 machine are referred to as the init scripts. The init scripts are an optional set of scripts that run when the instance is launched on the instance machine. For example, one of these scripts could install any monitoring software that you use or configure the machine to use custom package repo locations.
Okera has provided a template launch script in the
Copy the script and adapt it to your requirements.
The user-configurable values are at the top of the file, marked
USER in a comment.
At a minimum, a subnet ID and security group ID must be added to the script.
Example: Migrating the launch script for editing
cp /opt/okera/deployment-manager/bin/start-ec2-machine-example.sh /etc/okera/launch-ec2.sh
To verify the launch script, Okera recommends running the script with the
If the configurations are valid, the script should exit with a status of
Example: Launch script validation
/etc/okera/launch-ec2.sh --dryrun Dry run succeeded echo $? 0
When a launch script is invoked without the
--dryrun flag, it is expected to return a string of the form:
Supplied cluster launch scripts do not need to support the –dryrun flag, as the associated AWS cli commands do not support that functionality.
Cluster launch scripts are expected to return the same output as the launch scripts with the exception that cluster launch scripts can return n lines, where each is formatted:
and each represent a host that was launched within the specified autoscaling group (ASG).
Note: The public IP value is optional. If your setup does not use it, its spot will appear blank.
Prep the Cluster
We’re almost ready to start a cluster from the CLI.
First create an environment. An environment captures all the configurations required to launch clusters.
Examples of environments might be for development or production, with the basic template:
ocadm environments create --name=<environment_name> --provider=AWS --launchScript=<DM_path_to_launch_script> --initScripts=<comma-separated_list_of_launch_scripts>
To run your cluster on an AWS autoscaling group (ASG), specify a –clusterLaunchScript instead of a –launchScript (those two flags are mutually exclusive)
ocadm environments create --name=<environment_name> --provider=AWS --clusterLaunchScript=<DM_path_to_cluster_launch_script> --initScripts=<comma-separated_list_of_launch_scripts>
Example: Environment definition
ocadm environments create --name=DevEnv --provider=AWS --launchScript=/etc/okera/launch-ec2.sh
Example: Environment definition that uses AWS ASGs
ocadm environments create --name=DevEnv --provider=AWS --clusterLaunchScript=/etc/okera/launch-asg.sh
Create a Cluster
Next we create at least one cluster in the environment. Multiple clusters can be created with the same environment.
Note: See Cluster Types for the available option when using the
Cluster creation follows this template:
ocadm clusters create --name=<cluster_name> --numNodes=<node_count> --type=STANDALONE_CLUSTER --environmentid=<environment_ID>
<environment_ID> is the numeric ID assigned after environment creation in the previous section.
Example: Creating a new cluster for environment
ocadm clusters create --name=fintech_prod --numNodes=1 --type=STANDALONE_CLUSTER --environmentid=1
List the running clusters.
ocadm clusters list
The newly created cluster should be returned and shown as “Provisioning”. You can see more details, including how long it has been in this state using the Okera CLI.
ocadm clusters status --detail <cluster_ID>
It might be convenient to watch the cluster until the state transitions to “READY”. It takes a few minutes to provision the machines, install ODAS, and start up all the services.
watch ocadm clusters status <cluster_ID>
At this point all of Okera is up and running!
To see the externally configured endpoints, run the
clusters endpoints command.
ocadm clusters endpoints 1
Note: The host and port of the Web UI are listed with the component named
okera_web:webui. The Web UI will respond at
https://<host>:<port>if using SSL.
Restarting a Failed Cluster after Addressing Issues
During an Okera environment creation, the launch scripts, init scripts, and config files are copied to the Okera install directory. Copying the config files is done to ensure that the clusters using that environment are unaffected by any new environments or clusters with different configurations.
If the script or config files in the environment contain errors, then the errors need to be corrected and the affected clusters restarted.
This includes issues with the
launch-ec2.sh script and any specified init scripts.
The Okera install scripts and config files are located at the
DEPLOYMENT_MANAGER_INSTALL_DIR environment variable defaults to
Starting Multiple Okera Catalogs Sharing the Same Metadata
Okera catalogs can be configured to share underlying metadata.
This feature is supported for “active-passive” configurations.
When creating the multiple clusters, supply the
catalogDbNamePrefix option using the same argument value in both cases.
Catalogs that share the same name also share the same metadata.
Example: Creating two clusters that share metadata:
ocadm clusters create --name=prod --numNodes=1 --type=STANDALONE_CLUSTER --environmentid=1 --catalogDbNamePrefix=metadata ocadm clusters create --name=prod-backup --numNodes=1 --type=STANDALONE_CLUSTER --environmentid=1 --catalogDbNamePrefix=metadata
Note: If the
--catalogDbNamePrefixwas not explicitly specified during cluster creation, then it is the name of the cluster.