This is the manual for tlp-cluster, a provisioning tool for Apache Cassandra designed for developers looking to benchmark and test Apache Cassandra. It assists with builds and starting instances on AWS.
If you are looking for a tool to aid in benchmarking these clusters please see the companion project tlp-stress.
There are some AWS resources tlp-cluster expects to exist in us-west-2 (support for other regions will come soon):
The cluster provisioning tool is somewhat self explanatory (hopefully).
It launches Cassandra clusters in AWS. In addition to Cassandra, the tool can also launch stress nodes and a prometheus node to gather all the metrics.
The tool itself is written in Kotlin, leveraging Docker containers to avoid too many external dependencies.
To get started, add the bin directory of tlp-cluster to your $PATH. For example:
export PATH="$PATH:/path/to/tlp-cluster/bin" cd /path/to/tlp-cluster ./gradlew assemble
We currently have a dependency on shell scripts meaning you’ll need a Mac or Linux box to use this tool.
If you’ve never used to the tool before, the first time you run a command you’ll be asked to supply some information, which will generate a configuration file which will be placed in your
|We currently only support the ubuntu 16 ami in us-west-2. We’d love a pull request to improve this!|
Running the command without any arguments will print out the usage:
You’ll see the help. It looks like this:
Usage: tlp-cluster [options] [command] [command options] Options: --help, -h Shows this help. Default: false Commands: init Initialize this directory for tlp-cluster Usage: init [options] Client, Ticket, Purpose Options: --ami AMI Default: ami-51537029 --cassandra, -c Number of Cassandra instances Default: 3 --instance Instance Type Default: c5d.2xlarge --monitoring, -m Enable monitoring (beta) Default: false --region Region Default: us-west-2 --stress, -s Number of stress instances Default: 0 --up Start instances automatically Default: false up Starts instances Usage: up [options] Options: --auto-approve, -a, --yes Auto approve changes Default: false start Start cassandra on all nodes via service command Usage: start [options] Options: --all, -a Start all services on all instances. This overrides all other options Default: false --monitoring, -m Start services on monitoring instances Default: false stop Stop cassandra on all nodes via service command Usage: stop [options] Options: --all, -a Start all services on all instances. This overrides all other options Default: false --monitoring, -m Start services on monitoring instances Default: false install Install Everything Usage: install down Shut down a cluster Usage: down [options] Options: --auto-approve, -a, --yes Auto approve changes Default: false build Create a custom named Cassandra build from a working directory. Usage: build [options] Path to build Options: -n Name of build ls List available builds Usage: ls use Use a Cassandra build Usage: use [options] Options: --config, -c Configuration settings to change in the cassandra.yaml file specified in the format key:value,... Default:  clean null Usage: clean hosts null Usage: hosts Done
Initialize a Cluster
The tool uses the current working directory as a project, of sorts. To get started, run the following, substituting your customer / project, ticket and purpose.
tlp-cluster init CLIENT TICKET PURPOSE
CLIENT- Name of the customer, client, or project associated with the work you are doing with
TICKET- Jira or github ticket number associated with the work you are doing with
PURPOSE- Reason why you are creating the cluster.
This will initialize the current directory with a terraform.tf.json. You can open this up in an editor. Here you can change the number of nodes in the cluster, as well as configure the number of stress nodes you want. You can also change the instance type. Generally speaking though, you shouldn’t have to do this. If you find yourself doing it often, please submit an issue describing your requirements and we’ll work with you to solve the problem.
Certain instances types may not work with the ami that’s hard coded at the moment, we’re looking to fix / improve this.
Launch your instances with the following:
Terraform will eventually ask you to type
yes and fire up your instances. Optionally you can pass
--yes to the
-up command and you won’t be prompted.
The Easy Way - Use a Released Build
The easiest path forward to getting a cluster up and running is the following:
tlp-cluster use 3.11.4 tlp-cluster install tlp-cluster start
Simply replace 3.11.4 with the release version.
The Hard Way - Use a custom Build
To install Cassandra on your instances, you will need to follow these steps:
Build the version you need and give it a build name (optional)
Tell tlp-cluster to use the custom build
The first step is optional because you may already have a build in the
~/.tlp-cluster/build directory that you want to use.
If you have no builds you will need to run the following:
tlp-cluster build -n BUILD_NAME /path/to/repo
BUILD_NAME- Name you want to give the build e.g. my-build-cass-4.0.
/path/to/repo- Full path to clone of the Cassandra repository.
If you already have a build that you would like to use you can run the following:
tlp-cluster use BUILD_NAME
This will copy the binaries and configuration files to the
provisioning/cassandra directory in your
tlp-cluster repository. The
provisioning directory contains a number of files that can be used to set up your instances. Being realistic, since we do so much non-standard work (EBS vs instance store, LVM vs FS directly on a device, caches, etc) we need the ability to run arbitrary commands. This isn’t a great use case for puppet / chef / salt / ansible (yet), so we are just using easy to modify scripts for now.
If you want to install other binaries or perform other operations during provisioning of the instances, you can add them to the
provisioning/cassandra directory. Note that any new scripts you add should be prefixed with a number which is used to determine the order they are executed by the
To provision the instances run the following:
SSH_KEY_PATH- Is the full path to the private key from the key pair used when creating the instances.
This will push the contents of the
provisioning/cassandra directory up to each of the instances you have created and install Cassandra on them.