This is the manual for tlp-cluster, a provisioning tool for Apache Cassandra designed for developers looking to both benchmark and test the correctness of Apache Cassandra. It assists with builds and starting instances on AWS.

Prerequisites

There are some AWS resources tlp-cluster expects to exist:

  • A security group that allows inbound SSH traffic.

  • A key pair, private part of which should be present in your ~/.ssh with chmod 400 on it.

Installation

The cluster provisioning tool is somewhat self explanatory (hopefully).

It launches Cassandra clusters in AWS. In addition to Cassandra, the tool can also launch stress nodes (soon) and a prometheus node to gather all the metrics.

The tool itself is written in Kotlin, leveraging Docker and Docker Compose (for now).

To get started, add the bin directory in tlp-cluster to your $PATH. For example:

export PATH="$PATH:/path/to/tlp-cluster/bin"
cd /path/to/tlp-cluster
./gradlew assemble

Now you can use the tool, yay.

Setup

This tool requires a bit of bash and docker. I don’t know if will run on Windows. We assume most people doing any serious Cassandra development will have access to a Linux box or virtual machine.

First, make sure you’ve set up an AWS profile profile.

The rest of this doc will refer to the profile tlp.

Second, run the tool without any parameters. It’ll tell you that it’s set up a configuration file at ~/.tlp-cluster/user.tfvars. Open that file with your editor and fill in the variables. They all coorespond to the same vocabulary in AWS.

We currently only support the ubuntu 16 ami in us-west-2. We’d love a pull request to improve this!

Running the command without any arguments will print out the usage:

tlp-tools

Initialize a Cluster

The tool uses the current working directory as a project, of sorts. To get started, run the following, substituting your customer, ticket and purpose. This is an artifact of how we work on projects at TLP.

tlp-cluster init Customer TICKET-ID "Reason for cluster"

This will initialize the current directory with a variables file, terraform.tfvars. Open it up in an editor. Here you can change the number of nodes in the cluster, as well as configure the number of stress nodes you want. You can also change the instance type.

Certain instances types may not work with the ami that’s hard coded.

Launch Instances

Launch your instances with the following:

tlp-cluster provision

There are a few things Terraform will ask for if they are not present in terraform.tfvars:

name = "Your Name"
email = "your@email.com"

# AWS profile from your ~/.aws/credentials
profile = "default"

# name of the key you intend to use for connecting to the instance
key_name = "yourkey"

# securty group the nodes should belong to
security_groups = ["yourgroup"]

Whether you pass these values is on the fly, or place them in the file, Terraform should eventually ask you to type yes and fire up your instances.

The provisioning directory contains a number of files that can be used to set up your instances. Being realistic, since we do so much non-standard work (EBS vs instance store, LVM vs FS directly on a device, caches, etc) we need the ability to run arbitrary commands. This isn’t a great use case for puppet / chef / salt / ansible (yet), so we’ll just use easy to modify scripts for now.

You can run arbitrary commands on all hosts:

tlp-cluster cs ls