tlp-stress is a workload-centric stress tool, written in Kotlin. Workloads are easy to write and because they are written in code, you have the ultimate flexibility. Workloads can be tweaked via command line parameters to make them fit your environment more closely.

One of the goals of tlp-stress is to provide enough pre-designed workloads out of the box so it’s unnecessary to code up a workload for most use cases. For instance, it’s very common to have a key value workload, and want to test that. tlp-stress allows you to customize a pre-configured key-value workload, using simple parameters to modify the workload to fit your needs. Several workloads are included, such as:

  • Time Series

  • Key / Value

  • Materialized Views

  • Collections (maps)

  • Counters

The tool is flexible enough to design workloads which leverage multiple (thousands) of tables, hitting them as needed. Statistics are automatically captured by the Dropwizard metrics library.

Quickstart Example

The goal of this project is to be testing common workloads (time series, key value) against a Cassandra cluster in 15 minutes or less.

Building the Stress Tool

First you’ll need to clone and build the repo. You can grab the source here and build via the included gradle script:

git clone https://github.com/thelastpickle/tlp-stress.git
cd tlp-stress
./gradlew assemble

Run Your First Stress Workload

Assuming you have either a CCM cluster or are running a single node locally, you can run this quickstart.

Either add the bin directory to your PATH or from within tlp-stress run the following command to execute 10,000 queries:

$ bin/tlp-stress run KeyValue -n 10000

You’ll see the output of the keyspaces and tables that are created as well as some statistical information regarding the workload:

Creating schema
Executing 10000 operations
Connected
Creating tlp_stress:
CREATE KEYSPACE
 IF NOT EXISTS tlp_stress
 WITH replication = {'class': 'SimpleStrategy', 'replication_factor':3 }

Creating Tables
CREATE TABLE IF NOT EXISTS keyvalue (
                        key text PRIMARY KEY,
                        value text
                        )
Preparing queries
Initializing metrics
Connecting
Preparing
1 threads prepared.
Running
[Thread 0]: Operations: 10000
[Thread 0]: All operations complete.  Validating.
Stress complete, 1.
                  Writes                                    Reads                  Errors
  Count  Latency (p99)  5min (req/s) |   Count  Latency (p99)  5min (req/s) |   Count  5min (errors/s)
   4955          38.52             0 |    5045          36.74             0 |       0                0

If you’ve made it this far, congrats! You’ve run your first workload.

Usage

You’ll probably want to do a bit more than simply run a few thousand queries against a KeyValue table with default settings. The nice part about tlp-stress is that it not only comes with a variety of workloads that you can run to test your cluster, but that it allows you to change many of the parameters. In the quickstart example we used the -n flag to change the total number of operations tlp-stress will execute against the database. There are many more options available, this section will cover some of them.

General Help

tlp-stress will display the help if the tlp-stress command is run without any arguments or if the --help flag is passed:

$ bin/tlp-stress
Usage: tlp-stress [options] [command] [command options]
  Options:
    --help, -h
      Shows this help.
      Default: false
  Commands:
    run      Run a tlp-stress profile
      Usage: run [options]
        Options:
          --compaction
            Compaction option to use.  Double quotes will auto convert to
            single for convenience.
            Default: <empty string>
          --compression
            Compression options
            Default: <empty string>
          --concurrency, -c
            Concurrent queries allowed.  Increase for larger clusters.
            Default: 250
          --drop
            Drop the keyspace before starting.
            Default: false
          --field.
            Override a field's data generator
            Syntax: --field.key=value
            Default: {}
          -h, --help
            Show this help
          --host

            Default: 127.0.0.1
          --id
            Identifier for this run, will be used in partition keys.  Make
            unique for when starting concurrent runners.
            Default: 001
          --iterations, -i, -n
            Number of operations to run.
            Default: 1000
          --keyspace
            Keyspace to use
            Default: tlp_stress
          --partitions, -p
            Max value of integer component of first partition key.
            Default: 1000000
          --password, -P

            Default: cassandra
          --populate
            Pre-population the DB
            Default: false
          --rate
            Rate limiter, accepts human numbers. 0 = disabled
            Default: 0
          --readrate, --reads, -r
            Read Rate, 0-1.  Workloads may have their own defaults.  Default
            is dependent on workload.
          --replication
            Replication options
            Default: {'class': 'SimpleStrategy', 'replication_factor':3 }
          --threads, -t
            Threads to run
            Default: 1
          --username, -U

            Default: cassandra

    info      Get details of a specific workload.
      Usage: info

    list      List all workloads.
      Usage: list

Listing All Workloads

$ bin/tlp-stress list
Available Workloads:

BasicTimeSeries
Maps
CountersWide
MaterializedViews
KeyValue
LWT

Done.

Getting infomration about a workload

It’s possible to get (some) information about a workload by using the info command. This area is a bit lacking at the moment. It currently only provides the schema and default read rate.

$ bin/tlp-stress info KeyValue
CREATE TABLE IF NOT EXISTS keyvalue (
                        key text PRIMARY KEY,
                        value text
                        )
Default read rate: 0.5

Running a Customized Stress Workload

Whenever possible we try to use human friendly numbers. Typing out -n 1000000000 is error prone and hard to read, -n 1B is much easier.

Table 1. Table Human Friendly Values
Suffix Implication Example Equivilent

k

Thousand

1k

1,000

m

Million

1m

1,000,000

b

Billion

1b

1,000,000,000

Partition Keys

A very useful feature is controlling how many partitions are read and written to for a given stress test. Doing a billion operations across a billion partitions is going to have a much different performance profile than writing to one hundred partitions, especially when mixed with different compaction settings. Using -p we can control how many partition keys a stress test will leverage. The keys are randomly chosen at the moment.

Read Rate

It’s possible to specify the read rate of a test as a double. For example, if you want to use 1% writes, you’d specify -r .01.

Compaction

It’s possible to change the compaction strategy used. At the moment this changes the compaction strategy of every table in the test. This will be addressed in the future to be more flexible.

Compression

It’s possible to change the compression options used. At the moment this changes the compression options of every table in the test. This will be addressed in the future to be more flexible.

Customzing Fields

To some extent, workloads can be customized by leveraging the --fields flag. For instance, if we look at the KeyValue workload, we have a table called keyvalue which has a value field.

To customize the data we use for this field, we provide a generator at the command line. By default, the value field will use 100-200 characters of random text. What if we’re storing blobs of text instead? Ideally we’d like to tweak this workload to be closer to our production use case. Let’s use random sections from various books:

$ tlp-stress run KeyValue --field.keyvalue.value='book(20,40)`

Instead of using random strings of garbage, the KeyValue workload will now use 20-40 words extracted from books.

There are other generators available, such as names, gaussian numbers, and cities. Not every generator applies to every type. It’s up to the workload to specify which fields can be used this way.

Developer Docs

Building the documentation

First generate the command examples with the following shell script:

manual/generate_examples.sh

There’s a docker service to build the HTML manual:

$ docker-compose up docs

Writing a Custom Workload

tlp-stress is a work in progress. Writing a stress workload isn’t documented yet as it is still changing.