Integration tests with Testcontainers and Aerospike

Written on September 2, 2021

Testing is a crucial part of modern software development. As a software engineer or software developer in test, you want your tests to be stable, fast, and easily maintained.

There is a software testing pyramid that shows the proportion of types of tests you want to keep in your project.

The unit test validates the functionality of each class(unit), integration test checks the functionality of a module with external systems like database, web service, and UI/e2e test the entire business flow or UI flow.

Unit tests are fast - if you do everything right, but what about integration and end-to-end tests? They can be slow because of external dependencies like databases and other services. Usually, developers choose:

  • Either to use some already running services
  • Or start them just for testing purposes.

In the former case, you use some already running instance, fill it with data, or just use an already existent one, test the logic of SUT, and then clean up the results of the work from the DB.

In the latter case you start a database, fill it with some data, test the logic of SUT and stop the database, data cleanup is done automatically.

The Problem

Whats’s the most effective way to manage service/DB for an integration test? Use dockerized versions of that service/DB! But working with docker using CLI is not a convenient option. Using  API or library is a much more preferable option.

Solution

At this point, I want to present to you the “Testcontainers”. The main idea of this software is - wrap the docker management into a library to have the ability to programmatically start/manage/stop containers for testing purposes. There’s a number of implementation for different langs: https://github.com/testcontainers/testcontainers-java/

https://github.com/testcontainers/testcontainers-scala

https://github.com/testcontainers/testcontainers-node

The most pleasant fact that you have a lot of modules for popular DBs, load balancers, and services already integrated. So starting MySQL or nginx is simple as

class GenericContainerSpec extends AnyFlatSpec with ForAllTestContainer {
  override val container: GenericContainer = GenericContainer("nginx:latest",
    exposedPorts = Seq(80),
    waitStrategy = Wait.forHttp("/")
  )
}

And that’s it, you can use Nginx in your test cases! If you need some kind of software not included in standard modules don’t worry, you can create your own generic containers. Let’s find out how to do this and create testcontainer module for Aerospike. We start by taking the docker image and adjusting the settings to make if work.

override val container: GenericContainer = GenericContainer(
    "aerospike/aerospike-server:5.6.0.4",
    exposedPorts = Seq(3000, 3001, 3002),
    env = Map(
      "NAMESPACE"    -> "testing",
      "SERVICE_PORT" -> "3000"
    ),
    waitStrategy = Wait.forLogMessage(".*migrations: complete.*", 1)
  )

These settings mean that you will have an aerospike-server running on 3000 port, with the “testing” namespace set. To make sure we don’t try to read from aerospike before the database is ready we add a special wait strategy. Standard Aerospike Test Container Class can look like this:

import com.aerospike.client.AerospikeClient
import com.aerospike.client.Host
import com.aerospike.client.async.EventPolicy
import com.aerospike.client.async.NioEventLoops
import com.aerospike.client.policy.BatchPolicy
import com.aerospike.client.policy.ClientPolicy
import com.aerospike.client.policy.WritePolicy
import com.dimafeng.testcontainers.ForAllTestContainer
import com.dimafeng.testcontainers.GenericContainer
import org.testcontainers.containers.wait.strategy.Wait

trait AerospikeTestContainer { self: ForAllTestContainer =>

  override val container: GenericContainer = GenericContainer(
    "aerospike/aerospike-server:5.6.0.4",
    exposedPorts = Seq(3000, 3001, 3002),
    env = Map(
      "NAMESPACE"    -> "testing",
      "SERVICE_PORT" -> "3000"
    ),
    waitStrategy = Wait.forLogMessage(".*migrations: complete.*", 1)
  )

  val writePolicy = new WritePolicy()
  val batchPolicy = new BatchPolicy()

  lazy val ip = container.containerIpAddress

  lazy val aerospikeHosts = Host.parseHosts(
    s"$ip:${container.mappedPort(3000)},$ip:${container.mappedPort(3001)},$ip:${container.mappedPort(3002)}",
    3000
  )


  lazy val client = {
    val eventLoops = new NioEventLoops(new EventPolicy(), 8)
    val clientPolicy = {
      val clientPolicy = new ClientPolicy()
      clientPolicy.eventLoops         = eventLoops
      clientPolicy.maxConnsPerNode    = 50
      clientPolicy.batchPolicyDefault = batchPolicy
      clientPolicy.writePolicyDefault = new WritePolicy()
      clientPolicy
    }

    new AerospikeClient(clientPolicy, aerospikeHosts: _*)
  }
}

And test case like this:

class AerospikeTestSuite
    extends IntegrationTestsBase
    with AerospikeTestContainer {
     
      "Aerospike client" should "do write/read" taggedAs (IntegrationTest) in {
        import com.aerospike.client.Key
        import com.aerospike.client.Bin

        // Write single value.
        val key = new Key("testing", "myset", "mykey")
        val bin = new Bin("mybin", "myvalue")
        client.put(writePolicy, key, bin)
        val record = client.get(batchPolicy, key)

        assert(record.bins.get("mybin") === "myvalue")
      }
    }

You can find a fully working example on the github: https://github.com/dkovalenko/aerospike-testcontainer

Voila! You have a database spawned in a docker container just for your test. You can fill it with data and check your application business logic in integration tests.

Have some comments?


Tags: scala, testcontaners, aerospike, scalatest