Smooth RavenDB Scenario Testing

Writing scenario tests with RavenDB included can be extremely time-consuming. It requires quite some effort in general to construct a dedicated test state that includes migrations, running subscription processors and deployed indices. This blog post contains a short list of basic code fragments that help you writing scenario tests in RavenDB context. We make heavy use of the mentioned mechanisms ourselves to test meshStack’s behavior.

Let’s kick off things with a helpful mechanism that RavenDB provides already out of the box:

1. Checking RavenDB state during tests

During development of tests, it is helpful to have a possibility to halt the execution and have a look into RavenDB’s state to double check existing collections and document’s contents to see whether a test setup works as intended or not.

RavenDB’s RavenTestDriver class provides a function waitForUserToContinueTheTest(IDocumentStore store) that blocks execution when reached and opens up a new browser window with the provided store pre-selected. This gives you all the time you need to explore the current state within the database.

You can continue test execution with the button marked in this screenshot:

Unfortunately this feature is a bit limited: For example you cannot browse ongoing tasks in this view and with that there is no possibility to see active subscriptions. Nonetheless we included the RavenTestDriver in our base class that we use for testing to have that feature conveniently ready to use.

2. Reading Compare Exchange values from RavenDB

Compare Exchange (cmpx) values are a great way to guarantee cross-cluster atomicity in RavenDB. At meshcloud, we implemented a migration pattern based on cmpx values, so we know that each of our migrations is applied exactly once.

On a few occasions we need databases to be created at runtime with all required migrations applied. For this, we built a simple mechanism to test whether all migrations are correctly applied by extracting the cmpx values from RavenDB.

This kotlin code snipped shows how:

// Autowire all required RavenDB migrations
@Autowired
private lateinit var migrations: List<DatabaseMigration>

[...] 

/**
 * Load all compare exchange values for a given store
 * and assert that for each required migration the expected
 * value is present with a success status
 */
val cmpxs = loadMigrationCmpxs(store)
migrations.forEach {
  val key = "migrations/${it.version}"
  assertThat(migrationCmpxs.keys).contains(key)
  assertThat(migrationCmpxs[key]!!.value.status).isEqualToIgnoringCase("success")
}

[...]

// Helper class to model the cmpx values
data class MigrationCmpx(
    val startedAt: Instant,
    val executedBy: String,
    val status: String,
    val lastUpdated: Instant
)

// Helper function that load the cmpx values with help of clusterTransaction
private fun loadMigrationCmpxs(store: IDocumentStore): Map<String, CompareExchangeValue<MigrationCmpx>> {
  store.openSession(
     SessionOptions().apply {
         transactionMode = TransactionMode.CLUSTER_WIDE
     }
  ).use { session->
    val clusterTx = session.advanced().clusterTransaction()
    return clusterTx.getCompareExchangeValues(
        MigrationCmpx::class.java, 
        "migrations"
    )
  }
}

3. Check RavenDB subscription states

Besides the required migrations on a database we also need to have certain active subscriptions. These are also partly created dynamically during runtime. To ensure that these are started correctly there is also a mechanism to query that information.

This is in particular helpful, because we cannot gain this insight from the method described in section 1 using the RavenTestDriver to pause test execution.

Have a look at the following snippet to see how we can check for subscription states:

/**
 * Using the names of the subscriptions we create a map
 * that has the subscription name as key and the according state
 * as the value.
 * The map’s values are instances of RavenDB’s OngoingTask abstract class.
 */
protected fun getSubscriptionStates(
    store: IDocumentStore, 
    subscriptionNames: List<String>
): Map<String, OngoingTask> {

  return subscriptionNames
      .associateWith {
        store
         .maintenance()
         .send(GetOngoingTaskInfoOperation(it, OngoingTaskType.SUBSCRIPTION))
      }
}

[...]

// it is possible to check that the subscriptions run e.g. like this:
val subscriptionStates = getSubscriptionStates(store, subscriptionNames)

subscriptionStates.forEach { (_, v) ->
    assertThat(v).isNotNull
    assertThat(v.taskState).isEqualTo(OngoingTaskState.ENABLED)
}

4. Handling asynchronous behavior in RavenDB

During the tests you may encounter the case in which you want to test certain documents, but it seems that they are not yet correctly updated because of some RavenDB internals. Most of the time that will be an index that is not completed yet.

To tackle that the RavenTestDriver also has a built-in function that allows you to wait for all indices to complete:

RavenTestDriver.waitForIndexing(store)

This has proven to be very helpful during test setup where all required indices have been created and all test documents have been inserted. To ensure that everything is ready for the tests to start, we explicitly wait for the indexing to complete so we know no side effects interfere with our tests.

Besides all the functionality that RavenDB provides and we can utilize, we have also built a small helper function ourselves to deal with complex asynchronous behavior that might come up. Especially when you have a setup where you need to wait for multiple indices and subscriptions before you want to assert the result, it makes sense to not explicitly mention everything in code but to just assert the expected values in the end.

object AssertEventually {

  fun <T> that(
      timeout: Duration = Duration.ofMillis(5000),
      backOffPeriod: Duration = Duration.ofMillis(50),
      block: () -> T
  ): T {

    val retryTemplate = RetryTemplate().apply {
      val fixedBackOffPolicy = FixedBackOffPolicy().apply {
        this.backOffPeriod= backOffPeriod.toMillis()
      }
      setBackOffPolicy(fixedBackOffPolicy)

      val retryPolicy = TimeoutRetryPolicy().apply {
        this.timeout= timeout.toMillis()
      }
      setRetryPolicy(retryPolicy)
    }

    return retryTemplate.execute<T, Exception> { block() }
  }
}

[...]

// usage:
AssertEventually.that() {
  // place your assertions here
}

With AssertEventually we utilize Spring’s RetryTemplate to assert that eventually all expectations are met. In case some asynchronous behavior is going on in RavenDB that is too complex to handle explicitly we can just wrap the assertions within this block and give RavenDB some time to calculate.

I encourage you not to use this pattern in production, but for tests it has proven to greatly improve readability and simplicity.

Of course, the above snippet can be modified to utilize different Backoff or Retry Policies and can be tweaked with suitable values for the timeout and backOffPeriod parameters.

Run your own RavenDB scenario tests

I hope with these rather simple tips you can improve the code quality around your scenario tests with RavenDB or feel motivated to start writing tests in this area. Please do not hesitate to get in touch, I will gladly hear your ideas and tips on this topic.