Wise men say that the whole is greater than the sum of its parts. In a microservice architecture, we could be tempted to unit testing the single services, forgetting these are just parts of the system.

With good reason, unit tests are seen as a guiding light by every good developer. As the name suggests, this type of test involves just a unit of the overall codebase, without any external dependencies like databases, HTTP calls, queues, topics or something like that. This characteristic makes unit testing deterministic and, above all, fast; so we could run an entire suite of thousands of tests in a bunch of seconds.

Back in the days of big fat monolithic projects, a good suite of unit tests may validate the entire system right after a refactoring work.

Today, with the rise of microservices architectures, this is simply undoable! What we used to call the system, in fact, is now spread into hundreds or thousands of tiny little projects, completely unaware one another.

For this reason, unit testing is not enough when it comes to testing microservices oriented architectures!

Framing the context

One of the microservice’s promise is the possibility to shrink the big complexity of a monolithic application. As we have seen, this promise is kept splitting the initial complexity into smaller pieces. By design, each piece has to complete a small and simple task; the rest of the work will be delegated to someone else.

Fig. 1 — The Orders microservice accept a synchronous (solid line) HTTP call, then publish an asynchronous (dashed line) event into the system if order request is correct.

Suppose we have a service called Orders exposed through a REST API (e.g. POST /orders). The task of this service is to wait for an HTTP call with an order request:

POST /orders
{
    "customer_id": "peter",
    "address": "14 North Moore Street, New York"

    // For readability sake we hadn’t place 
    // products in this order request!
}

Once the request is received, the service has to perform some business validations on user input (e.g. does that customer_id really exist? What about the address?) and make some model’s enrichment (e.g. compute a uniquely generated order_id and a timestamp).

If the above steps are successful, the Orders service will push an event into the system notifying an order has been confirmed:

{
    "type": "order_confirmed",
    "order_id": "3123",
    "timestamp": "2019-12-10T17:16:06.656Z",
    "customer_id": "peter",
    "address": "14 North Moore Street, New York"
}

Somewhere, other services may be interested in this “fact”, so they will grab that message and will contribute to the state of the system through their small task. For instance, the Shipping service may activate the carrier to ship the order; while the ListOfOrders service may populate the customer’s orders history.

Fig. 2 — Other microservices may be interested in the fact that a new order has been confirmed, so they grab that message and do some work on it.

The usefulness of unit testing

Now suppose we are developing the Orders microservice. As said before, its task is really simple: we have to validate the user input; then, if successful, an event has to be published into a topic. In pseudo-code:

function order(customer_id, address) {
  if(!repository.exist(customer_id))
    throw CustomerNotFoundException()

  // other validations

  event = makeOrderConfirmedEvent(customer_id, address)
  topic.publish(event)
}

With unit tests in place we have a tool to quickly validate the correctness of our business logic:

function WhenCustomerNotFoundAnExceptionShouldBeThrownTest() {
  // ARRANGE (mocking exist method)
  repository.exist(customer_id) => { return false }

  try {
    // ACT
    sut.order(customer_id, address)
  } catch(e) {
    // ASSERT
    expect(e).isType(CustomerNotFoundException);
  }
}

The above test is based on a well-known pattern, called AAA (Arrange, Act, Assert). In the Arrange part we initialize objects and mock any external dependency with a canned response (e.g. we mocked the repository.exist(customer_id) method so, whatever the customer_id, the response will always be false).

Note: In short, mocking is creating fake objects that simulate the behaviour of real objects. This is useful when the real objects are impractical to incorporate into the test (e.g. databases, web service calls, topics, queues and so on).

Even though mocking is foundational activity of good unit testing it is out of scope for this article.

In the Act phase, we stimulate the SUT (i.e. the System Under Test), calling the order method (i.e. our business logic). As soon as it starts, the mocked repository.exist(customer_id) method will be called; the response of the method will be surely false, so the SUT should throw a CustomerNotFoundException.

The test will terminate with the Assert phase; where we expect to receive a CustomerNotFoundException otherwise, the test will fail.

Looking at the Orders business code, we know another important thing: if the user input is correct, the service has to call the topic.publish(event) method to publish an event on the target topic. We can be sure about this behaviour writing another unit test:

function WhenInputIsCorrectAnEventShouldBePublished() {
  // ARRANGE
  spy = spyOn(topic, 'publish')
  repository.exist(customer_id) => { return true }

  // ACT
  sut.order(customer_id, address)

  // ASSERT
  expect(spy).toHaveBeenCalled()
}

In this test, during the Arrange, phase we first configure a spy: a special object that will detect if topic.publish(event) method has been actually called. Then we mock the repository.exist(customer_id) method, but since we don’t want our logic to raise an exception, an always true value will be returned — whatever the customer_id. In the Act phase, we will stimulate the SUT and, since the mock, no exception will be thrown. In the Assert phase we expect that the spy has been called; otherwise, the test will fail!

So, why unit test is not enough?

…or better:

Why unit test, alone, is not enough when it comes to microservices?

As far as we have seen, unit tests are great at testing local behaviour of a service, that means its internal logic but…

«The critical complexity of the most software projects is in understanding the business domain itself.»

– Eric Evans: Domain Driven Design

As Eric Evans well explains in his seminal book, what is really complex in a software project is the understanding of the business domain, and, when it comes to microservices this complexity doesn’t lie into the single services, but rather in the communication among them.

If the order request can be handled, our Orders service will publish a message that will be eventually received by other services: in this example the ListOfOrders service. Following the microservice jargon, the Orders service is the producer and the ListOfOrders is the consumer service and since we are good developers, we have a high unit test coverage for both of those.

Fig 3. — Microservices communication through a pub/sub pattern.

Until now the producer (i.e. the Orders service) has always sent message like this to its consumers:

{
    "type": "order_confirmed",
    "order_id": "3123",
    "timestamp": "2019-12-10T17:16:06.656Z",
    "customer_id": "peter",
    "address": "14 North Moore Street, New York"
}

One day, a junior developer that works on Orders service codebase, realize that customer_id is not a real id but rather it is a username, therefore he changes the method makeOrderConfirmedEvt so it will produce a message like this:

{
    "type": "order_confirmed",
    "order_id": "3123",
    "timestamp": "2019-12-10T17:16:06.656Z",
    "customer_name": "peter",
    "address": "14 North Moore Street, New York"
}

With a small work of refactoring, this change to the internal of Orders service will be easily accepted by a unit test.

Once new Orders service is deployed, the ListOfOrders service will fail as it still needs the customer_id property.

Fig 4 — Communication breakdown.

As supposed, even though we unit tested each service thoroughly, we still put our system into trouble.

The whole is greater than the sum of its parts!

This happens because the two microservices (i.e. Orders and ListOfOrders) are fine on their own – and their unit tests can confirm that – but their communication is broken!

To be safer we need to stop testing the parts and start testing the system as a whole!

A holistic approach

Beware: I’m not saying that unit testing isn’t useful! What I’m saying, instead, is that even though unit tests are a great tool for validating microservices’ internal behaviour; what we really need is a complementary tool for testing their external behaviour too.

Generally speaking, we need to check that a modification of providers does not impact their consumers. As this type of tests are consumer-first they are called Consumer-Driven Contract Testing (CDCT).

How do CDCT works?

This type of testing is a little bit different from the unit testing, where we have a business logic that we want to validate and the related tests to perform that validation. In CDCT, each test is split into two different test file: the provider’s declarations and the consumer’s expectations.

These two parts act in very different ways. Through the consumer’s expectations, our testing framework can build a contract (i.e a JSON file with the serialized version of consumer’s expectations). Later on, when we need to change the provider logic, we could run a test on it, so the provider’s declaration will be verified against the JSON contract previously produced by the consumer. If these declarations break the contract the test will fail.

Fig. 5 — The basic flow of a CDCT.

Consumer side

Remember: ListOfOrders service is a consumer of Orders service. ListOfOrders service has a method called validateMessage(message) that will (drumroll) validate each message that is sent by the provider.

To effectively create our first CDCT we have to define the expectations of ListOfOrders service:

// define the contract
contract = new ConsumerContract({
  consumer: "ListOfOrders",
  dir: "contracts",
  provider: "Orders",
});

// consumer's expectations
return contract
  .expectsToReceive("an order_confirmed event")
  .withContent({
    type: "order_confirmed",
    order_id: {regex: "^[0-9]{1,4}$", generate: "4121"},
    customer_id: {regex: "^[0-9A-Za-z_.-]+$", generate: "peter"}
    // some properties (timestamp, address) omitted
})

// verify consumer's ability to handle messages
.verify(consumer.validateMessage));

In details, we initialize our contract, specifying the consumer’s name, the name of the provider and especially where the JSON contract will be saved (i.e. in a directory called contracts).

Then we define the consumer’s expectations. In particular, we expect “an order_confirmed event” (have in mind this), in which:

• type property is exactly equal to the string “order_confirmed”;

• order_id property is a string that contains only numbers and we provide a test value like the string “4121”;

• customer_id property contains letters, numbers and some special characters, like “peter”;

When we run this test, the engine will build a message like this:

{
    "type": "order_confirmed",
    "order_id": "4121",
    "customer_id": "peter",
    // some properties (timestamp, address) omitted
}

that will be sent to validateMessage(message) method so we can be sure that our defined expectations really match the consumer's behaviour. If so, a JSON contract will be published into contracts directory.

Provider side

Remember: Orders service is the provider of many services. ListOfOrders is one of those. Orders service contains a method called makeOrderConfirmedEvent that produces the message that will be sent to the consumers.

Now we need to write the test on the provider side:

// build provider's output
contract = new ProviderContract({
  messageProviders: {
    "an order_confirmed event": () =>   
      provider.makeOrderConfirmedEvent("peter", "14 North ...")
  },
  provider: "Orders",
  contractsUrls: "contracts/listoforders-orders.json"
})

// verify it against the JSON contract
contract.verify();

During this test, we will create “an order_confirmed event” (i.e. the type of messages that are consumed by ListOfOrders service, in fact, we can found the same string in consumer’s side) using makeOrderConfirmedEvent, then the testing engine will validate it against the JSON contract calling contract.verify().

Contract testing in action

As far as we said, the first test that we have to run is on the consumer’s side. This will validate that the expectations that we have defined really match the consumer’s taste.

Fig. 6 – The output of a consumer’s test.

When testing framework detects that the expectations defined on consumer’s side match the consumer’s rules (i.e. consumer’s consumer.validateMessage doesn’t throw an exception) a JSON contract will be built.

Note: Once the contract is created we need to share it with the providers. In this example, we are considering local filesystem as a contract source. This is handy because the provider and the consumer are in the same repository. In a more enterprise scenario, these entities could be separated, so we could share a Git repository as a single source of truth.

Now that we have our contract we can run a test on the provider side so it can be validated.

Fig 7. – The output of provider’s test with the broken contract due to customer_name.

As we expected, it turns out the provider (i.e. Orders) broke the contract with one of its consumer (i.e. ListOfOrders) services. The testing framework provides a clear explanation of what is wrong:

Could not find key “customer_id” (keys present are: type, order_id, timestamp, customer_name)

With this type of testing in place, our junior colleague wouldn’t push a bug in production because he would have had a tool to understand, in advance, if refactoring customer_id into customer_name was really a wise move.

Summary

The web is full of blog posts about unit testing, but I haven’t found many articles about Consumer-driven contract testing. Of these, many articles concentrate on HTTP integration between providers and consumers. Only a bunch of them consider an integration through messaging.

With this post, I hope to give a gentle overview of the topic and why we really need it. So let’s summarize:

• Unit tests are great at validating microservices internal behaviour;

• Unit testing alone get no guarantees: as we have seen, a change to a service with a high unit test coverage could go unnoticed and broke a consumer service;

• Consumer-driven contract testing is a complementary approach to unit testing. Here we test the communication between two services: a provider and a consumer, leaving out the service’s internal;

• Who develops a consumer service, write the consumer’s test too. This produces a JSON contract;

• Who develops a provider service will verify it against this contract. If the contract is broken we have to stop pushing our code into the repository;

That’s all for now. Since I’m more interested in the concept rather than the mere implementation, the code shown in this post doesn’t compile. :)

Anyway, if you are interested in the implementation details here is a working example in NodeJS deployed on AWS Lambda. Enjoy!

If you liked this post, please support my work.

Further resources

• M. Fowler — Unit testing

• K. Back — Testing Driven Development: By examples

• E. Evans — Domain Driven Design

• S. Newman — Building Microservices

• B. Sayers, M. Edo — Verifying Microservice Integrations with CDCT