Unit Test Network

Motivation

The purpose of this article is to document in some way a software development technique I use to build complex components.

This is to be an example of test-driven development that might be useful to someone.

Storytime

This whole thing started at work, with the design of a parser for a JSON datastructure that is going to arrive to a microservice through an event queue.

I started the development of this parser from a similar one developed by another team for the same event. The thing is that this datastructure was to be discarded uppon detecting certain conditions on its fields.

This event is sent to many microservices, and depending on the service, a different set of actions need to be executed, or simply to be ignored upon those conditions.

Given that we had a working example, I felt confident and was not as disciplined as I should.

One week later I was 1000 lines of C++ into a blob of code that did not work at some unknown step. The parser I had based my code on still required a lot of work and only had superficial unit testing done on it.

Struggle is the symptom of progress, and so I began to aggresively refactor the blob into a monolith of functions and data.

The following image is a representation of a version of the parser before the refactoring, a single test suite testing only the upper layer of this monolithic parser.

The developers of the other team had done the “testing” of the underlying components in a fairly manual way, using the standard output to visually verify the internal state at each point of the parsing.

At the top of it, there was a fairly large suite of unit tests, which was the formal way to prove that at some point the code written was being run at some point by an unit test.

However, this test suite was useless to pinpoint the exact point where the code was failing.

The first option would be to launch a debugger or fill the flow of the program with messages to stdout in order to somehow infer what is going on. However, this is a pretty large parser, these traces soon get out of control and as hours pass, my eyes start to hurt.

We can do much better.

Unit Test Network

It would be nice to have a way to know exactly the point at which we have this error. Over the decades, several mechanisms have been developed to improve the way in which we detect inconsistencies and errors on software.

One of them is the strict syntax that some languages such as Java, C and C++ have in relation with type definitions and how we interact with those types in order to provide memory and some level of runtime safety.

Another powerful inovation was the development of linters such as clang as an additional security network while building features or modifying code. Nowadays, a good linter is a mandatory tool of any good IDE.

However, there are still many classes of errors that go through undetected. These are nothing but the traditional runtime errors, logic errors, overflow errors, etc.

There are some advanced techniques such as formal methods that show promise in producing software free of any kind of errors, although with their own set of challenges.

In modern software development we rely on a network of unit tests across the codebase to verify that each of the individual components our business logic relies on works as it is supposed to.

The idea here is to setup a test suite for each individual component or subcomponent that might have its own individual logic or behaviour. A complete set of unit tests for each of them, verifying ideally all the edge cases of the component, a couple of positive cases, and a set of negative cases. These tests provide the assurance needed to detect when a refactoring or a change on the code breaks anything and in the best case, pointing out exactly what is broken.

Notice that on this article we will refer to an entity as a unit of logic or behaviour.

On this example we are using the idea of a pyramid as a way of visualizing a monolith, a set of components organized on a hierarchy in order to provide a higher level functionality.

If any entity fails, we can see several test cases fail, usually connected to such entity.

Yes, we are probably doubling the ammount of code to write, but the code written will be more stable and of higher quality. The fact is that you are going to pay up a price anyway, so it is better to pay it as early as possible in order to to have a steady and predictable development progress.

The alternative is to hack away your code with traces to stdout or to launch up a debugger to try to somehow reproduce the bug and figure out the internal state.

I cannot talk for the experience of others, but in my personal experience all I can say is that those debugging sessions lasting hours, days and even weeks can hardly be considered neither efficient nor enjoyable.

Write your test cases

The conclussion for this post is fairly simple, this is just a call for myself and others to organize our ideas and attempt to see things clearly.

Writting unit tests as a rule for our software is one of such attempts.

The reality is that we will not be able to write clear and specific test cases for many functionalities without also using more advanced techniques such as mocking. Even if we do our best, complete testing of inputs and outputs is a problem that grows exponentially. Do not confuse this with coverity, which essentially measures the percentage of lines of code that are executed on at least one unit test, not if the test proves validity for a meaningful set of inputs or even if it is correctly done.

This is why it is a network of unit tests, the more dense it is, the more likely that we will catch those runtime or logic errors.

And even if we do everything right, we do not work in isolation, our components will need to get integrated inside a larger construct. That integration also needs to be tested.

We cannot spare from any weapon in our eternal fight against complexity, specially when dealing with software.

Sources

• Test-driven development