The initial problem
Shortly after joining a new project, I realized that while there were a considerable amount of tests in the application code, unit test integration have been disabled for the pipeline years before, and writing unit tests for new code was not a regular development practice. As a result, they were unmaintained for more than two years, unsuitable to serve their purpose as a quality tool.
Approach
The development team agreed to the practice of incremental refactoring, an approach where we would try to make small improvements only to the code parts we were working with anyway, as well as the "boyscout rule": leaving things in a better state than we had found them.
Since the test code was severely neglected, unit test execution could not be re-enabled in the pipeline as-is. We have evaluated the tests and put them into three different categories:
- The first category was for green tests (there were not a lot of them), which we used to start building a default unit test suite. The automated execution of this test suite for each build was immediately re-enabled on the pipeline.
- The second category was for broken tests that had to be temporary excluded from pipeline execution.
- The third category was for tests that were candidates for deletion (because related functions or classes did not exist anymore or the code it was testing has changed so drastically).
- Integration tests were excluded from this effort entirely, as we did not have sufficient resources to configure proper database fixtures for pipeline integration.
The team began to work on the second category gradually. Some tests were easier to fix without having to guess intended behavior (such as deprecations), while some others required a more careful approach.
Challenges
One counter-argument for starting similar efforts that I often hear is that the team had "no time" for improving the state of unit tests. The biggest challenge is the first step: getting the entire development team on board, and winning their support and willingness to put in a little extra effort.
As I have mentioned earlier, a small part of the existing unit tests that were green (therefore would not break the pipeline) could be re-enabled immediately.
Our approach was that we only ever attempted to fix tests for classes we were working on anyway, so we could harmonize this initiative with our daily development tasks better, therefore it did not require us to dedicate significant extra time to this, as it is often difficult to convince a product manager that this step towards improving code quality is absolutely necessary.
Code reviews provided a good opportunity to review test code changes, learn from each other (as the team had experienced testers and novices alike), discuss bad practices and discard bad unit tests and improve the general quality of our unit tests. For failing tests, it was important to be careful and assess each case individually, as an unmaintained test could be broken because the code it used to test has changed so much therefore the test needed to be updated, but it was also possible that the broken test indicated broken funtionality - serving its original purpose correctly.
It is important to note that existing unit tests are not supposed to change frequently. If tests have to be changed, it means that the code likely does not follow the open-close SOLID principle, or the test itself is checking implementation details (which might change over time due to refactoring) instead of unit behavior. Good unit tests allow the code to be refactored without breaking.
How we measured success
One crucial part of success was making unit testing a core part of our regular development activities. The development team was commited to adding unit test coverage to our Definition of Done, which meant that no new code could be submitted without unit tests.
The code coverage driver also allowed us to measure our current covarage, which was around 20% across the board, and the team was commited to measuring the coverage often, and try to improve this metric release-by-release. The coverage report also allowed us to identify business-critical parts of the application that were untested, so we could prioritize fixing the broken tests and writing the missing tests.
While code coverage can be a controversial metric, it is still a metric that can be presented to product management to showcase improvements, and I believe we are in the right to use it if we treat it as a transparent and honest measurement of our code quality, instead of "just a box to tick".
Results
In 6 months' time, the development team continued to stay dedicated to unit testing, despite the immense workload, and we have doubled our test coverage in this time, reaching almost 40% across the board and covering a significant part of the business-critical funtionality.
Newly added code with test coverage was much faster to improve and extend, and potential problems were discovered much sooner, which has lead to more efficient and higher quality development work - and fewer task rejections by QA, because many errors were caught early on. This has become especially crucial when the team size was decreased from 4 to 1 developer + 1 new joiner. The practice of unit testing played a part in our velocity dropping by only 35% due to downsizing, from an average of 34 to 22 story points per sprint.
Another important benefit was that developers could improve their unit testing skills, and wrote better (less frail and more FIRST) unit tests than before.
We have also learned that it is possible to factor in unit test improvements without convincing product management to dedicate extra resources to this, but it required the continued unconditional support of the entire development team.
I hope this inspires you to get your team on board and improve your unit testing practices.
Thank you for reading!