Remove the automated tests that do not provide any value

Most blogs on test automation are about how to add more and more tests to your automation suite, but rarely does anyone mention that you also should consider removing automated tests, especially those that do not provide any value.

If you are on a new and relatively fresh project, you probably focus most on adding automated tests across different layers, focusing on automating regression testing certain parts of your service that you create. As your service mature, you probably have built up a good chunk of automated tests and you keep on adding more and more automated tests as new parts are added. Now instead of 10-15 minutes, your test suite perhaps need 20-30 minutes to run, and it becomes heavier and heavier to regression test new code. (This can probably be solved by other means of course, but consider this an example, there could be more reasons why you have a test-heavy delivery pipeline)

Seldom does one consider the possibility of removing tests, not only in order to reduce execution time, and delivery time, but in order to re-check what is actually the value these automated tests give us. Previously some of these tests could have provided much value for the team, but as the service matured, and evolved, are these tests covered by other tests, or do we need to check all part individually?

What if the tests are unstable, and fail often? If we do not trust them, why should we keep them? Are they just there so that we can say we have covered x% of the entire code? Unstable tests are perhaps the worst there is, as they not only take time to execute, usually we also need to spend time to investigate why did they fail, but if it ends in a comment like "those are usually failing, just ignore them and move on", or something similar, then that is a pretty good sign that you should definitely remove them.

So consider from time to time what do we actually regression (automate) test, and can we remove some of these tests, the same way as we remove obsolete and unused code.


Comments

  1. Great points!

    Its like when deciding to automate something in the first place - one has to also consider the maintenance costs post implementation.

    ReplyDelete

Post a Comment

Popular posts from this blog

How do I stay "up to date" on software testing related stuff?

Does the language we speak shape the way we think?