Syntax Highlighting

Monday 8 December 2008

Measuring the effectiveness of unit testing

So this is supposed to be a blog about development, and indeed it is, but I am currently piloting unit testing within my team at work and wanted to post about my experiences so far.

You will be unsurprised to learn that in a project that currently has no formal unit testing the resistance to introducing a formalised unit testing process has been strong. Resistance has come from all corners; developers, managers and believe it or not, testers! I have tried to reduce the resistance by making it a voluntary process, which has worked so far. We have a small number of developers who are writing and maintaining tests and management seem to have been won over by the immediate feedback that unit testing provides.

We now have some support and need to take it to a wider audience, but in order to do this we have to prove that creating unit tests for code improves the quality of an application. You may think that this is a no-brainer, but a business case is required and so a business case I must create. I'm not the first, have a look at this article for more about convincing management of the benefits of unit testing.

A methodology to prove unit testing works.

This is the assertion I am trying to prove (or disprove!);

“Unit testing takes more time, but results in code with fewer defects which can be corrected in less time than if no test existed."

Our development work is broken down in to small, medium and large pieces of work. So my plan is to get two teams developing a piece of work of the same size. One is writing unit tests and the other is not. Both teams will record their development time with the second team recording the amount of time spent on unit testing as well. We will compare the output of the two teams against historic averages to offset anomalies.

A code coverage figure will also need to be calculated so that we can see how much of the development is covered. Time taken to develop will be compared against the number of defects which come out of each piece of work.

If anyone has any ideas or can pass on their experiences please do.