Unit Testing vs. Beta Testing

The paper “Unit Testing in Practice” (M. Ellims et al., ISSRE '04) had some interesting data on the effects of unit testing in three industrial software projects. They found that not only did unit testing detect large numbers of bugs, but just the activity of designing the unit tests found bugs. In fact, more bugs were found this way than by actually running the tests.

In two of the projects, unit testing was tacked on at the end. They found this reduced its effectiveness by about 50%. In other words, it appears that, of the bugs that would be found by unit testing, half of them would escape the rest of their process.