Impact of Test Reviews

We are under pressure with short deadlines and releases in <<Internet time>> already now. This leads to problems for test execution, an activity being done just before release. A natural focus is, therefore, to replace some of the need for testing, and some of the time consuming tasks during test execution, by other defect removal activities. There are two main answers to this: Static analysis and reviews.

There are different kinds of reviews, formal and informal. The most effective ones, for high defect detection rates, are inspections. On the other side of the spectrum are more informal and less effective techniques, like author reader cycles. Many organizations are not mature or disciplined enough to use formal reviews or inspections. And inspections take time, due to the need to coordinate work from several people, and organizing meetings.

This paper focuses on the less formal (and less effective) review techniques and shows how even small and immature organizations can benefit from easy to implement review techniques. The idea is that any review is better than no review at all.

Inspections

Inspections and reviews are a must for producing reliable software. They find errors at a fraction of the cost of testing, they tend to stabilize development, they spread knowledge inside the organization, and they grossly reduce the defect content in applications when they enter the test execution phase.
Less defects in software lead to less trouble in test execution, as less tests are blocked by defects, less tests need to be redone to isolate the causes of failures, and less regression testing is needed because less defects are corrected. Testing is still needed to finally measure quality, but ideally testing should find few or no defects.

Informal Reviews

Inspections require one or two meetings: An overview meeting and a defect-logging meeting. Organizing meetings always incurs trouble, and delays, as one has to find a point of time where all participants have time. This delays the feedback from an inspection. Inspections also require a trained inspection leader (moderator) and an infrastructure consisting of checklists, forms, standards, an inspection database, etc. Small or immature organizations don't have this infrastructure. Inspections also require that earlier documents have passed inspection. Such documents might not even exist

Review and Testing

The person who is responsible for testing the application should be interested in the quality of the specification or design. Thus, this person would be a good reviewer, if otherwise qualified. Actually, making test specifications can be combined with a review of the design or specification. This leads to an organization where two people could tightly interact. One person is responsible to the design and code, the other one for planning, specifying and preparing the test. The two people can work in parallel and review each other's work. This again saves some overhead, as both the reviewer and the tester have to understand the specification and design. A "buddy system" can be created where every person has a fixed other person who is responsible for reviewing and test of his/her work. Everyone would then use part of his or her time to be the "buddy" of someone else. This has successfully used in an informal way by several Scandinavian organizations. Such a system is, amongst other techniques, implemented in "extreme programming".