Problem #4: What Techniques To Use?
Testing experts rarely agree!
- (e.g., black-box vs. white-box or system-level vs. unit)
Other than the subsumes hierarchy, most evidence that relates defects found per technique to costs of that technique is anecdotal
Marginal evidence correlating testing methods and the types of errors they detect
- Reason: Taboo for organizations to reveal real error/fault and test-effectiveness data. ("Dirty laundry" syndrome)
- NIST: Error, Fault, Failure Database; little or no data collected after 2 years