Introduction to Combinatorial Testing

Combinatorial testing of software analyzes interactions among variables using a very small number of tests. This advanced approach has demonstrated success in providing strong, low-cost testing in real-world situations. Introduction to Combinatorial Testing presents a complete self-contained tutorial on advanced combinatorial testing methods for real-world software.

The book introduces key concepts and procedures of combinatorial testing, explains how to use software tools for generating combinatorial tests, and shows how this approach can be integrated with existing practice. Detailed explanations and examples clarify how and why to use various techniques. Sections on cost and practical considerations describe tradeoffs and limitations that may impact resources or funding. While the authors introduce some of the theory and mathematics of combinatorial methods, readers can use the methods without in-depth knowledge of the underlying mathematics.

Accessible to undergraduate students and researchers in computer science and engineering, this book illustrates the practical application of combinatorial methods in software testing. Giving pointers to freely available tools and offering resources on a supplementary website, the book encourages readers to apply these methods in their own testing projects.

Table of Contents

Chapter 1 – Combinatorial Methods in Testing
Chapter 2 – Combinatorial Testing Applied
Chapter 3 – Configuration Testing
Chapter 4 – Input Testing
Chapter 5 – Test Parameter Analysis
Chapter 6 – Managing System State
Chapter 7 – Measuring Combinatorial Coverage
Chapter 8 – Test Suite Prioritization by Combinatorial Coverage
Chapter 9 – Combinatorial Testing and Random Test Generation
Chapter 10 – Sequence-Covering Arrays
Chapter 11 – Assertion-Based Testing
Chapter 12 – Model-Based Testing
Chapter 13 – Fault Localization
Chapter 14 – Evolution from Design of Experiments
Chapter 15 – Algorithms for Covering Array Construction

Appendix A: Mathematics Review
Appendix B: Empirical Data on Software Failures
Appendix C: Resources for Combinatorial Testing
Appendix D: Test Tools