Beyond Combinatorial Interaction Testing: On the need for transition testing in dynamically adaptive context-aware systems

Testing context-aware systems is hard due to the combinatorial explosion of possible contexts and corresponding features that such systems can adapt to dynamically. Combinatorial interaction testing (CIT) has been successfully applied to generate small yet representative test suites covering all valid combinations of pairs of contexts and features of such systems. Such test suites can then be used to verify that the system behaves correctly in most cases. We claim that CIT may miss certain relevant scenarios that exhibit certain kinds of errors that only occur for particular sequences of context or features (de)activations (transitions). To generate test suites that capture such errors as well, we propose to complement the CIT approach with a new technique called Combinatorial Transition Testing (CTT). The combination of CIT and CTT presents some challenges but provides a better coverage of scenarios to be tested to find relevant errors in dynamically adaptive context-aware systems.