Finding the Needles in the Haystack: Generating Legal Test Inputs for Object-Oriented Programs

A test input for an object-oriented program typically consists of a sequence of method calls that use the API defined by the pro- gram under test. Generating legal test inputs can be challenging because, for some programs, the set of legal method sequences is much smaller than the set of all possible sequences; without a for- mal specification of legal sequences, an input generator is bound to produce mostly illegal sequences. We propose a scalable technique that combines dynamic analy- sis with random testing to help an input generator create legal test inputs without a formal specification, even for programs in which most sequences are illegal. The technique uses an example execu- tion of the program to infer a model of legal call sequences, and uses the model to guide a random input generator towards legal but behaviorally-diverse sequences. We have implemented our technique for Java, in a tool called Palulu, and evaluated its effectiveness in creating legal inputs for real programs. Our experimental results indicate that the tech- nique is effective and scalable. Our preliminary evaluation indi- cates that the technique can quickly generate legal sequences for complex inputs: in a case study, Palulu created legal test inputs in seconds for a set of complex classes, for which it took an expert thirty minutes to generate a single legal input.