Efficacy, Efficiency and Effectiveness of SMarty-based Software Product Line Inspection Techniques: a Controlled Quasi-Experiment.

Software quality is impacted by several factors, including the quality of the artifacts generated during the development process. Especially for Software Product Lines (SPL), as artifacts are reused for various specific products, a defect can be spread out over an SPL. Our research group has previously created two inspection techniques for UML-based SPLs: a checklist-based, named SMarty-Check, and a perspective-based, named SMartyPerspective. Seeking to understand the efficiency, efficacy, and effectiveness of such techniques, we carried out a controlled quasi-experiment with 16 participants from the Software Engineering area. It aimed at inspecting feature diagrams and use case, class, component, and sequence diagrams designed using the SMarty approach for UML-based variability support. We also considered ad hoc inspections in such a study. The results of this experiment provide incipient evidence of no statistical difference among the compared techniques for efficiency, efficacy, and effectiveness.