Examining Multi-Phased Online Crowdsourcing Contests Using Quality Level and Quality Ambiguity

In traditional crowdsourcing markets such as 99Designs for logo designs, every contest is stand-alone and serves as the sole source of information for participants to learn about the contest. However, crowdsourcing for software development is different, in that a software development project is divided into sequentially-related phases, and each phase is posted as a separate contest. With sequential interdependency between contests for different phases, uncertainty and reduction in uncertainty through information gained propagates from one contest to subsequent contests for the same software development project. This study represents the first attempt to analyze the relationships between outcomes of phased contests in crowdsourced software development. Our results indicate that the sequential interdependence is quantifiable through the significantly positive influence of the outcome of an earlier phase on the outcome of its subsequent phase. We further find that the effect of sequential interdependence between two phases is greater if the earlier phase involves less uncertainty. We also find that a software project, for which conceptualization – the most unstructured problem solving phase – is conducted on the crowdsourcing platform, exhibits higher sequential interdependency among phases than when conceptualization is not conducted on the platform. In addition to providing the first evidence of the impact of sequential interdependency among contests and contest uncertainty on contest outcomes, our findings also generate important insights on the design of software development contests.