Assessing the Risk of Software Development in Agile Methodologies Using Simulation

Agile methodologies aim to reduce software development risk using short iterations, feature-driven development, continuous integration, testing automation, and other practices. However, the risk of project failure or time and budget overruns is still a relevant problem. This paper aims to present and discuss a new approach to model some key risk factors in agile development, using software process simulation modeling (SPSM), which can complement other approaches, and whose usage is particularly suited for agile development. We introduce a new approach to modeling some key risk factors – namely project duration, number of implemented issues, and key statistics of issue completion time – using a simulator of agile development, which we developed for this purpose. The approach includes modeling the agile process, gathering data from the tool used for project management, and performing Monte Carlo simulations of the process, to get insights about the expected time and effort to complete the project, and about their distributions. The model’s parameters that can cause risk are errors in effort estimation of the features to develop, variations in developers’ assignment to these features, impediments related to developers’ availability and work completion. To validate the simulator, and to demonstrate how the method can be used, we analyzed three open-source projects, gathering their data from JIRA repositories. We ran Monte Carlo simulations of these projects, showing that the simulator can well approximate the progress of the real project, then varying the identified risk factors and statistically evaluating their effects on the risk parameters. The proposed approach is relevant for project managers, being able to quantitatively evaluate the risks, provided that the process and the project’s data are properly modeled and gathered. As for the future, we are working to improve our risk assessment method, evaluating it on more case studies, scaling the model from a single team to multiple teams involved in one or more projects.