Savery: A Framework for the Assessment and Comparison of Mobile Development Tools.

Different types of mobile development tools and frameworks to build web applications are constantly and rapidly emerging, making it difficult for developers to choose an appropriate one for their needs. In this work, we propose a novel framework to assess and benchmark development tools according to performance (and potentially dependability and security). The framework defines the components and procedures required to define and execute the benchmarks. To demonstrate the applicability of the proposed framework, we implemented it for a concrete benchmark focused on assessing the performance of a representative application developed by different development tools. For this, we defined a representative set of features for the application and a set of relevant metrics to characterize their performance. The benchmarking campaign is executed on each application in an automated manner. The metrics are collected during the execution of the campaign and the results are then analyzed to compare the applications, their features, and the development tools that produced them. The results show that the same applications implemented in different frameworks are different in terms of software metrics and build time and show different performance when executing different functions (features) in terms of computation time, memory, and CPU usage. This reveals the effectiveness of the approach and the framework proposed to assess and compare the mobile development tools.