Java Performance Mysteries

While assessing software performance quality in the cloud, we noticed some significant performance variation of several Java applications. At a first glance, they looked like mysteries. To isolate the variation due to cloud, system and software configurations, we designed a set of experiments and collected set of software performance data. We analyzed the data to identify the sources of Java performance variation. Our experience in measuring Java performance may help attendees in selecting the trade-offs in software configurations and load testing tool configurations to obtain the software quality measurements they need. The contributions of this paper are (1) Observing Java performance mysteries in the cloud, (2) Identifying the sources of performance mysteries, and (3) Obtaining optimal and reproducible performance data.