BlockCompass: A benchmarking platform for blockchain performance

Blockchain technology has gained momentum among researchers and other stakeholders due to its immutability and transparency. Several blockchain platforms with different kinds of consensus protocols have been proposed. However, this makes choosing and configuring such a platform, a non-trivial task. Several benchmarking tools have been presented to test the performance of blockchain solutions. However, these solutions are either limited to specific blockchain platforms or require complex configurations. Moreover, they tend to focus on transactional evaluation models, which may be counter-intuitive for longer-running instances under continuous workloads. In this work, we present BlockCompass, an all-inclusive blockchain benchmarking tool that can be easily configured and extended. We demonstrate how BlockCompass can evaluate the performance of a variety of blockchain platforms and configurations, including Ethereum Proof-of-Authority, Ethereum Proof-of-Work, Hyperledger Fabric Raft, Hyperledger Sawtooth with Proof-of-Elapsed-Time, Practical Byzantine Fault Tolerance and Raft consensus algorithms against workloads that continuously fluctuate over time. We also present the results of a usability study about the convenience and facility offered by BlockCompass in blockchain benchmarking.