Benchmarking Sparse Matrix-Vector Multiply in Five Minutes

We present a benchmark for evaluating the perfor- mance of Sparse matrix-dense vector multiply (abbreviated as SpMV) on scalar uniprocessor machines. Though SpMV is an important kernel in scientific computation, there are currently no adequate benchmarks for measuring its performance across many platforms. Our work serves as a reliable predictor of expected SpMV performance across many platforms, and takes no more than five minutes to obtain its results. I. INTRODUCTION Sparse matrix-dense vector multiply (SpMV) is a common operation in scientific codes. It is especially prevalent in iterative methods to solve linear systems. Given this, it would be very convenient for consumers to have a convenient way of knowing which machine to buy for their calculations, and for vendors to know how well their machines perform. There are currently no convenient ways for vendors to know how well their machines perform SpMV. The current standard method for ranking computers' ability to perform scientific compuatations, the Top 500 List (9), uses only the LINPACK benchmark (8). LINPACK measures the speed of solution of a system of linear equations, which is not representative of all the operations that are performed in scientific computing. There is a benchmark suite under development called the High Performance Computing Challenge Suite (HPCC) that seeks to remedy this (5). The HPCC suite contains benchmarks that seek to measure computers' performance in performing several different operations, including LINPACK. The benchmark we will present here is proposed for inclu- sion into this suite, as none of the other benchmarks in it are suited for approximating the performance of SpMV. We will see why in the next section. One requirement for inclusion in the HPCC suite is a short run-time, which explains our goal of running in five minutes.