Realizing the Full Potential of Heterogeneity through Processing in Memory

While many processing in memory (PIM) research studies demonstrate significant improvements in memory system energy efficiency, relatively little attention has been paid to the sources of overall energy efficiency of PIM systems. In this paper, we quantify the sources of energy efficiency of a GPU-based PIM design and show that selecting low-power operating points for the in-memory processors is an important aspect, accounting for a 1.9x improvement in energy efficiency compared to a mainstream implementation of the evaluated GPU design. Memory interface efficiency of PIM provides an additional 3.8x improvement over that. These results also demonstrate that, due to memory system inefficiencies, implementing high-performance and low-power heterogeneous cores on the same die attached to a conventional memory system can only realize a fraction of the overall improvement realized by PIM (52% in our study). While these results in general confirm conventional wisdom, we quantify the relative importance of these processor and memory efficiency factors across a wide range of benchmarks and encourage further research to enable and leverage the symbiosis between PIM and heterogeneous computing to further improve energy efficiency.