REMU: Enabling Cost-Effective Checkpointing and Deterministic Replay in FPGA-based Emulation

Left-shifted integration and evaluation of hardware and software design are increasingly crucial in pre-silicon validation of processor-centric computing systems. With the inherent cycle-accurate deployment of target processor design in programmable logic fabrics, FPGA-based emulation has attracted academic attention for early-stage performance evaluations. However, it is difficult to conduct system-level inspection and debugging within open-source academic FPGA-based emulation frameworks due to the limited HW-SW visibility at run-time. To fill such a gap, we present REMU, an FPGA-based emulation framework enabling hardware checkpointing and deterministic replay to acquire bit-accurate visibility of target processors and other system components. Specially, we first employ a cost-effective scan-chain insertion method and related implementation strategies within an optimized open-source synthesis tool for status capturing of the emulated circuit primitives. Then, we introduce mechanisms in the design of emulated memory and I/O peripheral components to precisely describe behaviors and ensure deterministic replay of system-level interactions. Experimental results show that REMU drastically speeds up the scan-chain insertion flow by 1.6x-32.7x, and the proposed mechanisms for deterministic replay in the emulated external memory introduce negligible overhead in FPGA resource utilization.