Accurate and Scalable Many-Node Simulation
CoRR(2024)
摘要
Accurate performance estimation of future many-node machines is challenging
because it requires detailed simulation models of both node and network.
However, simulating the full system in detail is unfeasible in terms of compute
and memory resources. State-of-the-art techniques use a two-phase approach that
combines detailed simulation of a single node with network-only simulation of
the full system. We show that these techniques, where the detailed node
simulation is done in isolation, are inaccurate because they ignore two
important node-level effects: compute time variability, and inter-node
communication.
We propose a novel three-stage simulation method to allow scalable and
accurate many-node simulation, combining native profiling, detailed node
simulation and high-level network simulation. By including timing variability
and the impact of external nodes, our method leads to more accurate estimates.
We validate our technique against measurements on a multi-node cluster, and
report an average 6.7
average 27
overhead are ignored. At higher node counts, the prediction error of ignoring
variable timings and scaling overhead continues to increase compared to our
technique, and may lead to selecting the wrong optimal cluster configuration.
Using our technique, we are able to accurately project performance to
thousands of nodes within a day of simulation time, using only a single or a
few simulation hosts. Our method can be used to quickly explore large many-node
design spaces, including node micro-architecture, node count and network
configuration.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要