ACCL: Architecting Highly Scalable Distributed Training Systems With Highly Efficient Collective Communication Library
IEEE Micro(2021)
摘要
Distributed systems have been widely adopted for deep neural networks model training. However, the scalability of distributed training systems is largely bounded by the communication cost. We design a highly efficient collective communication library, namely Alibaba Collective Communication Library (ACCL), to build distributed training systems with linear scalability. ACCL provides optimized algorithms to fully make use of heterogeneous interconnects simultaneously. And the experimental results show significant performance improvement.
更多查看译文
关键词
Servers,Training,Bandwidth,Routing,Fabrics,Payloads,Parallel algorithms
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要