UniFaaS: Programming across Distributed Cyberinfrastructure with Federated Function Serving
arxiv(2024)
摘要
Modern scientific applications are increasingly decomposable into individual
functions that may be deployed across distributed and diverse
cyberinfrastructure such as supercomputers, clouds, and accelerators. Such
applications call for new approaches to programming, distributed execution, and
function-level management. We present UniFaaS, a parallel programming framework
that relies on a federated function-as-a-service (FaaS) model to enable
composition of distributed, scalable, and high-performance scientific
workflows, and to support fine-grained function-level management. UniFaaS
provides a unified programming interface to compose dynamic task graphs with
transparent wide-area data management. UniFaaS exploits an
observe-predict-decide approach to efficiently map workflow tasks to target
heterogeneous and dynamic resources. We propose a dynamic heterogeneity-aware
scheduling algorithm that employs a delay mechanism and a re-scheduling
mechanism to accommodate dynamic resource capacity. Our experiments show that
UniFaaS can efficiently execute workflows across computing resources with
minimal scheduling overhead. We show that UniFaaS can improve the performance
of a real-world drug screening workflow by as much as 22.99
additional 19.48
an additional 47.83
contrast to using a single cluster
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要