One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversity
CoRR(2024)
Abstract
Traditional federated learning mainly focuses on parallel settings (PFL),
which can suffer significant communication and computation costs. In contrast,
one-shot and sequential federated learning (SFL) have emerged as innovative
paradigms to alleviate these costs. However, the issue of non-IID (Independent
and Identically Distributed) data persists as a significant challenge in
one-shot and SFL settings, exacerbated by the restricted communication between
clients. In this paper, we improve the one-shot sequential federated learning
for non-IID data by proposing a local model diversity-enhancing strategy.
Specifically, to leverage the potential of local model diversity for improving
model performance, we introduce a local model pool for each client that
comprises diverse models generated during local training, and propose two
distance measurements to further enhance the model diversity and mitigate the
effect of non-IID data. Consequently, our proposed framework can improve the
global model performance while maintaining low communication costs. Extensive
experiments demonstrate that our method exhibits superior performance to
existing one-shot PFL methods and achieves better accuracy compared with
state-of-the-art one-shot SFL methods on both label-skew and domain-shift tasks
(e.g., 6
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined