Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
CoRR(2024)
Abstract
Statistical heterogeneity of clients' local data is an important
characteristic in federated learning, motivating personalized algorithms
tailored to the local data statistics. Though there has been a plethora of
algorithms proposed for personalized supervised learning, discovering the
structure of local data through personalized unsupervised learning is less
explored. We initiate a systematic study of such personalized unsupervised
learning by developing algorithms based on optimization criteria inspired by a
hierarchical Bayesian statistical framework. We develop adaptive algorithms
that discover the balance between using limited local data and collaborative
information. We do this in the context of two unsupervised learning tasks:
personalized dimensionality reduction and personalized diffusion models. We
develop convergence analyses for our adaptive algorithms which illustrate the
dependence on problem parameters (e.g., heterogeneity, local sample size). We
also develop a theoretical framework for personalized diffusion models, which
shows the benefits of collaboration even under heterogeneity. We finally
evaluate our proposed algorithms using synthetic and real data, demonstrating
the effective sample amplification for personalized tasks, induced through
collaboration, despite data heterogeneity.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined