SLMFed: A Stage-Based and Layer-Wise Mechanism for Incremental Federated Learning to Assist Dynamic and Ubiquitous IoT

Along with the vast application of Internet of Things (IoT) and the ever-growing concerns about data protection, a novel type of learning, named incremental federated learning (IFL), is rising to further elevate the intelligence and quality of various IoT systems and services by consistently learning and updating their models, e.g., deep neural networks, in dynamic contexts, where clients and data can increase and accumulate gradually. Since IFL is still in its infancy, to overcome its emerging challenges as represented in 1) periodic learning about how to initialize the model update rationally to avoid catastrophic performance dropping, and 2) iterative learning about how to update the model cost-efficiently to remedy overlearning on duplicated information, this paper proposes a stage-based and layer-wise mechanism for IFL, called SLMFed, in which, the periodic learning is managed by a stage transition and client selection strategy to trigger model update according to the quantitative and qualitative changes on clients, data, and user experience, and the iterative learning is enhanced by an adaptive layer uploading and aggregation strategy to update the global model by measuring representational consistencies and information richness of local model layers. As shown by the evaluation results, SLMFed can not only stabilize the learning across various learning stages but also boost the performance in terms of learning accuracy, communication cost, and stage contribution by about 32.09%, 105.94%, and 22.02%, respectively.