Self-Organizing Clustering for Federated Learning

The emergence of widely utilized IoT devices and the vast amount of data they host provide excellent candidates for machine learning and deep learning applications. However, the distributed nature of these devices, the growing emphasis on user privacy, the sensitivity of users' and organizations' data, as well as regulations that prohibit data transfer to centralized data centers, have necessitated the adoption of new privacy-preserving machine learning techniques. Federated learning, a recently introduced distributed paradigm to train a model in a distributed system of individual user devices, has emerged in response to such essential needs. Within this study, we present a new architecture framework for federated learning that aims to mitigate the risk of bottlenecks and facilitate learning in a more distributed manner than the conventional central-server-based approach. By employing our fully decentralized clustering algorithm, nodes present in the network assume the roles of aggregating servers or trainers in an adaptive self-organizing manner. Through a series of experimental results, leveraging our clustering algorithm, we established two different two-layer federated learning frameworks and compared their performances with traditional centralized and fully decentralized federated learning approaches. The results unequivocally demonstrate the promise of the proposed algorithm, showcasing its potential without significantly compromising accuracy. This innovative framework introduces a distributed learning opportunity for federated learning, mitigating communication volume on the server and reducing the risk of network failures or bottlenecks. Our analysis also reveals a remarkable reduction in message exchange among network entities, presenting a substantial improvement over decentralized federated learning.