Decentralized Federated Learning on a High-performance Computing Platform: Blockchain-based Security and Optimization
DOI:
https://doi.org/10.13052/jwe1540-9589.2483Keywords:
HPC, federated learning, blockchain, simulationAbstract
As various artificial intelligence (AI)-related services emerge, attempts to apply AI models to industrial and service fields are ongoing. In preparation for the increase in demand for such AI services, the importance of data required to create AI models is increasing. As data for applying services is a key element in training AI models and improving performance, the privacy and security of data have also recently emerged as significant issues. Although various AI learning platforms exist, users must share data that they have personally created and refined. Consequently, users who do not want to share personal data tend to be reluctant to access and use these platforms. Therefore, in this paper, we built a platform that can create and share specific AI models without sharing data. We designed and built a platform that can perform blockchain-based federated learning to ensure the authenticity and privacy of the user’s learning model and global model, allowing the learning of the final result model according to the sharing model of each dataset during each learning session. Each user learns based on their personal dataset, and the weights are integrated based on the learned models to create a single global model. The contribution of each user is measured based on the blockchain-based model creation, leading to the development of a high-performance AI model. We conducted experiments using the MNIST and COVID-19 datasets, focusing on data independence. The results showed that significant results could be achieved with just 10–20% data sharing. These experimental results confirmed that federated learning is possible in an environment where data independence is maintained and individual users do not share data.
Downloads
References
Shokri, Reza, and Vitaly Shmatikov, “Privacy-preserving deep learning.” Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015, pp. 1310–1321.
Ryffel, Theo, et al., “A generic framework for privacy preserving deep learning.” arXiv preprint arXiv:1811.04017, 2018, pp. 1–5.
Tanuwidjaja, Harry Chandra, et al., “Privacy-preserving deep learning on machine learning as a service—a comprehensive survey.” IEEE Access 8, 2020, pp. 167425–167447.
Aono, Yoshinori, et al., “Privacy-preserving deep learning via additively homomorphic encryption.” IEEE Transactions on Information Forensics and Security 13, no. 5, 2017, pp. 1333–1345.
Kaplan, Jared, et al., “Scaling laws for neural language models.” arXiv preprint arXiv:2001.08361, 2020, pp. 1–30.
Shen, Jiahao, et al., “Blockchain-based distributed multi-agent reinforcement learning for collaborative multi-object tracking framework.” IEEE Transactions on Computers, 2023, pp. 778–788.
Qiu, Xiaoyu, et al., “Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing.” IEEE Transactions on Vehicular Technology 68, no. 8, 2019, pp. 8050–8062.
Gadiraju, Divija Swetha, V. Lalitha, and Vaneet Aggarwal, “An optimization framework based on deep reinforcement learning approaches for PRISM blockchain.” IEEE Transactions on Services Computing 16, no. 4, 2023, pp. 2451–2461.
Chowdhury, Sujit, Arnab Mukherjee, and Raju Halder, “FedRLChain: Secure federated deep reinforcement learning with blockchain.” IEEE Transactions on Services Computing, 2023, pp. 3865–3878.
Cam, Nguyen Tan, and Vu Tuan Kiet, “FlwrBC: Incentive mechanism design for federated learning by using blockchain.” IEEE Access, 2023, pp. 107855–107866.
Goh, Eunsu, et al., “Blockchain-enabled federated learning: A reference architecture design, implementation, and verification.” IEEE Access, 2023, pp. 145747–145762.
Han, Boyuan, et al., “Research on blockchain-based decentralized federated learning.” 2023 International Conference on Computer Applications Technology (CCAT). IEEE, 2023, pp. 23–29.
Nilsson, Adrian, et al., “A performance evaluation of federated learning algorithms.” Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning, 2018, pp. 1–9.
Zhou, Yuhao, Qing Ye, and Jiancheng Lv, “Communication-efficient federated learning with compensated Overlap-FedAvg.” IEEE Transactions on Parallel and Distributed Systems 33, no. 1, 2021, pp. 192–205.
Casella, Bruno, and Samuele Fonio, “Architecture-based FedAvg for vertical federated learning.” Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud Computing, 2023, pp. 1–6.
Mills, Jed, Jia Hu, and Geyong Min, “Communication-efficient federated learning for wireless edge intelligence in IoT.” IEEE Internet of Things Journal 7, no. 7, 2019, pp. 5986–5994.
Li, Tian, et al., “Federated optimization in heterogeneous networks.” Proceedings of Machine Learning and Systems 2, 2020, pp. 429–450.
Nguyen, Hung T., et al., “Fast-convergent federated learning.” IEEE Journal on Selected Areas in Communications 39, no. 1, 2020, pp. 201–218.
Zahri, Sofia, Hajar Bennouri, and Ahmed M. Abdelmoniem, “An empirical study of efficiency and privacy of federated learning algorithms.” arXiv preprint arXiv:2312.15375, 2023, pp. 1–7.
Geng, Jiahui, et al., “Improved gradient inversion attacks and defenses in federated learning.” IEEE Transactions on Big Data, 2023, pp. 839–850.
Kholod, Ivan, et al., “Open-source federated learning frameworks for IoT: A comparative review and analysis.” Sensors 21, no. 1, 2020, pp. 167.
Wei, Wenqi, et al., “Gradient-leakage resilient federated learning.” 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS). IEEE, 2021, pp. 797–807.
Li, Zengpeng, Vishal Sharma, and Saraju P. Mohanty, “Preserving data privacy via federated learning: Challenges and solutions.” IEEE Consumer Electronics Magazine 9, no. 3, 2020, pp. 8–16.
Wang, Yingcheng, Songtao Guo, and Dewen Qiao, “FedSG: Subgraph federated learning on multiple non-IID graphs.” 2023 19th International Conference on Mobility, Sensing and Networking (MSN). IEEE, 2023, pp. 504–511.
Asad, Muhammad, Ahmed Moustafa, and Takayuki Ito, “FedOPT: Towards communication efficiency and privacy preservation in federated learning.” Applied Sciences 10, no. 8, 2020, pp. 1–17.
Ahmed, Syed Thouheed, et al., “FedOPT: Federated learning-based heterogeneous resource recommendation and optimization for edge computing.” Soft Computing, 2024, pp. 1–12.
Wang, Hongyi, et al., “Federated learning with matched averaging.” arXiv preprint arXiv:2002.06440, 2020, pp. 1–16.
Sannara, E. K., et al., “A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison.” 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 2021, pp. 1–10.
Ek, Sannara, et al., “Evaluation of federated learning aggregation algorithms: Application to human activity recognition.” Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, 2020, pp. 638–643.
Ek, Sannara, et al., “Evaluation and comparison of federated learning algorithms for human activity recognition on smartphones.” Pervasive and Mobile Computing 87, 2022, pp. 14453–14464.
Sun, Yan, et al. “Efficient federated learning via local adaptive amended optimizer with linear speedup.” IEEE Transactions on Pattern Analysis and Machine Intelligence 45.12, 2023, pp. 14453–14464.
Reddi, Sashank, et al., “Adaptive federated optimization.” arXiv preprint arXiv:2003.00295, 2020. pp. 1–9.
Zhao, Yue, et al., “Federated learning with non-IID data.” arXiv preprint arXiv:1806.00582, 2018, pp. 1–12.
Li, Xiang, et al., “On the convergence of FedAvg on non-IID data.” arXiv preprint arXiv:1907.02189, 2019, pp. 1–26.
Ma, Xiaodong, et al., “A state-of-the-art survey on solving non-IID data in federated learning.” Future Generation Computer Systems 135, 2022, pp. 244–258.
Zhu, Hangyu, et al., “Federated learning on non-IID data: A survey.” Neurocomputing 465, 2021, pp. 371–390.
Calvin, Christophe, and France Boillod-Cerneux. “HPC and Data: When Two Becomes One.” Turbulence and Interactions. Cham: Springer International Publishing, 2018, pp. 14–25.
Yuhang Chen, Wenke Huang, and Mang Ye, “Fair federated learning under domain skew with local consistency and domain diversity”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 12077–12086.
Wang, Zheng, et al. “Federated Learning with Domain Shift Eraser.” Proceedings of the Computer Vision and Pattern Recognition Conference(CVPR), 2025, pp. 4978–4987.
