Privacy and Performance in Virtual Reality: The Advantages of Federated Learning in Collaborative Environments∗
DOI:
https://doi.org/10.13052/jwe1540-9589.2382Keywords:
Virtual reality, federated learning, neural networks, training, cloud computingAbstract
Federated Learning has emerged as a promising approach for maintaining data privacy across distributed environments, enabling training on a diverse range of devices from high-performance servers to low-power gadgets. Despite its potential, managing numerous data sources can strain these devices, particularly those with limited capabilities, leading to increased latency. This is especially critical in virtual reality, where real-time responsiveness is crucial due to the need for constant data connectivity. Historically, virtual reality systems have relied on tethered computer setups, restricting their flexibility and the benefits of wireless technology. However, recent advancements have enhanced the computational power of VR devices, allowing them to perform certain tasks independently. This work explores the feasibility of training a neural network on VR devices, using a federated learning approach, to develop a collaborative model aggregated and stored in the cloud. The goal is to assess the computational demands and explore the potential and constraints of leveraging VR devices for artificial intelligence applications.
Downloads
References
AbdulRahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., Guizani, M.: A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE Internet of Things Journal 8(7), 5476–5497 (2020)
Aledhari, M., Razzak, R., Parizi, R.M., Saeed, F.: Federated learning: A survey on enabling technologies, protocols, and applications. IEEE Access 8, 140699–140725 (2020)
Angelov, V., Petkov, E., Shipkovenski, G., Kalushkov, T.: Modern virtual reality headsets. In: 2020 International congress on human-computer interaction, optimization and robotic applications (HORA). pp. 1–5. IEEE (2020)
Bejani, M.M., Ghatee, M.: A systematic review on overfitting control in shallow and deep neural networks. Artificial Intelligence Review 54(8), 6391–6438 (2021)
Bhugaonkar, K., Bhugaonkar, R., Masne, N.: The trend of metaverse and augmented & virtual reality extending to the healthcare system. Cureus 14(9) (2022)
Brecko, A., Kajati, E., Koziorek, J., Zolotova, I.: Federated learning for edge computing: A survey. Applied Sciences 12(18) (2022), https://www.mdpi.com/2076-3417/12/18/9124
Chen, M., Semiari, O., Saad, W., Liu, X., Yin, C.: Federated deep learning for immersive virtual reality over wireless networks. In: 2019 IEEE Global Communications Conference (GLOBECOM). pp. 1–6 (2019). https://doi.org/10.1109/GLOBECOM38437.2019.9013419
Chen, Y., Huang, S., Gan, W., Huang, G., Wu, Y.: Federated learning for metaverse: A survey. In: Companion Proceedings of the ACM Web Conference 2023. p. 1151–1160. WWW ’23 Companion, Association for Computing Machinery, New York, NY, USA (2023), https://doi.org/10.1145/3543873.3587584
Cwierz, H., Díaz-Barrancas, F., Llinás, J.G., Pardo, P.J.: On the validity of virtual reality applications for professional use: A case study on color vision research and diagnosis. IEEE Access 9, 138215–138224 (2021)
Deng, L.: The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine 29(6), 141–142 (2012)
Díaz-Barrancas, F., Flores-Martin, D., Berrocal, J.: Training a neural network on virtual reality devices: Challenges and limitations. In: 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). pp. 787–788. IEEE (2024)
Ding, Y., Li, Y., Cheng, L.: Application of internet of things and virtual reality technology in college physical education. Ieee Access 8, 96065–96074 (2020)
Dohan, M., Mu, M., Ajit, S., Hill, G.: Real-walk modelling: deep learning model for user mobility in virtual reality. Multimedia Systems 30(1), 44 (2024)
Elor, A., Kurniawan, S.: Deep reinforcement learning in immersive virtual reality exergame for agent movement guidance. In: 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH). pp. 1–7 (2020). https://doi.org/10.1109/SeGAH49190.2020.9201901
Flores-Martin, D., Berrocal, J., Garcia-Alonso, J., Murillo, J.M.: Towards a runtime devices adaptation in a multi-device environment based on people’s needs. In: 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). pp. 304–309. IEEE (2019)
Flores-Martin, D., Galán-Jiménez, J., Berrocal, J., M. Murillo, J.: An analysis about federated learning in low-powerful devices. In: Proceedings of the 1st International Workshop on Middleware for the Computing Continuum. pp. 7–11 (2023)
Freina, L., Ott, M.: A literature review on immersive virtual reality in education: state of the art and perspectives. In: The international scientific conference elearning and software for education. vol. 1, pp. 10–1007 (2015)
Gheibi, O., Weyns, D., Quin, F.: Applying machine learning in self-adaptive systems: A systematic literature review. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 15(3), 1–37 (2021)
Khan, L.U., Saad, W., Han, Z., Hossain, E., Hong, C.S.: Federated learning for internet of things: Recent advances, taxonomy, and open challenges. IEEE Communications Surveys & Tutorials 23(3), 1759–1799 (2021)
Li, L., Fan, Y., Tse, M., Lin, K.Y.: A review of applications in federated learning. Computers & Industrial Engineering 149, 106854 (2020)
Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine 37(3), 50–60 (2020)
Mäkinen, H., Haavisto, E., Havola, S., Koivisto, J.M.: User experiences of virtual reality technologies for healthcare in learning: an integrative review. Behaviour & Information Technology 41(1), 1–17 (2022)
Makransky, G., Borre-Gude, S., Mayer, R.E.: Motivational and cognitive benefits of training in immersive virtual reality based on multiple assessments. Journal of Computer Assisted Learning 35(6), 691–707 (2019)
Malik, A.A., Masood, T., Bilberg, A.: Virtual reality in manufacturing: immersive and collaborative artificial-reality in design of human-robot workspace. International Journal of Computer Integrated Manufacturing 33(1), 22–37 (2020)
Mevlevioğlu, D., Tabirca, S., Murphy, D.: Real-time classification of anxiety in virtual reality therapy using biosensors and a convolutional neural network. Biosensors 14(3) (2024). https://doi.org/10.3390/bios14030131, https://www.mdpi.com/2079-6374/14/3/131
Naraha, T., Akimoto, K., Yairi, I.E.: Survey of the vr environment for deep learning model development. In: Annual Conference of the Japanese Society for Artificial Intelligence. pp. 154–164. Springer (2021)
Qiao, D., Qian, L., Guo, S., Zhao, J., Zhou, P.: AMFL: Resource-efficient adaptive metaverse-based federated learning for the human-centric augmented reality applications. IEEE Trans. Neural Netw. Learn. Syst. PP, 1–15 (Jun 2024)
Rentero-Trejo, R., Flores-Martín, D., Galán-Jiménez, J., García-Alonso, J., Murillo, J.M., Berrocal, J.: Using federated learning to achieve proactive context-aware iot environments. Journal of web engineering 21(1), 53–74 (2022)
Scavarelli, A., Arya, A., Teather, R.J.: Virtual reality and augmented reality in social learning spaces: a literature review. Virtual Reality 25, 257–277 (2021)
Sharma, G., Chandra, S., Venkatraman, S., Mittal, A., Singh, V.: Artificial neural network in virtual reality: A survey. International Journal of Virtual Reality 15(2), 44–52 (2016)
Xie, B., Liu, H., Alghofaili, R., Zhang, Y., Jiang, Y., Lobo, F.D., Li, C., Li, W., Huang, H., Akdere, M., et al.: A review on virtual reality skill training applications. Frontiers in Virtual Reality 2, 645153 (2021)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 10(2), 1–19 (2019)
Zhang, T., Gao, L., He, C., Zhang, M., Krishnamachari, B., Avestimehr, A.S.: Federated learning for the internet of things: Applications, challenges, and opportunities. IEEE Internet of Things Magazine 5(1), 24–29 (2022)
Zhao, J., Yin, J., Shi, Y., Qiao, L., Ma, G.: User entertainment experience analysis of artificial intelligence entertainment robots based on convolutional neural networks in park plant landscape design. Entertainment Computing 52, 100817 (2025)
Zhao, Y., Liu, S., et al.: A deep learning model with virtual reality technology for second language acquisition. Mobile Information Systems 2022 (2022)
Zhou, X., Liu, C., Zhao, J.: Resource allocation of federated learning for the metaverse with mobile augmented reality. IEEE Transactions on Wireless Communications (2023)
