An Effective Scheme to Accelerate NeRF for Web Applications Using Hash-based Caching and Precomputed Features
DOI:
https://doi.org/10.13052/jwe1540-9589.2376Keywords:
Neural radiance field, multiresolution hash encoding, 3D reconstruction, cacheAbstract
In recent years, 3D reconstruction and rendering technologies have become increasingly important in various web-based applications within the field of web technology. In particular, with the emergence of technologies such as WebGL and WebGPU, which enable real-time 3D content rendering in web browsers, immersive experiences and interactions on the web have been significantly enhanced. These technologies are widely used in applications such as 3D visualization of virtual products or 3D exploration of building interiors on real estate websites. Through these advancements, users can experience 3D content directly in their browsers without the need to install additional software, greatly expanding the possibilities of the web. Amidst this trend, the neural radiance field (NeRF) has garnered attention as a cutting-edge technology that improves the accuracy of 3D reconstruction and rendering.
NeRF is a technique widely used in computer vision and graphics for reconstructing 3D spaces from 2D images taken from multiple viewpoints. By predicting the color and density of each pixel, NeRF captures the complex 3D structure and optical properties of a scene, enabling highly accurate 3D reconstructions. However, NeRF’s primary limitation is the time-consuming nature of both the training and inference processes. Research efforts to address this issue have focused on two key areas: optimizing network architectures and training procedures to accelerate scene learning, and improving inference speed for faster rendering. While progress has been made in enhancing training speed, challenges remain in improving the inference process.
To address these limitations, we propose a two-step approach to significantly improve NeRF’s performance. First, we optimize the training phase through a multi-resolution hash encoding technique, reducing the computational complexity and speeding up the learning process. Second, we accelerate the inference phase by caching the input data of the NeRF MLP, which allows for faster rendering without sacrificing quality. Our experimental results demonstrate that this approach reduces training time by 68.42% and increases inference speed by 98.18%.
Downloads
References
Mildenhall, Ben, et al. “Nerf: Representing scenes as neural radiance fields for view synthesis.” Communications of the ACM 65.1 (2021): 99–106.
Wang, Guangming, et al. “NeRF in Robotics: A Survey.” arXiv preprint arXiv:2405.01333 (2024).
Gao, Ruicheng, and Yue Qi. “A Brief Review on Differentiable Rendering: Recent Advances and Challenges.” Electronics 13.17 (2024): 3546.
Cai, Jintong, and Huimin Lu. “NeRF-based Multi-View Synthesis Techniques: A Survey.” 2024 International Wireless Communications and Mobile Computing (IWCMC). IEEE, 2024.
Zhu, Fang, et al. “Deep review and analysis of recent nerfs.” APSIPA Transactions on Signal and Information Processing 12.1 (2023).
Rabby, A. K. M., and Chengcui Zhang. “BeyondPixels: A comprehensive review of the evolution of neural radiance fields.” arXiv preprint arXiv:2306.03000 (2023).
Jin, Yiwei, Diqiong Jiang, and Ming Cai. “3d reconstruction using deep learning: a survey.” Communications in Information and Systems 20.4 (2020): 389–413.
Vora, Suhani, et al. “Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes.” arXiv preprint arXiv:2111.13260 (2021).
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
Fridovich-Keil, Sara, et al. “Plenoxels: Radiance fields without neural networks.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
Xu, Qiangeng, et al. “Point-nerf: Point-based neural radiance fields.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
Müller, Thomas, et al. “Instant neural graphics primitives with a multiresolution hash encoding.” ACM transactions on graphics (TOG) 41.4 (2022): 1–15.
Yu, Alex, et al. “Plenoctrees for real-time rendering of neural radiance fields.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
Chen, Zhiqin, et al. “Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
Garbin, Stephan J., et al. “Fastnerf: High-fidelity neural rendering at 200fps.” Proceedings of the IEEE/CVF international conference on computer vision. 2021.
