Blending Real and Virtual Objects in Augmented Reality Environments
DOI:
https://doi.org/10.13052/jmm1550-4646.1868Keywords:
Augmented Reality, Photorealism, Ray TracingAbstract
Although Augmented Reality applications are becoming increasingly popular, the lack of visual realism in rendering still remains an open problem due to its computational cost. Physically-based algorithms can generate renderings with a high degree of photorealism, and they are becoming popular after the recent development of hardware accelerators. This work shows how to integrate Augmented Reality frameworks with ray tracing frameworks to create scenes with high-quality, real-time reflections and refractions, with emphasis on the blending of virtual objects to the real environment. To support the interaction between real and virtual elements, a textured cube using images from the real environment must be provided. Our framework does not add processing overhead to the application when comparing the use of the proposed middleware to the use of ray tracing frameworks alone. We will show that with our approach, photorealistic augmented reality rendering can be achieved in real time without the use of any special equipment.
Downloads
References
Ronald T. Azuma, Yohan Baillot, Reinhold Behringer, Steven K. Feiner, Simon Julier, and Blair MacIntyre. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6):34–47, 2001.
Kangsoo Kim, Mark Billinghurst, Gerd Bruder, Henry Been-Lirn Duh, and Gregory F. Welch. Revisiting trends in augmented reality research: A review of the 2nd decade of ismar (2008–2017). IEEE Transactions on Visualization and Computer Graphics, 24(11):2947–2962, 2018.
Insu Yu, Jesper Mortensen, Pankaj Khanna, Bernhard Spanlang, and Mel Slater. Visual realism enhances realistic response in an immersive virtual environment – part 2. IEEE Computer Graphics and Applications, 32(6):36–45, 2012.
Cha Lee, Gustavo A. Rincon, Greg Meyer, Tobias Höllerer, and Doug A. Bowman. The effects of visual realism on search tasks in mixed reality simulation. IEEE Transactions on Visualization and Computer Graphics, 19(4):547–556, 2013.
Lidiane Teixeira Pereira, Wellingston Cataldo Roberti Junior, and Rodrigo Luis de Souza da Silva. Photorealism in mixed reality: A systematic literature review. International Journal of Virtual Reality, 21(1):15–29, 2021.
Aleksander Yacovenco and Rodrigo Luis de Souza da Silva. Towards photorealism in augmented reality–a brief review on the main requirements. Relatórios Técnicos do DCC/UFJF, 2022.
Alexander. Keller, Luca Fascione, M. Fajardo, Iliyan Georgiev, Per H. Christensen, Johannes Hanika, Christian Eisenacher, and Greg Boyd Nichols. The path tracing revolution in the movie industry. In ACM SIGGRAPH 2015 Courses, SIGGRAPH ’15, New York, NY, USA, 2015. Association for Computing Machinery.
Alexander Keller, Timo Viitanen, Colin Barré-Brisebois, Christoph Schied, and Morgan McGuire. Are we done with ray tracing? In ACM SIGGRAPH 2019 Courses, SIGGRAPH ’19, New York, NY, USA, 2019. Association for Computing Machinery.
Ronald T. Azuma. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6(4):355–385, 08 1997.
Lidiane Teixeira Pereira, Wellingston Cataldo Junior, Jairo Francisco de Souza, and Rodrigo Luis de Souza da Silva. Array-tracing-a middleware to provide ray tracing capabilities to augmented reality libraries. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR), pages 391–397. IEEE, 2020.
Artur Lira dos Santos, Diego Lemos, Jorge Eduardo Falcão Lindoso, and Veronica Teichrieb. Real time ray tracing for augmented reality. In 2012 14th Symposium on Virtual and Augmented Reality, pages 131–140. IEEE, 2012.
Peter Kán and Hannes Kaufmann. Differential irradiance caching for fast high-quality light transport between virtual and real worlds. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 133–141. IEEE, 2013.
Tobias Schwandt and Wolfgang Broll. A single camera image based approach for glossy reflections in mixed reality applications. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 37–43. IEEE, 2016.
Christian Vazquez, Nicole Tan, and Shrenik Sadalgi. Fused photo: Augmented reality staging for photorealistic visualization in online home retail. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2021.
A’aeshah Alhakamy and Mihran Tuceryan. Cubemap360: Interactive global illumination for augmented reality in dynamic environment. In 2019 SoutheastCon, pages 1–8, 2019.
Paul Milgram and Fumio Kishino. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12):1321–1329, 1994.
Maximilian Speicher, Brian D Hall, and Michael Nebeling. What is mixed reality? In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–15, 2019.
Thomas Porter and Tom Duff. Compositing digital images. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’84, page 253–259, New York, NY, USA, 1984. Association for Computing Machinery.
Tom McReynolds and David Blythe. Advanced graphics programming using OpenGL. Elsevier, 2005.
Andrew S Glassner. An introduction to ray tracing. Morgan Kaufmann, 1989.
Steve Marschner and Peter Shirley. Fundamentals of computer graphics. CRC Press, 2018.
John F Hughes, Andries Van Dam, Morgan McGuire, James D Foley, David Sklar, Steven K Feiner, and Kurt Akeley. Computer graphics: principles and practice. Pearson Education, 2014.
Wnêiton Gomes, Celso Camilo, Leonardo Lima, Alexandre Cardoso, Edgard Lamounier Jr, and Keiji Yamanaka. Artificial neural networks to recognize artoolkit markers. In International Conference on Artificial Intelligence and Pattern Recognition, pages 464–469, 01 2007.
Héctor Antonio Villa Martınez. Accelerating algorithms for ray tracing. Technical report, Department of Computer Science, University of Sonora, 2006.
