Blending Real and Virtual Objects in Augmented Reality Environments

Authors

  • Lidiane Teixeira Pereira Computer Science Department, Federal University of Juiz de Fora, Brazil
  • Jairo Francisco de Souza Computer Science Department, Federal University of Juiz de Fora, Brazil
  • Rodrigo Luis de Souza da Silva Computer Science Department, Federal University of Juiz de Fora, Brazil https://orcid.org/0000-0002-4187-8798

DOI:

https://doi.org/10.13052/jmm1550-4646.1868

Keywords:

Augmented Reality, Photorealism, Ray Tracing

Abstract

Although Augmented Reality applications are becoming increasingly popular, the lack of visual realism in rendering still remains an open problem due to its computational cost. Physically-based algorithms can generate renderings with a high degree of photorealism, and they are becoming popular after the recent development of hardware accelerators. This work shows how to integrate Augmented Reality frameworks with ray tracing frameworks to create scenes with high-quality, real-time reflections and refractions, with emphasis on the blending of virtual objects to the real environment. To support the interaction between real and virtual elements, a textured cube using images from the real environment must be provided. Our framework does not add processing overhead to the application when comparing the use of the proposed middleware to the use of ray tracing frameworks alone. We will show that with our approach, photorealistic augmented reality rendering can be achieved in real time without the use of any special equipment.

Downloads

Download data is not yet available.

Author Biographies

Lidiane Teixeira Pereira, Computer Science Department, Federal University of Juiz de Fora, Brazil

Lidiane Teixeira Pereira holds a Master’s degree in Computer Science from the Federal University of Juiz de Fora, the same institution where she received her Bachelor’s degree in Computer Science. Throughout her undergraduate and master’s degrees, she participated in research projects on Virtual and Augmented Reality.

Jairo Francisco de Souza, Computer Science Department, Federal University of Juiz de Fora, Brazil

Jairo Francisco de Souza holds a Ph.D. (2012) in Informatics from the Pontifical Catholic University of Rio de Janeiro, Brazil. Jairo Souza is a full professor in Information Retrieval and Natural Language Processing, member of LApIC research group at Federal University of Juiz de Fora, Brazil. He has worked on several research projects funded by the Brazilian National Research and Educational Network (RNP), Coordination for the Improvement of Higher Education Personnel (CAPES), National Council for Scientific and Technological Development (CNPq), and others. His research interests are knowledge representation, semantic web, information integration, and natural language processing.

Rodrigo Luis de Souza da Silva, Computer Science Department, Federal University of Juiz de Fora, Brazil

Rodrigo Luis de Souza da Silva is an Associate Professor in the Department of Computer Science at Federal University of Juiz de Fora. He has a B.S. in Computer Science from the Catholic University of Petropolis (1999), M.S. in Computer Science from Federal University of Rio de Janeiro (2002), Ph.D. in Civil Engineering from Federal University of Rio de Janeiro (2006) and a postdoc in Computer Science from the National Laboratory for Scientific Computing (2008). His main research interests are Augmented Reality, Virtual Reality, Scientific Visualization and Computer Graphics.

References

Ronald T. Azuma, Yohan Baillot, Reinhold Behringer, Steven K. Feiner, Simon Julier, and Blair MacIntyre. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6):34–47, 2001.

Kangsoo Kim, Mark Billinghurst, Gerd Bruder, Henry Been-Lirn Duh, and Gregory F. Welch. Revisiting trends in augmented reality research: A review of the 2nd decade of ismar (2008–2017). IEEE Transactions on Visualization and Computer Graphics, 24(11):2947–2962, 2018.

Insu Yu, Jesper Mortensen, Pankaj Khanna, Bernhard Spanlang, and Mel Slater. Visual realism enhances realistic response in an immersive virtual environment – part 2. IEEE Computer Graphics and Applications, 32(6):36–45, 2012.

Cha Lee, Gustavo A. Rincon, Greg Meyer, Tobias Höllerer, and Doug A. Bowman. The effects of visual realism on search tasks in mixed reality simulation. IEEE Transactions on Visualization and Computer Graphics, 19(4):547–556, 2013.

Lidiane Teixeira Pereira, Wellingston Cataldo Roberti Junior, and Rodrigo Luis de Souza da Silva. Photorealism in mixed reality: A systematic literature review. International Journal of Virtual Reality, 21(1):15–29, 2021.

Aleksander Yacovenco and Rodrigo Luis de Souza da Silva. Towards photorealism in augmented reality–a brief review on the main requirements. Relatórios Técnicos do DCC/UFJF, 2022.

Alexander. Keller, Luca Fascione, M. Fajardo, Iliyan Georgiev, Per H. Christensen, Johannes Hanika, Christian Eisenacher, and Greg Boyd Nichols. The path tracing revolution in the movie industry. In ACM SIGGRAPH 2015 Courses, SIGGRAPH ’15, New York, NY, USA, 2015. Association for Computing Machinery.

Alexander Keller, Timo Viitanen, Colin Barré-Brisebois, Christoph Schied, and Morgan McGuire. Are we done with ray tracing? In ACM SIGGRAPH 2019 Courses, SIGGRAPH ’19, New York, NY, USA, 2019. Association for Computing Machinery.

Ronald T. Azuma. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6(4):355–385, 08 1997.

Lidiane Teixeira Pereira, Wellingston Cataldo Junior, Jairo Francisco de Souza, and Rodrigo Luis de Souza da Silva. Array-tracing-a middleware to provide ray tracing capabilities to augmented reality libraries. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR), pages 391–397. IEEE, 2020.

Artur Lira dos Santos, Diego Lemos, Jorge Eduardo Falcão Lindoso, and Veronica Teichrieb. Real time ray tracing for augmented reality. In 2012 14th Symposium on Virtual and Augmented Reality, pages 131–140. IEEE, 2012.

Peter Kán and Hannes Kaufmann. Differential irradiance caching for fast high-quality light transport between virtual and real worlds. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 133–141. IEEE, 2013.

Tobias Schwandt and Wolfgang Broll. A single camera image based approach for glossy reflections in mixed reality applications. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 37–43. IEEE, 2016.

Christian Vazquez, Nicole Tan, and Shrenik Sadalgi. Fused photo: Augmented reality staging for photorealistic visualization in online home retail. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2021.

A’aeshah Alhakamy and Mihran Tuceryan. Cubemap360: Interactive global illumination for augmented reality in dynamic environment. In 2019 SoutheastCon, pages 1–8, 2019.

Paul Milgram and Fumio Kishino. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12):1321–1329, 1994.

Maximilian Speicher, Brian D Hall, and Michael Nebeling. What is mixed reality? In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–15, 2019.

Thomas Porter and Tom Duff. Compositing digital images. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’84, page 253–259, New York, NY, USA, 1984. Association for Computing Machinery.

Tom McReynolds and David Blythe. Advanced graphics programming using OpenGL. Elsevier, 2005.

Andrew S Glassner. An introduction to ray tracing. Morgan Kaufmann, 1989.

Steve Marschner and Peter Shirley. Fundamentals of computer graphics. CRC Press, 2018.

John F Hughes, Andries Van Dam, Morgan McGuire, James D Foley, David Sklar, Steven K Feiner, and Kurt Akeley. Computer graphics: principles and practice. Pearson Education, 2014.

Wnêiton Gomes, Celso Camilo, Leonardo Lima, Alexandre Cardoso, Edgard Lamounier Jr, and Keiji Yamanaka. Artificial neural networks to recognize artoolkit markers. In International Conference on Artificial Intelligence and Pattern Recognition, pages 464–469, 01 2007.

Héctor Antonio Villa Martınez. Accelerating algorithms for ray tracing. Technical report, Department of Computer Science, University of Sonora, 2006.

Published

2022-07-18

Issue

Section

Articles

Most read articles by the same author(s)