Computer Vision Mobile System for Education Using Augmented Reality Technology

Authors

  • Misiuk Tetiana Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine https://orcid.org/0000-0001-6793-2185
  • Yuriy Kondratenko Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine https://orcid.org/0000-0001-7736-883X
  • Ievgen Sidenko Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine https://orcid.org/0000-0001-6496-2469
  • Galyna Kondratenko Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine https://orcid.org/0000-0001-7736-883X

DOI:

https://doi.org/10.13052/jmm1550-4646.1744

Keywords:

Augmented reality, computer vision, mobile application, image processing, target, 3D model, education

Abstract

This article analyzes the algorithms of computer vision, the features of the application of augmented reality technology and existing software modules, frameworks and libraries. The result is a computer vision mobile system (application) using augmented reality technology, which allows users (for example, students) to obtain additional virtual information about the research object and to be able to interact with it. The functional model of the system is formed, the process of application development using the Vuforia library is described and the results of the work are given. The result is an Android application, which using augmented reality tools, allows the user to obtain a virtual environment object in a real-world. This computer vision mobile system is intended for educational purposes, in particular for use in schools and universities for more effective interaction between users and educational material.

Downloads

Download data is not yet available.

Author Biographies

Misiuk Tetiana, Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine

Misiuk Tetiana is a master’s student of the Intelligent Information Systems Department of Petro Mohyla Black Sea National University. She received his bachelor and master degrees in specialty 122 “Computer Science” at Petro Mohyla Black Sea National University. From January 2019 till June 2019 Tetiana took part in the implementation of the different projects related to creating mobile applications with the use of AR technology. Tatiana is interested in studying augmented reality technologies; her job is related to this area. Both of her graduate projects are related to the study of augmented reality and the use of this technology in the field of education.

Yuriy Kondratenko, Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine

Yuriy Kondratenko is Doctor of Science, Professor, Honour Inventor of Ukraine (2008), Corr. Academician of Royal Academy of Doctors (Barcelona, Spain), Head of the Intelligent Information Systems Department at Petro Mohyla Black Sea National University (PMBSNU), Ukraine. He has received (a) the Ph.D. (1983) and Dr.Sc. (1994) in Elements and Devices of Computer and Control Systems from Odessa National Polytechnic University, (b) several international grants and scholarships for conducting research at Institute of Automation of Chongqing University, P.R.China (1988–1989), Ruhr-University Bochum, Germany (2000, 2010), Nazareth College and Cleveland State University, USA (2003), (c) Fulbright Scholarship for researching in USA (2015/2016) at the Dept. of Electrical Engineering and Computer Science in Cleveland State University. Research interests include robotics, automation, sensors and control systems, intelligent decision support systems, fuzzy logic.

Ievgen Sidenko, Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine

Ievgen Sidenko is a Ph.D., Associate Professor, Associate Professor of the Intelligent Information Systems Department at Petro Mohyla Black Sea National University (PMBSNU), Ukraine. He has received master degree in speciality “Intelligent decision making systems” (2010) at Petro Mohyla Black Sea State University and Ph.D. degree in “Information technologies” (2015) at PMBSNU. Since 2013 took part in the implementation of a number of international and state projects (Tempus Cabriolet, Erasmus+ Aliot) related to intelligent systems, university-industry cooperation, neural networks, Internet of Things, etc. His research interests include fuzzy sets and fuzzy logic, decision-making, optimization methods, neural networks, data mining, clustering and classification.

Galyna Kondratenko, Intelligent Information Systems Department. Petro Mohyla Black Sea National University, Mykolaiv, Ukraine

Galyna Kondratenko is a Ph.D., Associate Professor, Associate Professor of the Intelligent Information Systems Department at Petro Mohyla Black Sea National University (PMBSNU), Ukraine. She is a specialist in control systems, decision-making, fuzzy logic. She worked in the framework of international scientific university cooperation during the implementation of international projects with the European Union: TEMPUS (Cabriolet), Erasmus + (Aliot) and DAAD-Ostpartnerschaftsprogramm (project with the University of Saarland, Germany). Her research interests include computer control systems, fuzzy logic, decision-making, intelligent robotic devices.

References

L. Englestone, ‘.NET Developer’s Guide to Augmented Reality in iOS’, Apress, Berkeley, CA, 2021, doi: 10.1007/978-1-4842-6770-7.

G. R. Shinde, P. S. Dhotre, P. N. Mahalle, N. Dey, ‘Internet of Things Integrated Augmented Reality’, Springer, Singapore, 2021, doi: 10.1007/978-981-15-6374-4.

T. Jung, M. C. tom Dieck, P. A. Rauschnabel, ‘Augmented Reality and Virtual Reality’, Springer, Cham, 2020, doi: 10.1007/978-3-030-37 869-1.

V. Geroimenko, ‘Augmented Reality in Education. A New Technology for Teaching and Learning’, Springer, Cham, 2020, doi: 10.1007/978-3-030-42156-4.

R. Dörner, W. Broll, P. Grimm, B. Jung, ‘Virtual und Augmented Reality (VR/AR)’, Springer Vieweg, Berlin, Heidelberg, 2019, doi: 10.1007/978-3-662-58861-1.

J. Peddie, ‘Augmented Reality’, Springer, Cham, 2017, doi: 10.1007/978-3-319-54502-8.

M. Spanke, ‘Augmented Reality’, in: Retail Isn’t Dead, Palgrave Macmillan, Cham, 2020, doi: 10.1007/978-3-030-36650-6_5.

T. Gek-Siang, K. Ab. Aziz, Z. Ahmad, ‘Correction to: Augmented Reality: The Game Changer of Travel and Tourism Industry in 2025’, in: Park S. H., Gonzalez-Perez M. A., Floriani D. E. (eds) The Palgrave Handbook of Corporate Sustainability in the Digital Era, Palgrave Macmillan, Cham, 2021, doi: 10.1007/978-3-030-42412-1_42.

S. H. Kidd, H. Crompton, ‘Augmented Learning with Augmented Reality’. in: Churchill D., Lu J., Chiu T., Fox B. (eds) Mobile Learning Design. Lecture Notes in Educational Technology, Springer, Singapore, 2016, doi: 10.1007/978-981-10-0027-0_6.

W. Wang, ‘Understanding Augmented Reality and ARKit’, in: Beginning ARKit for iPhone and iPad, Apress, Berkeley, CA, 2018, doi: 10.1007/978-1-4842-4102-8_1.

L. Soussi, Z. Spijkerman, S. Jansen, ‘A Case Study of the Health of an Augmented Reality Software Ecosystem: Vuforia’, in: Maglyas A., Lamprecht AL. (eds) Software Business. ICSOB 2016. Lecture Notes in Business Information Processing, volume 240, Springer, Cham, 2016, doi: 10.1007/978-3-319-40515-5_11.

M. Sharma, R. Vaidya, A. K. Saxena, I. Aggarwal, J. Khurana, ‘Design and Development of Mobile Augmented Reality’, in: Abraham A., Castillo O., Virmani D. (eds) Proceedings of 3rd International Conference on Computing Informatics and Networks. Lecture Notes in Networks and Systems, volume 167, Springer, Singapore, 2021, doi: 10.1007/978-981-15-9712-1_46.

M. Smith, A. Maiti, A. D. Maxwell, A. A. Kist, ‘Using Unity 3D as the Augmented Reality Framework for Remote Access Laboratories’, in: Auer M., Langmann R. (eds) Smart Industry & Smart Education. REV 2018. Lecture Notes in Networks and Systems, volume 47, Springer, Cham, 2019, doi: 10.1007/978-3-319-95678-7_64.

A. Fowler, ‘The Unity ARKit’, in: Beginning iOS AR Game Development, Apress, Berkeley, CA, 2019, doi: 10.1007/978-1-4842-3618-5_3.

S. T. Dougherty, ‘Affine and Projective Planes’, in: Combinatorics and Finite Geometry. Springer Undergraduate Mathematics Series, Springer, Cham, 2020, doi: 10.1007/978-3-030-56395-0_4.

O. Striuk, Y. Kondratenko, I. Sidenko, A. Vorobyova, ‘Generative Adversarial Neural Network for Creating Photorealistic Images’, IEEE 2nd International Conference on Advanced Trends in Information Theory (ATIT), Kyiv, Ukraine, 2020, doi: 10.1109/ATIT50783.2020.9349326.

F. Shi et al., ‘Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19’, in: IEEE Reviews in Biomedical Engineering, 14, 2021, doi: 10.1109/RBME.2020.2987975.

V. Zinchenko, G. Kondratenko, I. Sidenko, Y. Kondratenko, ‘Computer Vision in Control and Optimization of Road Traffic’, IEEE Third International Conference on Data Stream Mining & Processing (DSMP), Lviv, Ukraine, 2020, doi: 10.1109/DSMP47368.2020.9204329.

I. Sova, I. Sidenko, Y. Kondratenko, ‘Machine Learning Technology for Neoplasm Segmentation on Brain MRI Scans’, in: PhD Symposium at ICT in Education, Research, and Industrial Applications (ICTERI-PhD 2020), CEUR Workshop Proceedings, volume 2791, Kharkiv, Ukraine, 2020.

A. Potlapally, P. S. R. Chowdary, S. S. Raja Shekhar, N. Mishra, C. S. V. D. Madhuri, A. V. V. Prasad, ‘Instance Segmentation in Remote Sensing Imagery using Deep Convolutional Neural Networks’, International Conference on contemporary Computing and Informatics (IC3I), Singapore, 2019, doi: 10.1109/IC3I46837.2019.9055569.

D. Mikhov, Y. Kondratenko, G. Kondratenko, I. Sidenko, ‘Fuzzy Logic Approach to Improving the Digital Images Contrast’, IEEE 2nd Ukraine Conference on Electrical and Computer Engineering (UKRCON), Lviv, Ukraine, 2019, doi: 10.1109/UKRCON.2019.8879961.

Y. Pomanysochka, Y. Kondratenko, I. Sidenko, ‘Noise Filtration in the Digital Images Using Fuzzy Sets and Fuzzy Logic’, 15th International Conference on ICT in Education, Research, and Industrial Applications: PhD Symposium (ICTERI 2019: PhD Symposium), CEUR Workshop Proceedings, volume 2403, Kherson, Ukraine, 2019.

D. Oliva, M. Abd Elaziz, S. Hinojosa, ‘Clustering Algorithms for Image Segmentation’, in: Metaheuristic Algorithms for Image Segmentation: Theory and Applications. Studies in Computational Intelligence, volume 825, Springer, Cham, 2019, doi: 10.1007/978-3-030-12931-6_14.

I. Khortiuk, G. Kondratenko, I. Sidenko, Y. Kondratenko, ‘Scoring System Based on Neural Networks for Identification of Factors in Image Perception’, 4th International Conference on Computational Linguistics and Intelligent Systems (COLINS), CEUR Workshop Proceedings, volume 2604, Lviv, Ukraine, 2020.

H. R. Turkar, N. V. Thakur, ‘Performance Comparison of Clustering Algorithms Based Image Segmentation on Mobile Devices’, in: Mallick P., Balas V., Bhoi A., Zobaa A. (eds) Cognitive Informatics and Soft Computing. Advances in Intelligent Systems and Computing, volume 768, Springer, Singapore, 2019, doi: 10.1007/978-981-13-0617-4_56.

K. Ivanova, G. Kondratenko, I. Sidenko, Y. Kondratenko, ‘Artificial Intelligence in Automated System for Web-Interfaces Visual Testing’, 4th International Conference on Computational Linguistics and Intelligent Systems (COLINS), CEUR Workshop Proceedings, volume 2604, Lviv, Ukraine, 2020.

H. Ismkhan, ‘I-k-means-+: An iterative clustering algorithm based on an enhanced version of the k-means’, Pattern Recognition, 79, 2018, doi: 10.1016/j.patcog.2018.02.015.

H. Zhang, Y. Zhu, ‘KSLIC: K-mediods Clustering Based Simple Linear Iterative Clustering’, in: Lin Z. et al. (eds) Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science, volume 11858, Springer, Cham, 2019, doi: 10.1007/978-3-030-31723-2_44.

M. A. Mahboob, B. Genc, ‘Evaluation of ISODATA Clustering Algorithm for Surface Gold Mining Using Satellite Data’, International Conference on Electrical, Communication, and Computer Engineering (ICECCE), Swat, Pakistan, 2019, doi: 10.1109/ICECCE47252.2019.89 40673.

E. Küçükkülahlı, P. Erdoğmuş, K. Polat, ‘Histogram-based automatic segmentation of images’, Neural Comput & Applic, 27, 2016, doi: 10.1007/s00521-016-2287-7.

S. Kumar, M. Pant, M. Kumar et al., ‘Colour image segmentation with histogram and homogeneity histogram difference using evolutionary algorithms’, Int. J. Mach. Learn. & Cyber., 9, 2018, doi: 10.1007/s13042-015-0360-7.

B. Hou, X. Zhang, D. Gong, S. Wang, X. Zhang, L. Jiao, ‘Fast graph-based SAR image segmentation via simple superpixels’, IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 2017, doi: 10.1109/IGARSS.2017.8127073.

G. Toscana, S. Rosa, B. Bona, ‘Fast Graph-Based Object Segmentation for RGB-D Images’, in: Bi Y., Kapoor S., Bhatia R. (eds) Proceedings of SAI Intelligent Systems Conference (IntelliSys). Lecture Notes in Networks and Systems, volume 16, Springer, Cham, 2018, doi: 10.1007/978-3-319-56991-8_5.

C. Chen, Z. Mu, ‘An Impoved Image Registration Method Based on SIFT and SC-RANSAC Algorithm’, Chinese Automation Congress (CAC), Xi’an, China, 2018, doi: 10.1109/CAC.2018.8623265.

H. Duan, X. Zhang, W. He, ‘Optimization of SURF Algorithm for Image Matching of Parts’, in: Chen S., Zhang Y., Feng Z. (eds) Transactions on Intelligent Welding Manufacturing. Transactions on Intelligent Welding Manufacturing, Springer, Singapore, 2018, doi: 10.1007/978-981-13-3651-5_11.

S. Routray, A. K. Ray, C. Mishra, ‘Analysis of various image feature extraction methods against noisy image: SIFT, SURF and HOG’, Second International Conference on Electrical, Computer and Communication Technologies (ICECCT), Coimbatore, India, 2017, doi: 10.1109/ICECCT.2017.8117846.

A. B. Dos Santos, J. B. Dourado, A. Bezerra, ‘ARToolkit and Qualcomm Vuforia: An Analytical Collation’, XVIII Symposium on Virtual and Augmented Reality (SVR), Gramado, Brazil, 2016, doi: 10.1109/SVR.2016.46.

D. Khan et al., ‘Robust Tracking Through the Design of High Quality Fiducial Markers: An Optimization Tool for ARToolKit’, in: IEEE Access, 6, 2018, doi: 10.1109/ACCESS.2018.2801028.

X. Luo, ‘Design of full-link digital marketing in business intelligence era with computer software Edraw Max’, Management Science Informatization and Economic Innovation Development Conference (MSIEID), Guangzhou, China, 2020, doi: 10.1109/MSIEID52046.2020.00077.

P. Chaudhari, R. Kamte, K. Kunder, A. Jose, S. Machado, “Street Smart’: Safe Street App for Women Using Augmented Reality’, Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, 2018, doi: 10.1109/ICCUBEA.2018.8697863.

M. A. Bintang, R. Harwahyu, R. F. Sari, ‘SMARIoT: Augmented Reality for Monitoring System of Internet of Things using EasyAR’, 4th International Conference on Informatics and Computational Sciences (ICICoS), Semarang, Indonesia, 2020, doi: 10.1109/ICICoS51170.2020.9299088.

E. Gandolfi, R. E. Ferdig, Z. Immel, ‘Educational Opportunities for Augmented Reality’, in: Voogt J., Knezek G., Christensen R., Lai KW. (eds) Second Handbook of Information Technology in Primary and Secondary Education. Springer International Handbooks of Education, Springer, Cham, 2018, doi: 10.1007/978-3-319-71054-9_112.

L. Soussi, Z. Spijkerman, S. Jansen, ‘A Case Study of the Health of an Augmented Reality Software Ecosystem: Vuforia’, in: Maglyas A., Lamprecht AL. (eds) Software Business. ICSOB 2016. Lecture Notes in Business Information Processing, volume 240, Springer, Cham, 2016, doi: 10.1007/978-3-319-40515-5_11.

Z. Oufqir, A. El Abderrahmani, K. Satori, ‘ARKit and ARCore in serve to augmented reality’, International Conference on Intelligent Systems and Computer Vision (ISCV), Fez, Morocco, 2020, doi: 10.1109/ISCV49265.2020.9204243.

V. Lytvyn, A. Gozhyj, I. Kalinina, V. Vysotska, V. Shatskykh, L. Chyrun, Y. Borzov, ‘An intelligent system of the content relevance at the example of films according to user needs’, CEUR Workshop Proceedings, 1st International Workshop on Information-Communication Technologies and Embedded Systems, ICT and ES, volume 2516, 2019.

S. Kryvyi, O. Grinenko, V. Opanasenko, ‘Logical Approach to the Research of Properties of Software Engineering Ecosystem,’ IEEE 11th International Conference on Dependable Systems, Services and Technologies (DESSERT), Kyiv, Ukraine, 2020, doi: 10.1109/DESSERT50317.2020.9125033.

P. Parekh, S. Patel, N. Patel, et al., ‘Systematic review and meta-analysis of augmented reality in medicine, retail, and games’, Vis. Comput. Ind. Biomed. Art 3 (21), 2020, doi: 10.1186/s42492-020-00057-7.

O. Drozd, K. Zashcholkin, O. Martynyuk, J. Drozd, Y. Sulima, ‘Development of ICT Models in Area of Safety Education’, IEEE East-West Design & Test Symposium, Varna, Bulgaria, 2020, doi: 10.1109/EWDTS50664.2020.9224861.

Y. Boltov, I. Skarga-Bandurova, I. Kotsiuba, M. Hrushka, G. Krivoulya, R. Siriak, ‘Performance Evaluation of Real-Time System for Vision-Based Navigation of Small Autonomous Mobile Robots’, 10th International Conference on Dependable Systems, Services and Technologies (DESSERT), Leeds, UK, 2019, doi: 10.1109/DESSERT.2019.8770045..

D. Skorokhodov, et al., ‘Using Augmented Reality Technology to Improve the Quality of Transport Services’, in: Sukhomlin V., Zubareva E. (eds) Convergent Cognitive Information Technologies. Convergent 2018. Communications in Computer and Information Science, volume 1140, Springer, Cham, 2020, doi: 10.1007/978-3-030-37436-5_30 9.

J. M. T. Ribeiro, J. Martins, R. Garcia, ‘Augmented Reality Technology as a Tool for Better Usability of Medical Equipment’, in: Lhotska L., Sukupova L., Lacković I., Ibbott G. (eds) World Congress on Medical Physics and Biomedical Engineering 2018. IFMBE Proceedings, volume 68, Springer, Singapore, 2019, doi: 10.1007/978-981-10-9023-3_61.

Downloads

Published

2021-06-21

How to Cite

Tetiana, M., Kondratenko, Y., Sidenko, I., & Kondratenko, G. (2021). Computer Vision Mobile System for Education Using Augmented Reality Technology. Journal of Mobile Multimedia, 17(4), 555–576. https://doi.org/10.13052/jmm1550-4646.1744

Issue

2021: Vol 17 Iss 4

Section

Mobile Communication and Computing for Internet of Things and Industrial Automat