RECOGNIZING AND EXPLORING AZULEJOS ON HISTORIC BUILDINGS’ FACADES BY COMBINING COMPUTER VISION AND GEOLOCATION IN MOBILE AUGMENTED REALITY APPLICATIONS
Keywords:
Image Recognition, Machine Learning Algorithms, Mobile Augmented Reality, AzulejosAbstract
Mobile augmented reality (MAR) applications assist users in navigating and exploring their actual surroundings, displaying virtual contents that correspond to objects and scenes in the real world. However, despite the growing popularity of these applications, some experiences can be frustrating when users are unable to correctly recognize Points of Interest (POI), objects, or places they want to visit or obtain more information. The misleading recognition can occur due to imprecise Global Positioning System (GPS) data or a lack of QR codes for interaction. Hence, this article presents a proposal that combines pattern recognition in images with geolocation information to improve the accuracy of the identification of POIs. The usage scenario is the identification of azulejos (tiles) on the facades of historic buildings in the city of Belém of Pará, Brazil. This issue is relevant based on similarities between azulejos and its huge amount of different types, whose variety of designs and colors of geometric forms can make the identification a hard task. The used methods to extract the azulejos’ features were the co-occurrence matrix combined with color percentage, and the global positioning data to increase the accuracy of classification because similar azulejos can be geographically far apart. Tests were conducted using six machine learning algorithms (neural network, decision tree, k-nearest neighbors, naive Bayes, random forest, and support vector machine) of different paradigms. The first results show that the pattern recognition in images combined with geolocation information is a promising approach for better identification of the POIs in MAR applications.
Downloads
References
Kamruzzaman, M., Islam, T., Poddar, S., (2014). Accuracy of handheld GPS comparing with total station in
land use survey: A case study in ruet campus. International Journal of Innovation and Applied Studies. Vol.
, Nº 1, pp. 343–352.
Grubert, J., Langlotz, T., Grasset, R., (2011). Augmented reality browser survey. Technical Report, Institute
for Computer Graphics and Vision, Graz University of Technology, Austria.
Andersen, D, Popescu, V, Cabrera, M, Shanghavi, A, Gomez, G., Marley, S., Wachs, J., (2016). Virtual
annotations of the surgical field through an augmented reality transparent display. The Visual Computer Vol.
, Nº 11, pp. 1–18
Liarokapis, F., Macan, L., Malone, G., RebolledoMendez, G., De Freitas, S., (2009). Multimodal augmented
reality tangible gaming. The Visual Computer. Vol. 25, Nº 12, pp. 1109–1120.
Wu, H., Lee, S., Chang, H., Liang, J., (2013). Current status, opportunities and challenges of augmented
reality in education. Computers & Education. Vol. 62, pp. 41–49.
Quesada, C., González, D., Alfaro, I., Cueto, E., Huerta, A., Chinesta, F., (2016). Real-time simulation
techniques for augmented learning in science and engineering. The Visual Computer. Vol. 32, Nº 11, pp.
–1479.
Kourouthanassis, P., Boletsis, C., Bardaki, C., Chasanidou, D. (2015). Tourists responses to mobile
augmented reality travel guides: The role of emotions on adoption behavior. Pervasive and Mobile
Computing Vol. 18, pp. 71–87.
Yovcheva, Z., Buhalis, D., Gatzidis, C., (2012). Overview of smartphone augmented reality applications for
tourism. e-Review of Tourism Research. Vol. 10, Nº 2, pp. 63–66.
Billinghurst, M., Clark, A., Lee, G., (2015). A survey of augmented reality. Foundations and Trends in
Human-Computer Interaction. Vol. 8, Nº 3, pp. 73–272.
Huang, Z., Hui, P., Peylo, C., Chatzopoulos, D., (2013). Mobile augmented reality survey: A bottom-up
approach. Computing Research Repository, pp. 1–35.
Chen, D., Tsai, S., Hsu, C. H., Singh, J. P., & Girod, B., (2011). Mobile augmented reality for books on a
shelf. In IEEE International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.
Baheti, P. K., Swaminathan, A., Chari, M., Diaz, S., & Grzechnik, S., (2011). Information-theoretic database
building and querying for mobile augmented reality applications. In 10th IEEE International Symposium on
Mixed and Augmented Reality (ISMAR) (pp. 47-53). IEEE.
Maidi, M., Preda, M., Lehiani, Y., & Lavric, T., (2014). Vision-based tracking in large image database for
real-time mobile augmented reality. In IEEE 16th International Workshop on Multimedia Signal Processing
(MMSP) (pp. 1-6). IEEE.
Tsai, F. S., Etoh, M., Xie, X., Lee, W. C., & Yang, Q., (2010). Introduction to mobile information retrieval.
IEEE Intelligent Systems, 25(1), 11-15.
Hartl, A., Schmalstieg, D., & Reitmayr, G., (2014). Client-side mobile visual search. In International
Conference on Computer Vision Theory and Applications (VISAPP) (Vol. 3, pp. 125-132). IEEE.
Li, D., & Chuah, M. C., (2013). Emovis: An efficient mobile visual search system for landmark recognition.
In IEEE Ninth International Conference on Mobile Ad-hoc and Sensor Networks (MSN) (pp. 53-60). IEEE.
Kuo, Y. H., & Hsu, W. H., (2017). Dehashing: Server-Side Context-Aware Feature Reconstruction for
Mobile Visual Search. IEEE Transactions on Circuits and Systems for Video Technology, 27(1), 139-148.
Chen, D. M., & Girod, B., (2015). A hybrid mobile visual search system with compact global signatures.
IEEE Transactions on Multimedia, 17(7), 1019-1030.
Zhang, J., Hallquist, A., Liang, E., & Zakhor, A., (2011). Location-based image retrieval for urban
environments. In 18th IEEE International Conference on Image Processing (ICIP) (pp. 3677-3680). IEEE.
Salarian, M., & Ansari, R., (2016). Improved Image Retrieval for Efficient Localization in Urban Areas
Using Location Uncertainty Data. In Multimedia (ISM), 2016 IEEE International Symposium on (pp. 181-
. IEEE.
Paucher, R., & Turk, M., (2010). Location-based augmented reality on mobile phones. In IEEE Computer
Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 9-16). IEEE.
Reitmayr, G., & Schmalstieg, D., (2003). Location based applications for mobile augmented reality. In
Proceedings of the Fourth Australasian user interface conference on User interfaces 2003-Volume 18 (pp. 65-
. Australian Computer Society, Inc..
Alappanavar, P. B., Kurvey, B., Karad, M., & Bhagwatkar, S., (2013). Location Based Augmented Reality.
International Journal of Scientific & Engineering Research, 4(5), 566-568.
Geiger, P., Schickler, M., Pryss, R., Schobel, J., & Reichert, M., (2014). Location-based mobile augmented
reality applications: Challenges, examples, lessons learned.
Bianconi, F., González E., Fernández, A., Saetta, A., (2012). Automatic classification of granite tiles through
colour and texture features. Expert Systems and Applications. Vol. 39, Nº 12, pp. 212–218.
Yang, X., Yuan, S., Tian, Y., (2011). Recognizing clothes patterns for blind people by confidence margin
based feature combination. In: ACM International Conference on Multimedia, pp. 1097–1100.
Thakare, V., Patil, N., Sonawane, J., (2013). Survey on image texture classification techniques. International
Journal of Advancements in Technology. Vol. 4, Nº 1, pp. 97–104.
Patel, B., Jaliya, U., Patel, K., (2013). A survey on content based image retrieval. International Journal for
Scientific Research & Development. Vol. 1, Nº 10, pp. 71–87.
González, E., Fernández, A., Bianconi, F., (2014). General framework for rotation invariant texture
classification through co-occurrence of patterns. Journal of Mathematical Imaging and Vision. Vol. 50, Nº 3,
pp. 300–313.
Bradski, G., (2000). The opencv library. Doctor Dobbs Journal. Vol. 25, Nº 11, pp. 120–126.
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I., (2009). The weka data mining
software: An update. SIGKDD Explorations. Vol. 11, Nº 1, pp. 300–313.
Rosten, E., Porter, R., Drummond, T., (2010). Faster and better: A machine learning approach to corner
detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 32, pp. 105–119.
Capizzi, G., Sciuto, G., Napoli, C., Tramontana, E., Wózniak, M., (2015). Automatic classification of fruit
defects based on co-occurrence matrix and neural networks. In: Federated Conference on Computer Science
and Information Systems, pp. 5–32.
Subrahmanyama, M., Wua, Q., Maheshwarib, R., Balasubramanian, R., (2013). Modified color motif
cooccurrence matrix for image indexing and retrieval. Computers & Electrical Engineering. Vol. 39, Nº 3, pp.
–774.
Chang, Y., Zaman, Q., Schumann, A., Percival, D., Esau, T., Ayalew, G., (2012). Development of color cooccurrence
matrix based machine vision algorithms for wild blueberry fields. Applied Engineering in
Agriculture. Vol. 28, Nº3, pp. 315–323.
Huang, W., Daib, J., Wu, Q., (2015). Image retrieval based on weighted block color histogram and texton cooccurrence
matrix. In: International Conference on Information Technology and Management Innovation, pp.
–853.
Li, H., Toyoura, M., Shimizu, K., Yang, W., Mao, X., (2016). Retrieval of clothing images based on
relevance feedback with focus on collar designs. The Visual Computer. Vol. 32, Nº 10, pp. 1351–1363.
Lin, H., Lin, C., (2010). Image retrieval system based on adaptive color histogram and texture features. The
Computer Journal. Vol. 54, Nº 7, pp. 1136–1147.
Bianconi, F., Fernández, A., González, E., Caride, D., Calviño, A., (2009). “Rotation-invariant colour texture
classification through multilayer CCR”. Pattern Recognition Letters. pp. 765–773.
Davis, L. S., Johns, S. A., Aggarwal, J. K, (1979). Texture analysis using generalized co-occurrence matrices.
IEEE Transactions on pattern analysis and machine intelligence v3. pp. 251-259.
Alcântara D, Brito S, Sanjad T (2016). Azulejaria em Belém do Pará - Inventário – Arquitetura civil e
religiosa – Século XVII ao XX. IPHAN.
