Auto-Extraction and Integration of Metrics for Web User Interfaces

Authors

  • Maxim Bakaev Novosibirsk State Technical University, Novosibirsk, Russia
  • Sebastian Heil Technische Universität Chemnitz, Chemnitz, Germany
  • Vladimir Khvorostov Novosibirsk State Technical University, Novosibirsk, Russia
  • Martin Gaedke Technische Universität Chemnitz, Chemnitz, Germany

Keywords:

Automated metrics, HCI Vision, web design mining, visual complexity

Abstract

Metric-based assessment of web user interface (WUI) quality attributes is shifting from code (HTML/CSS) analysis to mining webpages’visual representations based on image recognition techniques. In our paper, we describe a visual analysis tool which takes a WUI screenshot and produces structured and machine-readable representation (JSON) of the interface elements’ spatial allocation. The implementation is based on OpenCV (image recognition functions), dlib (trained detector for the elements’ classification), and Tesseract (label and content text recognition). The JSON representation is used to automatically calculate several metrics related to visual complexity, which is known to have major effect on user experience with UIs. We further describe a WUI measurement platform that allows integration of the currently dispersed sets of metrics from different providers and demonstrate the platform’s use with several remote services. We perform statistical analysis of the collected metrics in relation to complexity-related subjective evaluations obtained from 63 human subjects of various nationalities. Finally, we build predictive models for visual complexity and show that their accuracy can be improved by integrating the metrics from different sets. Regressions with the single index of visual complexity metric that we proposed had R2=0.460, while the best joint model with 4 metrics had R2=0.647.

Downloads

Download data is not yet available.

Author Biographies

Maxim Bakaev, Novosibirsk State Technical University, Novosibirsk, Russia

Maxim Bakaev got his PhD degree in 2012 and currently works as Associate Professor at the Automated Control Systems department of Novosibirsk State Technical University (NSTU), Russia. His research interests include Human-Computer Interaction, Web Design, User Behavior Models, and Machine Learning. He has served as reviewer for several international conferences and journals, including Large-Scale Control Systems, Algorithms, Multimodal Technologies and Interaction, etc. Under his supervision, more than 20 Master and Bachelor students graduated.

Sebastian Heil, Technische Universität Chemnitz, Chemnitz, Germany

Sebastian Heil PhD student at the Distributed and Self-organizing Systems Professorship of Chemnitz Technical University, Germany. His research interests include Web Engineering, Software Migration, Crowdsourcing and Collaborative Development, Linked Open Data and Web of Things. He is a member of Institute of Systems and Technologies of Information, Control and Communication and has served as reviewer and PC member for several international journals and conferences including ACM TWEB and MONET.

Vladimir Khvorostov, Novosibirsk State Technical University, Novosibirsk, Russia

Vladimir Khvorostov received his B.Sc (in 2002) and M.Sc. (in 2004) degrees in Information Systems from Novosibirsk State Technical University (NSTU), Russia. Vladimir held software developer position in 2004-2009, then Chief Engineer position in 2009-2015, and now he is individual entrepreneur in software development and Assistant Professor at NSTU. He has professional skills and wide experience in Web Programming, Software Development Management, Internet Advertising and E-Commerce.

Martin Gaedke, Technische Universität Chemnitz, Chemnitz, Germany

Martin Gaedke Full Professor, chair of Distributed and Self-organizing Systems group and Vice Dean of the Department of Computer Science, Chemnitz University of Technology. His research focuses on Web Engineering, Service Engineering, and trustworthy Service-based Systems Engineering to address needs of future distributed hyper-connected collaborative work environments. He serves as invited expert for W3C’s WebID and Social Web and served as reviewer for FP7 and ERC and as member of FP7 FAST Advisory Expert Group and NMC’s Horizon.JISC Advisory Board.

References

J. Grigera, et al., ‘Automatic detection of usability smells in web applications’, International Journal of Human-Computer Studies, 97, 2017, pp. 129–148.

A. Miniukovich, A. De Angeli, ‘Computation of interface aesthetics’, In Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1163–1172. ACM, 2015.

T. Schmidt, C. Wolff, ‘The Influence of User Interface Attributes on Aesthetics’, i-com 17, no. 1, 2018, pp. 41–55.

R. Rosenholtz, Y. Li, L. Nakano, ‘Measuring visual clutter’, Journal of Vision, 7, 2, 2007, pp. 1–22.

A. Oulasvirta, ‘Can computers design interaction?’, In Proc. of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 1–2. ACM, 2016.

M. Zen, J. Vanderdonckt, ‘Towards an evaluation of graphical user interfaces aesthetics based on metrics’, In IEEE Int Conf on Research Challenges in Information Science (RCIS), pp. 1–12, 2014.

A. Dingli, J. Mifsud, ‘Useful: A framework to mainstream web site usability through automated evaluation’, International Journal of Human Computer Interaction (IJHCI), 2(1), 10, 2011.

D. Cai, et al., ‘Extracting content structure for web pages based on visual representation’, In Asia-Pacific Web Conference, pp. 406–417, 2003.

R. Kumar, et al., ‘Bricolage: example-based retargeting for web design’, In Proc. of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 2197–2206, 2011.

A. Sanoja, B. Gançarski, ‘Block-o-matic: A web page segmentation framework’, In Proc. IEEE Int. Conf. Multimedia Computing and Systems (ICMCS), pp. 595–600, 2014.

J. Pu, J. Liu, J. Wang, ‘A Vision-Based Approach for Deep Web Form Extraction’, In Advanced Multimedia and Ubiquitous Engineering, pp. 696–702, Springer, 2017.

J. Kong, et al., ‘Web Interface Interpretation Using Graph Grammars’, IEEE Trans. Syst. Man, Cybern. Part C (Applications Rev.), 42 (4), pp. 590–602, 2012.

K. Reinecke, et al., ‘Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness’, In Proc. of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 2049–2058, 2013.

P. Machado, et al., ‘Computerized measures of visual complexity’, Acta psychologica, 160, pp. 43–57, 2015.

E. Michailidou, S. Harper, S. Bechhofer, ‘Visual complexity and aesthetic perception of web pages’, In Proc. of the 26th annual ACM international conference on Design of communication, pp. 215–224, ACM, 2008.

S.E.S. Taba, et al., ‘An exploratory study on the relation between user interface complexity and the perceived quality’, In International Conference on Web Engineering (ICWE), pp. 370–379, Springer, Cham, 2014.

V. Chikhman, et al., ‘Complexity of images: Experimental and computational estimates compared,’ Perception, 41(6), pp. 631–647, 2012.

A. Miniukovich, A. De Angeli, ‘Quantification of interface visual complexity. In Proc. of the 2014 international working conference on advanced visual interfaces, pp. 153–160, ACM, 2014.

M. Bakaev, O. Razumnikova, ‘Opredeleine slozhnosti zadach dlya zritelno-prostranstvennoi pamyati i propustkoi spospobnosti cheloveka-operatora’, Upravlenie bol’shimi sistemami = Large-Scale Systems Control, 70, pp. 25–57, 2017. – In Russian.

A. Carballal, et al., ‘Distinguishing paintings from photographs by complexity estimates’, Neural Computing and Applications, 30, no. 6, pp. 1957–1969, 2018.

A. Oulasvirta, et al., ‘Aalto Interface Metrics (AIM): A Service and Codebase for Computational GUI Evaluation’, In The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings, pp. 16–19. ACM, 2018.

O. Wu, W. Hu, L. Shi, ‘Measuring the visual complexities of web pages’, ACM Transactions on the Web (TWEB), 7(1), p.1, 2013.

K. Alemerien, K. Magel, ‘GUIEvaluator: A Metric-tool for Evaluating the Complexity of Graphical User Interfaces’, In SEKE, pp. 13–18, 2014.

C. Stickel, M. Ebner, A. Holzinger, ‘The XAOS metric– understanding visual complexity as measure of usability’, In Symposium of the Austrian HCI and Usability Engineering Group, pp. 278-290, Springer, Berlin, Heidelberg, 2010.

M. Bakaev, S. Heil, V. Khvorostov, M. Gaedke, ‘HCI Vision for Automated Analysis and Mining of Web User Interfaces’, In International Conference on Web Engineering (ICWE), pp. 136–144. Springer, Cham, 2018.

S.T. Bow, Pattern recognition and image preprocessing. New York: Marcel Dekker, 2002.

M. Bakaev, M. Gaedke, S. Heil, ‘Kansei Engineering Experimental Research with University Websites’, TU Chemnitz Technical Report, CSR-16-01, 2016.

M. Bakaev, T. Mamysheva, M. Gaedke, ‘Current trends in automating usability evaluation of websites: Can you manage what you can’t measure?’ In Proc. IEEE 11th International Forum on Strategic Technology (IFOST), pp. 510–514, 2016.

Downloads

Published

2019-02-23

How to Cite

Bakaev, M. ., Heil, S. ., Khvorostov, V. ., & Gaedke, M. . (2019). Auto-Extraction and Integration of Metrics for Web User Interfaces. Journal of Web Engineering, 17(6-7), 561–590. Retrieved from https://journals.riverpublishers.com/index.php/JWE/article/view/6305

Issue

2018: Vol 17 Iss 6-7

Section

Articles