Applying Importance-Performance Analysis (IPA) to Interpret the Results of the User Experience Questionnaire (UEQ)

Authors

  • Andreas Hinderks Department of Computer Languages and Systems, University of Seville, Seville, Spain
  • Francisco José Domínguez-Mayo Department of Computer Languages and Systems, University of Seville, Seville, Spain
  • Anna-Lena Meiners University of Applied Sciences Emden/Leer, Germany
  • Jörg Thomaschewski University of Applied Sciences Emden/Leer, Germany

DOI:

https://doi.org/10.13052/jwe1540-9589.1926

Keywords:

Importance-performance analysis, IPA, user experience, UX factors, User Experience Questionnaire, UEQ

Abstract

In recent years, user experience questionnaires have established themselves to measure various aspects of User Experience (UX). In addition to these questionnaires, an evaluation tool is usually offered so that the results of a study can be evaluated in the light of the questionnaire. As a rule, the evaluation consists of preparing the data and comparing it with a benchmark. Often this interpretation of the data is not sufficient as it only evaluates the current User Experience. However, it is desirable to determine exactly where there is a need for action. The User Experience Questionnaire (UEQ) is a common and valid questionnaire with an evaluation tool to measure and analyse the User Experience for a product or service. In our article we present an approach that evaluates the results from the User Experience Questionnaire using the importance-performance analysis (IPA). The aim is to create another possibility to interpret the results of the UEQ and to derive recommendations for action from them. In a study with 467 participants, we validated the approach presented with YouTube, WhatsApp, and Facebook. The results show that the IPA provides additional insights from which further recommendations for action can be derived.

Downloads

Download data is not yet available.

Author Biographies

Andreas Hinderks, Department of Computer Languages and Systems, University of Seville, Seville, Spain

Andreas Hinderks holds a diploma in Computer Science and is Master of Science in Media Informatics by University of Applied Science Emden/Leer. He has worked in various management roles as a Business Analyst and a programmer from 2001 to 2016. His focus lay on developing user-friendly business software. Currently, he is a freelancing Product Owner, Business Analyst and Senior UX Architect. Also, he is a Ph.D. student at the University of Seville. He is involved in research activities dealing with UX questionnaires, measuring user experience and User Experience Management since 2011.

Francisco José Domínguez-Mayo, Department of Computer Languages and Systems, University of Seville, Seville, Spain

Francisco José Domínguez-Mayo received the Ph.D. degree in computer science from the University of Seville, Seville, Spain, in July 2013. He is currently an associate professor with the Department of Computing Languages and Systems, University of Seville. He collaborates with public and private companies in software development quality and quality assurance. His lines of interesting research are plotted in the areas of continuous quality improvement and quality assurance on software products, and software development processes.

Anna-Lena Meiners, University of Applied Sciences Emden/Leer, Germany

Anna-Lena Meiners holds a bachelor degree in Theatre Studies, Philosophy and Dutch Language and Literature from Freie Universität Berlin. Currently, she is completing her bachelor degree in Computer Science and Digital Media at University of Applied Sciences Emden/Leer and working as a project manager in web development. In her research she is focussing on different quality aspects of UX factors and metrics.

Jörg Thomaschewski, University of Applied Sciences Emden/Leer, Germany

Jörg Thomaschewski received a Ph.D. in physics from the University of Bremen (Germany) in 1996. He became a Full Professor at the University of Applied Sciences Emden/Leer (Germany) in September 2000. His teaching and research focus is on Human-Computer Interaction, UX-Management, Agile Software Development, and Requirements Engineering. Since 2012 he is the head of the research group “Agile Software Development and User Experience”. Dr. Thomaschewski has extensive experience in user experience training, UX questionnaires, agile methods, IT analysis, and consulting.

References

Abalo J, Varela J, Manzano V (2007) Importance values for Importance-Performance Analysis: A formula for spreading out values derived from preference rankings. Journal of Business Research 60(2): 115–121. doi: 10.1016/j.jbusres.2006.10.009

Azzopardi E, Nash R (2013) A critical evaluation of importance-performance analysis. Tourism Management 35: 222–233. doi: 10.1016/j.tourman.2012.07.007

Bacon DR (2003) A Comparison of Approaches to Importance-Performance Analysis. International Journal of Market Research 45(1): 1–15. doi: 10.1177/147078530304500101

Chang SE, Liu AY, Shen WC (2017) User trust in social networking services: A comparison of Facebook and LinkedIn. Computers in Human Behavior 69: 207–217. doi: 10.1016/j.chb.2016.12.013

Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16(3): 297–334. doi: 10.1007/BF02310555

Danaher PJ (1997) Using conjoint analysis to determine the relative importance of service attributes measured in customer satisfaction surveys. Journal of Retailing 73(2): 235–260. doi: 10.1016/S0022-4359(97)90005-1

Danaher PJ, Mattsson J (1994) Customer Satisfaction during the Service Delivery Process. European Journal of Marketing 28(5): 5–16. doi: 10.1108/03090569410062005

Dwyer L, Cvelbar LK, Edwards D et al. (2012) Fashioning a destination tourism future: The case of Slovenia. Tourism Management 33(2): 305–316. doi: 10.1016/j.tourman.2011.03.010

Hassenzahl M (2001) The Effect of Perceived Hedonic Quality on Product Appealingness. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION(13(4)): 481-499

Hinderks A (2016) UEQ – User Experience Questionnaire. http://www.ueq-online.org. Accessed 24 Mar 2016

Hinderks A, Schrepp M, Mayo FJD et al. (2019) Developing a UX KPI based on the User Experience Questionnaire. Computer Standards & Interfaces. doi: 10.1016/j.csi.2019.01.007

Hinderks A, Meiners A-L, Mayo F et al. (2019) Interpreting the Results from the User Experience Questionnaire (UEQ) using Importance-Performance Analysis (IPA). In: Proceedings of the 15th International Conference on Web Information Systems and Technologies. SCITEPRESS – Science and Technology Publications, pp 388–395

Hinderks A, Winter D, Schrepp M et al. (2020) Applicability of User Experience and Usability Questionnaires. Journal of Universal Computer Science(25): 1717–1735

ISO9241-210 (2010) Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems (ISO 9241-210:2010)

Landauer TK, Galotti KM, Hartwell S (1983) Natural command names and initial learning. A study of text-editing terms. Commun. ACM 26(7): 495–503. doi: 10.1145/358150.358157

Laugwitz B, Held T, Schrepp M (2008) Construction and Evaluation of a User Experience Questionnaire. In: Holzinger A (ed) HCI and Usability for Education and Work, vol 5298. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 63–76

Lazar J, Feng JH, Hochheiser H (2010) Research methods in human-computer interaction. Wiley, Chichester, West Sussex, U.K

Martilla JA, James JC (1977) Importance-Performance Analysis. Journal of Marketing Management 1977(41): 77–79

Moshagen M, Thielsch MT (2010) Facets of visual aesthetics. International journal of human-computer studies 68(10): 689–709. doi: 10.1016/j.ijhcs.2010.05.006

Norman DA (2007) Emotional Design. Why We Love (or Hate) Everyday Things. Basic Books, New York

Nunnally JC, Bernstein IH (2010) Psychometric theory, 3. ed., Tata McGraw-Hill ed. McGraw-Hill higher education. Tata McGraw Hill Education Private Ltd, New Delhi

Öz M (2012) A research to evaluate the airline companies’ websites via a consumer oriented approach. Afr. J. Bus. Manage. 6(14). doi: 10.5897/AJBM11.2145

Pokryshevskaya E, Antipov E (2013) Importance-Performance Analysis for Internet Stores: A System Based on Publicly Available Panel Data. SSRN Journal. doi: 10.2139/ssrn.2257770

Preece J, Rogers Y, Sharp H (2015) Interaction design. Beyond human-computer interaction, 4. ed. Wiley, Chichester

Sauro J (2015) SUPR-Q: A Comprehensive Measure of the Quality of the Website User Experience. Journal of Usability Studies 2015(10): 68–86

Schrepp M, Thomaschewski J (2019) Design and Validation of a Framework for the Creation of User Experience Questionnaires. IJIMAI InPress(InPress): 1. doi: 10.9781/ijimai.2019.06.006

Schrepp M, Hinderks A, Thomaschewski J (2017) Construction of a Benchmark for the User Experience Questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Inteligence 4(4): 40–44. doi: 10.9781/ijimai.2017.445

Tontini G (2016) Identifying opportunities for improvement in online shopping sites. Journal of Retailing and Consumer Services 31: 228–238. doi: 10.1016/j.jretconser.2016.02.012

Tractinsky N (1997) Aesthetics and apparent usability. In: Pemberton S (ed) the SIGCHI conference, pp 115–122

Downloads

Published

2020-06-03

How to Cite

Hinderks, A., Domínguez-Mayo, F. J., Meiners, A.-L., & Thomaschewski, J. (2020). Applying Importance-Performance Analysis (IPA) to Interpret the Results of the User Experience Questionnaire (UEQ). Journal of Web Engineering, 19(2), 243–266. https://doi.org/10.13052/jwe1540-9589.1926

Issue

2020: Vol 19 Iss 2

Section

SPECIAL ISSUE: Advanced Practices in Web Engineering 2020