ON CONTEXT-SENSITIVE USABILITY EVALUATION IN MOBILE HCI
Keywords:
Mobile HCI, Field Usability Evaluation, Context-Sensitivity, Wireless Sensor NetworkAbstract
In usability evaluations, experiments are often conducted in closed laboratory situations to avoid disturbing influences. Due to non-realistic usage assumptions, this approach has important shortcomings when mobile Human Computer Interactions (m-HCIs) have to be evaluated. On the other hand, field studies allow to perform experiments close to real-world conditions, but potentially introduce influences caused by the environment, which have not been fully investigated so far. With this work, we contribute to distinguishing application shortcomings from environmental disturbances which both potentially cause decreased user performance. Our approach is based on monitoring environmental conditions during the usability experiment, such as light, acceleration, sound, temperature, and humidity, and relating them to user actions. Therefore, a mobile context-framework has been developed based on a Wireless Sensor Network (WSN) carried together with a mobile PC. We present results of a small study (seven persons) in the lab which pointed at increased delays and error rates of user interactions under induced environmental disturbances. Hereby, we demonstrate the potential of environmental monitoring for understanding user performance. Additionally, we present novel results of a usability study carried out in the field where we tested 19 persons under varying environmental conditions. The results showed that error rate and delay are influenced by environmental parameters, but in a more complex way than expected a-priori.
Downloads
References
M. Aleksy, T. Butter, and M. Schader. Architecture for the Development of Context-Sensitive
Mobile Applications. Mobile Information Systems, 4(2):105–117, 2008.
M. Assad, D. J. Carmichael, J. Kay, and B. Kummerfeld. PersonisAD: Distributed, Active,
Scrutable Model Framework for Context-Aware Services. In Pervasive Computing 2007, pages
–72, 2007.
J. Bardram. The Java Context Awareness Framework (JCAF) - A Service Infrastructure and
Programming Framework for Context-Aware Applications. In Pervasive Computing 2005, pages
–115, 2005.
L. Barnard, J. Yi, J. Jacko, and A. Sears. Capturing the Effects of Context on Human Performance
in Mobile Computing Systems. Personal and Ubiquitous Computing, 11(2):81–96, 2007.
H. W. Bristow, C. Baber, J. Cross, J. F. Knight, and S. I. Woolley. Defining and Evaluating Con
text for Wearable Computing. International Journal of Human-Computer Studies, 60(5–6):798–
, 2004.
C. Challiol, A. Fortier, S. Gordillo, and G. Rossi. Architectural and Implementation Issues for a
Context-Aware Hypermedia Platform. Mobile Multimedia, 4(2):118–138, 2008.
D. Chen, A. Schmidt, and H.-W. Gellersen. An Architecture for Multi-Sensor Fusion in Mobile
Environments. In International Conference on Information Fusion, pages 861–868, 1999.
K. Cheverst, N. Davies, K. Mitchell, A. Friday, and C. Efstratiou. Developing a Context-Aware
Electronic Tourist Guide: Some Issues and Experiences. In SIGCHI Conference on Human Factors
in Computing Systems, pages 17–24, 2000.
Crossbow Technology Incorporated. MPR-MIB Users Manual, June 2006.
Crossbow Technology Incorporated. MTS/MDA Sensor Board Users Manual, June 2006.
H.-L. Duh, G. Tan, and V.-H. Chen. Usability Evaluation for Mobile Device: A Comparison
of Laboratory and Field Tests. In 8th Conference on Human-Computer Interaction with Mobile
Devices and Services, pages 181–186, New York, NY, USA, 2006.
D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, and D. Culler. The nesC Language: A
Holistic Approach to Networked Embedded Systems. In ACM SIGPLAN 2003 Conference on
Programming Language Design and Implementation, pages 1–11, 2003.
T. Gu, H. Pung, and D. Zhang. A Service-Oriented Middleware for Building Context-Aware
Services. Journal of Network and Computer Applications, 28(1):1–18, 2005.
S. Gulliver, G. Ghinea, M. Patel, and T. Serif. A Context-Aware Tour Guide: User Implications.
Mobile Information Systems, 3(2):71–88, 2007.
J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. Culler, and K. Pister. System Architecture Directions
for Networked Sensors. ACM SIGPLAN Notices, 35(11):93–104, 2000.
P. Hu, J. Indulska, and R. Robinson. An Autonomic Context Management System for Pervasive
Computing. In Pervasive 2008, pages 213–223, 2008.
K. A. Hummel, A. Hess, and T. Grill. Environmental Context Sensing for Usability Evaluation
in Mobile HCI by Means of Small Wireless Sensor Networks. In 6th International Conference on
Advances in Mobile Computing and Multimedia Workshops, pages 302–306, 2008.
J. Kjeldskov and C. Graham. A Review of Mobile HCI Research Methods. In 5th International
Symposium on Human-Computer Interaction with Mobile Devices and Services, pages 317–335,
J. Kjeldskov and M. Skov. Exploring Context-Awareness for Ubiquitous Computing in the Healthcare
Domain. Personal and Ubiquitous Computing, 11(7):549–562, 2007.
T. Mustonen, M. Olkkonen, and J. Hakkinen. Examining Mobile Phone Text Legibility While
Walking. In CHI’04 Extended Abstracts on Human Factors in Computing Systems, pages 1243–
, 2004.
S. Pedell, C. Graham, J. Kjeldskov, and J. Davies. Mobile Evaluation: What the Data and the
Metadata Told Us. In OZCHI 2003, pages 96–105, 2003.
K. Pousttchi and B. Thurnher. Understanding Effects and Determinants of Mobile Support Tools:
A Usability-Centered Field Study on IT Service Technicians. In ICMB ’06: International Conference
on Mobile Business, page 10, 2006.
C. Ryan and A. Gonsalves. The Effect of Context and Application Type on Mobile Usability:
An Empirical Study. In ACSC ’05: 28th Australasian Conference on Computer Science, pages
–124, 2005.
D. Salber, A. Dey, and G. Abowd. The Context Toolkit: Aiding the Development of Context-
Enabled Applications. In SIGCHI Conference on Human Factors in Computing Systems, pages
–441, 1999.
