Correlating Cross-Platform Usability Problems with Eye Tracking Patterns

Evaluating the cross-platform usability of multiple interactive systems has become increasingly essential. Despite eye tracking being used to supplement traditional usability assessment, there is little research on its use for cross-platform usability evaluation. Our exploratory study seeks relationship between eye-tracking metrics and cross-platform usability problems. We user-tested three cross-platform services and identified a set of usability problems. We separated the identified problems into traditional and cross-platform usability problems. Some of the cross-platform usability problems were associated with users’ eye-tracking patterns. We found that consistency on many levels is a major problem cross-platform and we recommend some considerations for evaluators to use as indicators to predict possible cross-platform usability problems.

Content

Author and article information

Contributors

Khalid Majrashi

Margaret Hamilton

Alexandra L. Uitdenbogerd

Conference

Publication date: July 2016

Publication date (Print): July 2016

Pages: 1-11

Affiliations

[0001]School of Computer Science & IT

RMIT University, Australia

[0002]Institute of Public Administration, Saudi Arabia

Article

DOI: 10.14236/ewic/HCI2016.40

SO-VID: cd630531-975c-40b2-a1d6-4ccf6d354706

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 30th International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 30

Conference location: Bournemouth University, Poole, UK

Conference date: 11 - 15 July 2016

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Fusion

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2016.40

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

M. D. ByrneJ. R. AndersonS. DouglassM. Matessa 1999 Eye tracking the visual search of click-down menus M. G. WilliamsM. W. Altom Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Pittsburgh, PA, USA ACM 402 409
S. C. CuadratM. S. VelosoJ. J. R. Soler 2012 Towards the validation of a method for quantitative mobile usability testing based on desktop eyetracking Proceedings of the 13th International Conference on Interacción Persona-Ordenador. Elche, Spain ACM 49
D. DahanJ. S. MagnusonM. K. Tanenhaus 2001 Time course of frequency effects in spoken-word recognition: Evidence from eye movements Cognitive psychology 42 4 317 367
C. DenisL. Karsenty 2004 Inter-usability of multi-device systems: A conceptual framework S. AhmedJ. Homa Multiple user interfaces: Cross-platform applications and context-aware interfaces. New York Wiley 373 384
C. EhmkeS. Wilson 2007 Identifying web usability problems from eye-tracking data T. OrmerodC. Sas Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1. Swinton, UK ACM 119 128
J. H. GoldbergX. P. Kotval 1999 Computer interface evaluation using eye movements: methods and constructs International Journal of Industrial Ergonomics 24 6 631 645
Z. GuanS. LeeE. CuddihyJ. Ramey 2006 The validity of the stimulated retrospective think-aloud method as measured by eye tracking R. GrinterT. RoddenP. AokiE. CutrellR. JeffriesG. Olson Proceedings of the SIGCHI conference on Human Factors in computing systems. Montreal, Canada ACM 1253 1262
O. HohlfeldA. PompJ. Á. B. LinkD Guse 2015 On the Applicability of Computer Vision based Gaze Tracking in Mobile Scenarios S. BoringE. RukzioH. GellersenK. Hinckley Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM 427 434
R. J. JacobK. S. Karn 2003 Eye tracking in human-computer interaction and usability research: Ready to deliver the promises Mind 2 3 4
M. A. JustP. A. Carpenter 1976 Eye fixations and cognitive processes Cognitive psychology 8 4 441 480
K. MajrashiM. Hamilton 2015 A crossplatform usability measurement model Lecture Notes on Software Engineering 3 2 132
K. MajrashiM. HamiltonA. L Uitdenbogerd 2014 Cross-platform usability and eye-tracking measurement and analysis model T. RobertsonK. O’HaraG. WadleyL. LokeT. Leong Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design. ACM 418 421
K. MajrashiM. HamiltonA. L Uitdenbogerd 2015 Multiple user interfaces and crossplatform user experience: Theoretical foundations S. VaidyanathanD. Nagamalai Proceedings of the Fifth International Conference on Computer Science, Engineering and Applications (CCSEA-2015). AIRCC Publishing Corporation 43 57
J. A. McCallP. K. RichardsG. F. Walters 1977 Factors in software quality: General Electric, National Technical Information Service
J. NielsenK. Pernice 2010 Eyetracking web usability New Riders
A. PooleL. J. Ball 2005 Eye tracking in human-computer interaction and usability research: Current status and future Prospects C. Ghaoui ed) Encyclopedia of HumanComputer Interaction. Pennsylvania Idea Group 211 219
K. Rayner 1998 Eye movements in reading and information processing: 20 years of research Psychological bulletin 124 3 372
J. RenshawJ. FinlayD. TyfaR. D. Ward 2003 Designing for visual influence: An eye tracking study of the usability of graphical management information Proceedings of the IFIP conference on Human-computer interaction (INTERACT 2003). Zurich, Switzerland IOS Press 144 151
A. SeffahP. ForbrigH. Javahery 2004 Multidevices “Multiple” user interfaces: development models and research opportunities Journal of Systems and Software 73 2 287 300
Tobii Technology Inc 2010 Using eye tracking to test mobile devices what to consider and how to set it up. Available at http://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/whitepapers/tobii_pro_whitepaper_mobiledevicetesting.pdf February 28 2016
M. Van Den HaakM. De JongP. Jan Schellens 2003 Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue Behaviour & Information Technology 22 5 339 351
M. WäljasK. SegerståhlK. Väänänen-Vainio-MattilaH. Oinas-Kukkonen 2010 Cross-platform service user experience: a field study and an initial framework Proceedings of the 12th international conference on Human computer interaction with mobile devices and services Lisbon, Portugal ACM 219 228

Comments

Comment on this article

[1] M. D. ByrneJ. R. AndersonS. DouglassM. Matessa 1999 Eye tracking the visual search of click-down menus M. G. WilliamsM. W. Altom Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Pittsburgh, PA, USA ACM 402 409

[2] S. C. CuadratM. S. VelosoJ. J. R. Soler 2012 Towards the validation of a method for quantitative mobile usability testing based on desktop eyetracking Proceedings of the 13th International Conference on Interacción Persona-Ordenador. Elche, Spain ACM 49

[3] D. DahanJ. S. MagnusonM. K. Tanenhaus 2001 Time course of frequency effects in spoken-word recognition: Evidence from eye movements Cognitive psychology 42 4 317 367

[4] C. DenisL. Karsenty 2004 Inter-usability of multi-device systems: A conceptual framework S. AhmedJ. Homa Multiple user interfaces: Cross-platform applications and context-aware interfaces. New York Wiley 373 384

[5] C. EhmkeS. Wilson 2007 Identifying web usability problems from eye-tracking data T. OrmerodC. Sas Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1. Swinton, UK ACM 119 128

[6] J. H. GoldbergX. P. Kotval 1999 Computer interface evaluation using eye movements: methods and constructs International Journal of Industrial Ergonomics 24 6 631 645

[7] Z. GuanS. LeeE. CuddihyJ. Ramey 2006 The validity of the stimulated retrospective think-aloud method as measured by eye tracking R. GrinterT. RoddenP. AokiE. CutrellR. JeffriesG. Olson Proceedings of the SIGCHI conference on Human Factors in computing systems. Montreal, Canada ACM 1253 1262

[8] O. HohlfeldA. PompJ. Á. B. LinkD Guse 2015 On the Applicability of Computer Vision based Gaze Tracking in Mobile Scenarios S. BoringE. RukzioH. GellersenK. Hinckley Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM 427 434

[9] R. J. JacobK. S. Karn 2003 Eye tracking in human-computer interaction and usability research: Ready to deliver the promises Mind 2 3 4

[10] M. A. JustP. A. Carpenter 1976 Eye fixations and cognitive processes Cognitive psychology 8 4 441 480

[11] K. MajrashiM. Hamilton 2015 A crossplatform usability measurement model Lecture Notes on Software Engineering 3 2 132

[12] K. MajrashiM. HamiltonA. L Uitdenbogerd 2014 Cross-platform usability and eye-tracking measurement and analysis model T. RobertsonK. O’HaraG. WadleyL. LokeT. Leong Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design. ACM 418 421

[13] K. MajrashiM. HamiltonA. L Uitdenbogerd 2015 Multiple user interfaces and crossplatform user experience: Theoretical foundations S. VaidyanathanD. Nagamalai Proceedings of the Fifth International Conference on Computer Science, Engineering and Applications (CCSEA-2015). AIRCC Publishing Corporation 43 57

[14] J. A. McCallP. K. RichardsG. F. Walters 1977 Factors in software quality: General Electric, National Technical Information Service

[15] J. NielsenK. Pernice 2010 Eyetracking web usability New Riders

[16] A. PooleL. J. Ball 2005 Eye tracking in human-computer interaction and usability research: Current status and future Prospects C. Ghaoui ed) Encyclopedia of HumanComputer Interaction. Pennsylvania Idea Group 211 219

[17] K. Rayner 1998 Eye movements in reading and information processing: 20 years of research Psychological bulletin 124 3 372

[18] J. RenshawJ. FinlayD. TyfaR. D. Ward 2003 Designing for visual influence: An eye tracking study of the usability of graphical management information Proceedings of the IFIP conference on Human-computer interaction (INTERACT 2003). Zurich, Switzerland IOS Press 144 151

[19] A. SeffahP. ForbrigH. Javahery 2004 Multidevices “Multiple” user interfaces: development models and research opportunities Journal of Systems and Software 73 2 287 300

[20] Tobii Technology Inc 2010 Using eye tracking to test mobile devices what to consider and how to set it up. Available at http://www.tobiipro.com/siteassets/tobii-pro/learn-and-support/whitepapers/tobii_pro_whitepaper_mobiledevicetesting.pdf February 28 2016

[21] M. Van Den HaakM. De JongP. Jan Schellens 2003 Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue Behaviour & Information Technology 22 5 339 351

[22] M. WäljasK. SegerståhlK. Väänänen-Vainio-MattilaH. Oinas-Kukkonen 2010 Cross-platform service user experience: a field study and an initial framework Proceedings of the 12th international conference on Human computer interaction with mobile devices and services Lisbon, Portugal ACM 219 228

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65