Using Eye Tracking to Understand the Fidelity Effect when Evaluating Low-Fidelity Prototypes with Children

This study used eye tracking glasses to understand how children explore low-fidelity paper prototypes in the context of user experience studies. Twenty seven children aged between 9 and 11 participated in the study examining either a colour or black and white prototype of a mobile game. The research question being explored was whether the aesthetic refinement, either wireframe or high-resolution colour images, would affect children’s self-report and if so what could be learned from knowing where children looked when exploring the prototypes. The results showed that the aesthetic refinement had little influence over the children’s overall ratings of the game. The eye tracking data demonstrated that most of the children focused on both the visuals and text on the pages of the prototype. However, there were a higher number of fixations recorded in the wireframe prototype suggesting this may have been more cognitively demanding. This paper contributes to understanding of fidelity effects when evaluating low-fidelity prototypes and shows how eye tracking technology can help inform HCI methodologies.

Content

Author and article information

Contributors

Gavin Sim

Janet C Read

Conference

Publication date: July 2022

Publication date (Print): July 2022

Pages: 1-10

Affiliations

[0001]University of Central Lancashire

Preston, UK

Article

DOI: 10.14236/ewic/HCI2022.21

SO-VID: 9672d253-3398-4c42-bb85-fe24cbcf97a7

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 35th International BCS Human-Computer Interaction Conference

Conference acronym: HCI2022

Conference number: 35

Conference location: Keele, Staffordshire

Conference date: July 11th to 13th, 2022

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Towards a Human-Centred Digital Society

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2022.21

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

Al-Wabil, A., Al-Husian, L. Al-Murshad, R. and Al-Nafjan. A. (2010). Applying the retrospective think-aloud protocol in usability evaluations with children: Seeing through children's eyes. In Proceedings of the International Conference on User Science and Engineering. 13-15 Dec., 2010.
Al-Zeer, S. Al-Ghanim, A. and Al-Wakeel. L.A. (2010). Visual attention in interaction with Arabic augmentative and alternative communication apps. In Proceedings of the International Conference on User Science and Engineering. 2-5 Sept. 2014.
Bertou, E. and Shahid, S. (2014). Lowfidelity prototyping tablet applications for children. Proceedings of the Interaction design and children 2014.
Bilal, D. and Gwizdka, J. (2016) Children's eye-fixations on google search results. In Proceedings of the 79th ASIS Annual Meeting: Creating Knowledge, Enhancing Lives through Information Technology. Copenhagen, Denmark. American Society for Information Science.
Buxton, B. Sketching User Experiences - getting the design right and right design. (2007). Morgan Kaufmann, San Francisco.
Chen, Y. and Tsai, M.Y. (2015) Eye-hand coordination strategies during active video game playing: An eye-tracking study. Computers in Human Behavior, 51.
Currie, J. Bond, R., McCullagh, P. Black, P., Finlay D., and Peace, A. (2017). Eye tracking the visual attention of nurses interpreting simulated vital signs scenarios: mining metrics to discriminate between performance level. IEEE Transactions on Human-Machine Systems, 48, 2, 113-124.
Derboven, J. Roeck, D.D., Verstraete, M., Geerts, D., Schneider-Barnes, J. and Luten, K. (2010). Comparing User Interaction with Low and High Fidelity Prototypes of Tabletop Surfaces. In Proceedings of the NordiCHI. Reykjavik, Iceland. ACM.
Fischer, P.M., Richards, J.W., Berman E.J., and Krugman, D.M. (1989) Recall and eye tracking study of adolescents viewing tobacco advertisements. Jama, 261, 1, 84-89
.Giannakos, M.N., Papavlasopoulou, S. and Sharma, K. (2020). Monitoring Children's Learning Through Wearable Eye-Tracking: The Case of a Making-Based Coding Activity. IEEE Pervasive Computing, 19, 1.
Gomes, A., Oh, H., Chisik, Y. and Chen, M. (2012) Ilha Musical: a CAVE for nurturing cultural appreciation. In Proceedings of the Proceedings of the 11th International Conference on Interaction Design and Children. Bremen, Germany.
Gossen, T., Höbel, J., and Nürnberger, A. (2014). Usability and perception of young users and adults on targeted web search engines. In Proceedings of the 5th Information Interaction in Context Symposium. Regensburg, Germany.
Hanna, L., Neapolitan, D. and Risden, K. (2004) Evaluating computer game concepts with children. Proceedings of the 2004 conference on Interaction design and children: building a community.
Hershman, A., Nazare, J., Qi, J., Saveski, M., Roy, D. and Resnick, M. (2018). Light it up: Using paper circuitry to enhance low-fidelity paper prototypes for children. Proceedings of the 17th ACM conference on interaction design and children.
Hirschmanner, M., Lammer, L. and Vincze, M. (2015). Mattie: A simple educational platform for children to realize their first robot prototype. Proceedings of the 14th international conference on interaction design and children
Hoggenmüller, M., Tomitsch, M., Hespanhol, L., Tran, T.T.M., Worrall, S. and Nebot, E. (2021). Context-Based Interface Prototyping: Understanding the Effect of Prototype Representation on User Feedback. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.
Holmqvist, K. and Andersson, R. (2017)/ Eye tracking: A comprehensive guide to methods. paradigms and measures.
Huestegge, L., Radach, R., Corbic, D. and Huestegge, S.M. (2009). Oculomotor and linguistic determinants of reading development: A longitudinal study. Vision research, 49, 24, 2948-2959
Kleberg, J.L., Högström, J., Sundström, K., Frick, A. and Serlachius, E. (2020). Delayed gaze shifts away from others’ eyes in children and adolescents with social anxiety disorder. Journal of Affective Disorders.
Kohler, B., Haladjian, J., Simeonova, B. and Ismailovic, D. (2012) Feedback in Low vs High Fidelity Visuals for Game Prototypes. In Proceedings of the Games and Software Engineering. Zurich, 2012. IEEE.
A. N. Kornev, I. Balciuniene and S. R. Oganov. (2018). The strategic reading brain development: An eye-tracking study of the text reading in typically-developing and dyslexic children. International Journal of Psychophysiology, 131.
Krejtz, K., Krejtz, I., Duchowski, A., Szarkowska, A. and Walczak, A. (2012) Multimodal learning with audio description: an eye tracking study of children's gaze during a visual recognition task. In Proceedings of the ACM Symposium on Applied Perception. Los Angeles, California.
Levi, M.D. and Conrad, F.G. (1996). A heuristic evaluation of a world wide web prototype. Interactions, 3, 4, 50-61.
Lim, Y-K., Pangam, A., Periyasami, S. and Aneja, S. (2006). Comparative Analysis of High- and Low-fidelity Prototypes for More Valid Usability Evaluations of Mobile Devices. In Proceedings of the NordiCHI. Oslo, Norway. ACM.
Just, M.A. and Carpenter, P.A. (1980). A theory of reading: from eye fixations to comprehension. Psychol Rev, 87, 4, 329-354
Martín-Arnal, L.A., León, J.A. van den Broek, P. and Olmos, R. (2019). Understanding comics. A comparison between children and adults through a coherence/incoherence paradigm in an eye-tracking study. Psicología Educativa. Revista de los Psicólogos de la Educación, 25, 2, 127-137.
Moeller, K., Neuburger, S., Kaufmann, L., Landerl, K. and Nuerk, H. C. (2009). Basic number processing deficits in developmental dyscalculia: Evidence from eye tracking. Cognitive Development, 24, 4.
Othman, A. and Mohsin, M. (2017). How could robots improve social skills in children with Autism? 6th International Conference on Information and Communication Technology and Accessibility (ICTA).
Papavlasopoulou, S., Sharma, K. and Giannakos, M.N. (2020) Coding activities for children: Coupling eye-tracking with qualitative data to investigate gender differences. Computers in Human Behavior, 105.
Kent, M.P. Pauzé, E., Roy, E-A., de Billy N. and Czoli, C. (2019). Children and adolescents' exposure to food and beverage marketing in social media apps. Pediatric obesity, 14, 6.
Rayner, K. Pollatsek, A., Ashby, J and Clifton Jr, C. (2012). Psychology of reading. Psychology Press.
Read, J.C. MacFarlane, S and Casey, C. (2002). Endurability, Engagement and Expectations: Measuring Children's Fun. In Proceedings of the Interaction Design and Children. Eindhoven.
Read, J.C. (2012). Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun. In Proceedings of the 14th ACM International Conference on Multimodal Interaction. Santa Monica. ACM.
Reilly, D., Dearman, D., Welsman-Dinelle, M. and Inkpen, K. (2005). Evalusating Early Prototypes in Context: Trade-offs, Challenges, and Successe. Pervasive Computing, 4, 4, 42-50.
Rudd, J. Stern, K. and Isesensee, S. (1996).Low vs high-fidelity prototyping debate. Interactions, 3, 1, 75-85
Sauer, J., Holger, F. and Ruttinger, B. (2008). Designing interactive consumer products: utility of low-fidelity prototypes and effectiveness of enhanced control labeling. Applied Ergonomics, 39, 1, 71-85.
Sauer, J. and Sonderegger, A. (2009). The influence of prototype fidelity and aesthetics of design in usability tests: Effects on user behaviour, subjective evaluation and emotion. Applied Ergonomics, 40, 670-677.
Sefelin, R., Tscheligi, M. and Giller, V. (2003) Paper Prototyping - What is it good for? A comparison of Paper and Computer based Low fidelity prototyping. In Proceedings of the Conference on Human Factors in Computing Systems. Fort Lauderdale.
Sharma, K., Papavlasopoulou, S. and Giannakos, M.N. (2019). Coding games and robots to enhance computational thinking: How collaboration and engagement moderate children’s attitudes? International Journal of Child-Computer Interaction, 21.
Sim, G., & Cassidy, B. (2013). Investigating the fidelity effect when evaluating game prototypes with children. Paper presented at the Proceedings of the 27th International BCS Human Computer Interaction Conference, London, UK.
Sim, G., Cassidy, B., & Read, J. C. (2013). Understanding the Fidelity Effect when Evaluating Games with children. Paper presented at the Interaction Design and Children, New York.
Sim, G., Horton, M., & McKnight, L. (2016). iPad vs Paper Prototypes: Does Form Factor Affect Children's Ratings of a Game Concept? Paper presented at the Proceedings of the The 15th International Conference on Interaction Design and Children.
Sim, G., & Bond, R. (2021). Eye tracking in child computer interaction: Challenges and opportunities. International Journal of Child-Computer Interaction. 30.
Snyder, C. (2003) Paper Prototyping: The fast and Easy Way to Define and Refine User Interfaces. Morgan Kaufmann, San Francisco.
Su,Q., Chen, F., Li, H., Yan, N. and Wang, L. (2018). Multimodal Emotion Perception in Children with Autism Spectrum Disorder by Eye Tracking Study. Conference on Biomedical Engineering and Sciences (IECBES).
Thomsen, S.R. and Fulton, K. (2007). Adolescents’ Attention to Responsibility Messages in Magazine Alcohol Advertisements: An Eye-Tracking Approach. Journal of Adolescent Health, 41, 1.
Torney, H., Harvey, A., Finlay, D., Magee, J., Funston, R. and Bond, R. (2018) Eye-tracking analysis to compute the visual hierarchy of user interfaces on automated external defibrillators. British HCI Conference 2018.
Mestres, E.T. and Pellicer-Sánchez, A. (2019) Young EFL learners' processing of multimodal input: Examining learners’ eye movements. System, 80.
Velazquez, C.E. and Pasch, K.E. (2014). Attention to food and beverage advertisements as measured by eye-tracking technology and the food preferences and choices of youth. Journal of the Academy of Nutrition and Dietetics, 114, 4, 578-582.
Virzi, R.A. Sokolov, J.L. and Karis, D. (1996) Usability problem identification using both low- and high-fidelity prototypes. In Proceedings of the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Vancouver, British Columbia, Canada.
von Reumont, R. and Budke, A. (2020) Strategies for Successful Learning with Geographical Comics: An Eye-Tracking Study with Young Learners. Education Sciences, 10, 10, 293.
Wiklund, M., Thurrot, C. and Dumas, J. (1992)Does the Fidelity of Software Prototypes Affect the Perception of Usability. In Proceedings of the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Atlanta, USA.

Comments

Comment on this article

[1] Al-Wabil, A., Al-Husian, L. Al-Murshad, R. and Al-Nafjan. A. (2010). Applying the retrospective think-aloud protocol in usability evaluations with children: Seeing through children's eyes. In Proceedings of the International Conference on User Science and Engineering. 13-15 Dec., 2010.

[2] Al-Zeer, S. Al-Ghanim, A. and Al-Wakeel. L.A. (2010). Visual attention in interaction with Arabic augmentative and alternative communication apps. In Proceedings of the International Conference on User Science and Engineering. 2-5 Sept. 2014.

[3] Bertou, E. and Shahid, S. (2014). Lowfidelity prototyping tablet applications for children. Proceedings of the Interaction design and children 2014.

[4] Bilal, D. and Gwizdka, J. (2016) Children's eye-fixations on google search results. In Proceedings of the 79th ASIS Annual Meeting: Creating Knowledge, Enhancing Lives through Information Technology. Copenhagen, Denmark. American Society for Information Science.

[5] Buxton, B. Sketching User Experiences - getting the design right and right design. (2007). Morgan Kaufmann, San Francisco.

[6] Chen, Y. and Tsai, M.Y. (2015) Eye-hand coordination strategies during active video game playing: An eye-tracking study. Computers in Human Behavior, 51.

[7] Currie, J. Bond, R., McCullagh, P. Black, P., Finlay D., and Peace, A. (2017). Eye tracking the visual attention of nurses interpreting simulated vital signs scenarios: mining metrics to discriminate between performance level. IEEE Transactions on Human-Machine Systems, 48, 2, 113-124.

[8] Derboven, J. Roeck, D.D., Verstraete, M., Geerts, D., Schneider-Barnes, J. and Luten, K. (2010). Comparing User Interaction with Low and High Fidelity Prototypes of Tabletop Surfaces. In Proceedings of the NordiCHI. Reykjavik, Iceland. ACM.

[9] Fischer, P.M., Richards, J.W., Berman E.J., and Krugman, D.M. (1989) Recall and eye tracking study of adolescents viewing tobacco advertisements. Jama, 261, 1, 84-89

[10] .Giannakos, M.N., Papavlasopoulou, S. and Sharma, K. (2020). Monitoring Children's Learning Through Wearable Eye-Tracking: The Case of a Making-Based Coding Activity. IEEE Pervasive Computing, 19, 1.

[11] Gomes, A., Oh, H., Chisik, Y. and Chen, M. (2012) Ilha Musical: a CAVE for nurturing cultural appreciation. In Proceedings of the Proceedings of the 11th International Conference on Interaction Design and Children. Bremen, Germany.

[12] Gossen, T., Höbel, J., and Nürnberger, A. (2014). Usability and perception of young users and adults on targeted web search engines. In Proceedings of the 5th Information Interaction in Context Symposium. Regensburg, Germany.

[13] Hanna, L., Neapolitan, D. and Risden, K. (2004) Evaluating computer game concepts with children. Proceedings of the 2004 conference on Interaction design and children: building a community.

[14] Hershman, A., Nazare, J., Qi, J., Saveski, M., Roy, D. and Resnick, M. (2018). Light it up: Using paper circuitry to enhance low-fidelity paper prototypes for children. Proceedings of the 17th ACM conference on interaction design and children.

[15] Hirschmanner, M., Lammer, L. and Vincze, M. (2015). Mattie: A simple educational platform for children to realize their first robot prototype. Proceedings of the 14th international conference on interaction design and children

[16] Hoggenmüller, M., Tomitsch, M., Hespanhol, L., Tran, T.T.M., Worrall, S. and Nebot, E. (2021). Context-Based Interface Prototyping: Understanding the Effect of Prototype Representation on User Feedback. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.

[17] Holmqvist, K. and Andersson, R. (2017)/ Eye tracking: A comprehensive guide to methods. paradigms and measures.

[18] Huestegge, L., Radach, R., Corbic, D. and Huestegge, S.M. (2009). Oculomotor and linguistic determinants of reading development: A longitudinal study. Vision research, 49, 24, 2948-2959

[19] Kleberg, J.L., Högström, J., Sundström, K., Frick, A. and Serlachius, E. (2020). Delayed gaze shifts away from others’ eyes in children and adolescents with social anxiety disorder. Journal of Affective Disorders.

[20] Kohler, B., Haladjian, J., Simeonova, B. and Ismailovic, D. (2012) Feedback in Low vs High Fidelity Visuals for Game Prototypes. In Proceedings of the Games and Software Engineering. Zurich, 2012. IEEE.

[21] A. N. Kornev, I. Balciuniene and S. R. Oganov. (2018). The strategic reading brain development: An eye-tracking study of the text reading in typically-developing and dyslexic children. International Journal of Psychophysiology, 131.

[22] Krejtz, K., Krejtz, I., Duchowski, A., Szarkowska, A. and Walczak, A. (2012) Multimodal learning with audio description: an eye tracking study of children's gaze during a visual recognition task. In Proceedings of the ACM Symposium on Applied Perception. Los Angeles, California.

[23] Levi, M.D. and Conrad, F.G. (1996). A heuristic evaluation of a world wide web prototype. Interactions, 3, 4, 50-61.

[24] Lim, Y-K., Pangam, A., Periyasami, S. and Aneja, S. (2006). Comparative Analysis of High- and Low-fidelity Prototypes for More Valid Usability Evaluations of Mobile Devices. In Proceedings of the NordiCHI. Oslo, Norway. ACM.

[25] Just, M.A. and Carpenter, P.A. (1980). A theory of reading: from eye fixations to comprehension. Psychol Rev, 87, 4, 329-354

[26] Martín-Arnal, L.A., León, J.A. van den Broek, P. and Olmos, R. (2019). Understanding comics. A comparison between children and adults through a coherence/incoherence paradigm in an eye-tracking study. Psicología Educativa. Revista de los Psicólogos de la Educación, 25, 2, 127-137.

[27] Moeller, K., Neuburger, S., Kaufmann, L., Landerl, K. and Nuerk, H. C. (2009). Basic number processing deficits in developmental dyscalculia: Evidence from eye tracking. Cognitive Development, 24, 4.

[28] Othman, A. and Mohsin, M. (2017). How could robots improve social skills in children with Autism? 6th International Conference on Information and Communication Technology and Accessibility (ICTA).

[29] Papavlasopoulou, S., Sharma, K. and Giannakos, M.N. (2020) Coding activities for children: Coupling eye-tracking with qualitative data to investigate gender differences. Computers in Human Behavior, 105.

[30] Kent, M.P. Pauzé, E., Roy, E-A., de Billy N. and Czoli, C. (2019). Children and adolescents' exposure to food and beverage marketing in social media apps. Pediatric obesity, 14, 6.

[31] Rayner, K. Pollatsek, A., Ashby, J and Clifton Jr, C. (2012). Psychology of reading. Psychology Press.

[32] Read, J.C. MacFarlane, S and Casey, C. (2002). Endurability, Engagement and Expectations: Measuring Children's Fun. In Proceedings of the Interaction Design and Children. Eindhoven.

[33] Read, J.C. (2012). Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun. In Proceedings of the 14th ACM International Conference on Multimodal Interaction. Santa Monica. ACM.

[34] Reilly, D., Dearman, D., Welsman-Dinelle, M. and Inkpen, K. (2005). Evalusating Early Prototypes in Context: Trade-offs, Challenges, and Successe. Pervasive Computing, 4, 4, 42-50.

[35] Rudd, J. Stern, K. and Isesensee, S. (1996).Low vs high-fidelity prototyping debate. Interactions, 3, 1, 75-85

[36] Sauer, J., Holger, F. and Ruttinger, B. (2008). Designing interactive consumer products: utility of low-fidelity prototypes and effectiveness of enhanced control labeling. Applied Ergonomics, 39, 1, 71-85.

[37] Sauer, J. and Sonderegger, A. (2009). The influence of prototype fidelity and aesthetics of design in usability tests: Effects on user behaviour, subjective evaluation and emotion. Applied Ergonomics, 40, 670-677.

[38] Sefelin, R., Tscheligi, M. and Giller, V. (2003) Paper Prototyping - What is it good for? A comparison of Paper and Computer based Low fidelity prototyping. In Proceedings of the Conference on Human Factors in Computing Systems. Fort Lauderdale.

[39] Sharma, K., Papavlasopoulou, S. and Giannakos, M.N. (2019). Coding games and robots to enhance computational thinking: How collaboration and engagement moderate children’s attitudes? International Journal of Child-Computer Interaction, 21.

[40] Sim, G., & Cassidy, B. (2013). Investigating the fidelity effect when evaluating game prototypes with children. Paper presented at the Proceedings of the 27th International BCS Human Computer Interaction Conference, London, UK.

[41] Sim, G., Cassidy, B., & Read, J. C. (2013). Understanding the Fidelity Effect when Evaluating Games with children. Paper presented at the Interaction Design and Children, New York.

[42] Sim, G., Horton, M., & McKnight, L. (2016). iPad vs Paper Prototypes: Does Form Factor Affect Children's Ratings of a Game Concept? Paper presented at the Proceedings of the The 15th International Conference on Interaction Design and Children.

[43] Sim, G., & Bond, R. (2021). Eye tracking in child computer interaction: Challenges and opportunities. International Journal of Child-Computer Interaction. 30.

[44] Snyder, C. (2003) Paper Prototyping: The fast and Easy Way to Define and Refine User Interfaces. Morgan Kaufmann, San Francisco.

[45] Su,Q., Chen, F., Li, H., Yan, N. and Wang, L. (2018). Multimodal Emotion Perception in Children with Autism Spectrum Disorder by Eye Tracking Study. Conference on Biomedical Engineering and Sciences (IECBES).

[46] Thomsen, S.R. and Fulton, K. (2007). Adolescents’ Attention to Responsibility Messages in Magazine Alcohol Advertisements: An Eye-Tracking Approach. Journal of Adolescent Health, 41, 1.

[47] Torney, H., Harvey, A., Finlay, D., Magee, J., Funston, R. and Bond, R. (2018) Eye-tracking analysis to compute the visual hierarchy of user interfaces on automated external defibrillators. British HCI Conference 2018.

[48] Mestres, E.T. and Pellicer-Sánchez, A. (2019) Young EFL learners' processing of multimodal input: Examining learners’ eye movements. System, 80.

[49] Velazquez, C.E. and Pasch, K.E. (2014). Attention to food and beverage advertisements as measured by eye-tracking technology and the food preferences and choices of youth. Journal of the Academy of Nutrition and Dietetics, 114, 4, 578-582.

[50] Virzi, R.A. Sokolov, J.L. and Karis, D. (1996) Usability problem identification using both low- and high-fidelity prototypes. In Proceedings of the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Vancouver, British Columbia, Canada.

[51] von Reumont, R. and Budke, A. (2020) Strategies for Successful Learning with Geographical Comics: An Eye-Tracking Study with Young Learners. Education Sciences, 10, 10, 293.

[52] Wiklund, M., Thurrot, C. and Dumas, J. (1992)Does the Fidelity of Software Prototypes Affect the Perception of Usability. In Proceedings of the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Atlanta, USA.

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org