Can users recall their user experience with a technology? Temporal bias and the system usability scale.

Boyd, Kyle; Bond, Raymond; Magee, Justin; McCormack, Paul

doi:10.14236/ewic/HCI2018.25

Abstract

The System Usability Scale (SUS) score survey is a widely respected tool for measuring usability. While there are other surveys available such as the User Experience Questionnaire (UEQ) or the Single Ease Question (SEQ), the SUS is amongst the most popular and widely used instrument. SUS provides an easy-to-understand score with benchmarking. Generally, a SUS score is administered directly after a usability test to assess the user experience and the usability of a product, including websites and smartphone apps and more. However, some researchers have used it as a survey as part of a ‘in the wild’ trial which is often completed after the trial or indeed sometime after the subjects interacted with the technology. With this in mind the aim of this research was to see if a participant’s user experience would change if a SUS score was administered at different times after a test to understand if recalling the usability of technology led to temporal bias for the SUS.

Content

Author and article information

Contributors

Kyle Boyd

Raymond Bond

Justin Magee

Paul McCormack

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 1-6

Affiliations

[1 ] Ulster University

Belfast, BT15 1ED

[2 ] Ulster University

Jordanstown, BT37 0QB

Article

DOI: 10.14236/ewic/HCI2018.25

SO-VID: 048db3d3-9c72-4c28-bd42-fc7725bbb019

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 32

Conference location: Belfast, UK

Conference date: 4 - 6 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Human Computer Interaction Conference

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.25

Self URI (journal page): https://ewic.bcs.org/

Categories

Series title: Electronic Workshops in Computing

ScienceOpen disciplines: Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction

Keywords: Usability,System Usability Scale,User Experience,Usability Testing,Human-Computer Interaction,User Interfaces

Smashing Magazine 2018 A Comprehensive Guide To User Experience Design [online]. Available at: http://www.smashingmagazine.com/2018/02/comprehensive-guide-user-experience-design/ (Accessed: 9 April 2018)
Nielsen Norman Group 2012 Usability 101: Introduction to Usability [online]. Available at: http://www.nngroup.com/articles/usability-101-introduction-to-usability/ (Accessed: 9 April 2018)
K. BoydR. BondS. GallagherG. MooreE. O’Kane 2017 Usability and Behaviour Analysis of Prisoners using an Interactive Technology to Manage Daily Living Kaufmann
R.R. BondD.D. FinlayC.D. NugentG. MooreD. Guldenring 2014 A usability evaluation of medical software at an expert conferencesetting. Computer methods and programs in biomedicine 113 1 383 395
T. TullisB. AlbertW. Albert 2013 Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics USA Morgan Kaufmann
J. SauroJ.R. Lewis 2016 Quantifying the user experience: Practical statistics for user research Morgan Kaufmann
J. Brooke 2013 “SUS?: A Retrospective,” J. Usability Stud. 8 2 29 40 DOI?
S. Krug 2009 Don’t Make Me Think! A Common Sense Approach to Web Usability, vol. Second Edi. Berkley: Newriders
A. BangorP. KortumJ. Miller 2008 An empirical evaluation of the System Usability Scale International Journal of Human-Computer Interaction 24 6 574 594 doi:10.1080/10447310802205776
A. BangorP. KortumJ. Miller 2009 Determining what individual SUS Scores mean: Adding an adjective rating scale Journal of Usability Studies 4 3 114 123 DOI?
T.S. TullisJ.N. Stetson 2004 June A comparison of questionnaires for assessing website usability In Usability professional association conference 1 12
Y. H. KimD. J. KimK. Wachter 2013 “A study of mobile user engagement (MoEN): Engagement motivations, perceived value, satisfaction, and continued engagement intention,” Decis. Support Syst. 56 1 361 370
S. McLellanA. MuddimerS.C. Peres 2012 The effect of experience on System Usability Scale ratings Journal of usability studies 7 2 56 67 DOI?
A. Følstad 2017 Users’ design feedback in usability evaluation: a literature review Human-centric Computing and Information Sciences 7 1 19 DOI ?
B. Hepple 2010 The new single equality act in Britain. The Equal Rights Review 5 11 24
J. Nielsen 2003 Usability 101: Introduction to usability
Philip KortumMary Sorber 2015 Measuring the Usability of Mobile Applications for Phones and Tablets,International Journal of Human–Computer Interaction 31 8 518 529 10.1080/10447318.2015.1064658
K. OrfanouN. TseliosC. Katsanos 2015 Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale The International Review of Research in Open and Distributed Learning 16 2 DOI?
James R. Lewis 2014 Usability: Lessons Learned … and Yet to Be Learned, International Journal of Human–Computer Interaction 30 9 663 684 10.1080/10447318.2014.930311

Comments

[1] Smashing Magazine 2018 A Comprehensive Guide To User Experience Design [online]. Available at: http://www.smashingmagazine.com/2018/02/comprehensive-guide-user-experience-design/ (Accessed: 9 April 2018)

[2] Nielsen Norman Group 2012 Usability 101: Introduction to Usability [online]. Available at: http://www.nngroup.com/articles/usability-101-introduction-to-usability/ (Accessed: 9 April 2018)

[3] K. BoydR. BondS. GallagherG. MooreE. O’Kane 2017 Usability and Behaviour Analysis of Prisoners using an Interactive Technology to Manage Daily Living Kaufmann

[4] R.R. BondD.D. FinlayC.D. NugentG. MooreD. Guldenring 2014 A usability evaluation of medical software at an expert conferencesetting. Computer methods and programs in biomedicine 113 1 383 395

[5] T. TullisB. AlbertW. Albert 2013 Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics USA Morgan Kaufmann

[6] J. SauroJ.R. Lewis 2016 Quantifying the user experience: Practical statistics for user research Morgan Kaufmann

[7] J. Brooke 2013 “SUS?: A Retrospective,” J. Usability Stud. 8 2 29 40 DOI?

[8] S. Krug 2009 Don’t Make Me Think! A Common Sense Approach to Web Usability, vol. Second Edi. Berkley: Newriders

[9] A. BangorP. KortumJ. Miller 2008 An empirical evaluation of the System Usability Scale International Journal of Human-Computer Interaction 24 6 574 594 doi:10.1080/10447310802205776

[10] A. BangorP. KortumJ. Miller 2009 Determining what individual SUS Scores mean: Adding an adjective rating scale Journal of Usability Studies 4 3 114 123 DOI?

[11] T.S. TullisJ.N. Stetson 2004 June A comparison of questionnaires for assessing website usability In Usability professional association conference 1 12

[12] Y. H. KimD. J. KimK. Wachter 2013 “A study of mobile user engagement (MoEN): Engagement motivations, perceived value, satisfaction, and continued engagement intention,” Decis. Support Syst. 56 1 361 370

[13] S. McLellanA. MuddimerS.C. Peres 2012 The effect of experience on System Usability Scale ratings Journal of usability studies 7 2 56 67 DOI?

[14] A. Følstad 2017 Users’ design feedback in usability evaluation: a literature review Human-centric Computing and Information Sciences 7 1 19 DOI ?

[15] B. Hepple 2010 The new single equality act in Britain. The Equal Rights Review 5 11 24

[16] J. Nielsen 2003 Usability 101: Introduction to usability

[17] Philip KortumMary Sorber 2015 Measuring the Usability of Mobile Applications for Phones and Tablets,International Journal of Human–Computer Interaction 31 8 518 529 10.1080/10447318.2015.1064658

[18] K. OrfanouN. TseliosC. Katsanos 2015 Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale The International Review of Research in Open and Distributed Learning 16 2 DOI?

[19] James R. Lewis 2014 Usability: Lessons Learned … and Yet to Be Learned, International Journal of Human–Computer Interaction 30 9 663 684 10.1080/10447318.2014.930311

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65