The ABCD of Usability Testing

We introduce a methodology for tracking and auditing feedback, errors and suggestions for software packages. This short paper describes how we innovate on the evaluation mechanism, introducing an (Antecedent, Barrier, Consequence and Development) ABCD form, embedded within an eParticipation platform to enable end users to easily report on any usability issues. This methodology will be utilised to improve the STEP cloud e-Participation platform (part of the current STEP Horizon2020 project http://step4youth.eu . The platform is currently being piloted in real life contexts, with the participation of public authorities that are integrating the eParticipation platform into their regular decision-making practices. The project is involving young people, through engagement and motivation strategies and giving them a voice in Environmental decision making at the local level. The pilot evaluation aims to demonstrate how open engagement needs to be embedded within public sector processes and the usability methodology reported here will help to identify the key barriers for wide scale deployment of the platform.

Content

Author and article information

Contributors

Paula Forbes

Stefano de Paoli

Ken Scott-Brown

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 1-4

Affiliations

[0001]Abertay University

Bell St, Dundee

Article

DOI: 10.14236/ewic/HCI2017.3

SO-VID: ea92625a-de15-46e3-9805-a51c53344599

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 31st International BCS Human Computer Interaction Conference (HCI 2017)

Conference acronym: HCI

Conference number: 31

Conference location: Sunderland, UK

Conference date: 3 - 6 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: digital make-believe, with delegates considering our expansive

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2017.3

Self URI (journal page): https://ewic.bcs.org/

References

W. AlbertT. Tullis 2013 Measuring the user experience: collecting, analyzing, and presenting usability metrics 2nd Morgan Kaufmann Publishers Inc San Francisco, CA, USA 2013
W. AlbertT. TullisD. Tedesco 2009 Beyond the usability lab: Conducting large-scale online user experience studies Morgan Kaufmann
J. DorfmanV. A. ShamesJ. F. Kihlstrom 1996 Intuition, incubation, and insight: Implicit cognition in problem solving Geoffrey D. M. Underwood 1996 Implicit cognition 257 296 New York, NY, US Oxford University Press 305
M. R. EblingB. E. John 2000 On the contributions of different empirical data in usability testing Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '00) Daniel BoyarskiWendy A. Kellogg ACM New York, NY, USA 289 296
R.H. FrankeJ. D. Kaul 1978 The Hawthorne experiments: First statistical interpretation Am Sociol Rev 43 623 643 doi: 10.2307/2094540
F. GabbertL. HopeR. P. Fisher 2009 Protecting eyewitness evidence: Examining the efficacy of a self-administered interview tool Law and human behavior, 33 4 298 307
S. R. HawkB. L. Dos Santos 1991 Successful system development: The effect of situational factors on alternate user roles IEEE Transactions on Engineering Management 38 4 316 327
D. M. Kamps 2002 Preventing problems in improving behavior B. AlgozzineP. Kay Preventing problem behavior 11 36 Thousands Oaks, CA Corwin Press
S. Kujala 2003 User involvement: a review of the benefits and challenges Behaviour & information technology, 22 1 1 16
C. MacaulayD. SloanX. JiangP. ForbesS. LoyntonJ. R. SwedlowP. Gregor 2009 Usability and user-centered design in scientific software development IEEE Software 26 1 96
J. D. McKeenT. GuimaraesJ. C. Wetherbe 1994 The relationship between user participation and user satisfaction: an investigation of four contingency factors Mis Quarterly 427 451
A. L. NicholsJ. K. Maner 2008 The good subject effect: Investigating participant demand characteristics Journal of General Psychology 135 151 165
J. Nielsen 1994 Usability inspection methods Conference Companion on Human Factors in Computing Systems (CHI '94) Catherine Plaisant ACM New York, NY, USA 413 414
J. NielsenR. Molich 1990 Heuristic evaluation of user interfaces Proc ACM CHI’90 Seattle, WA 1-5 April 249 256
J. Nielsen 1992 Finding usability problems through heuristic evaluation Proceedings of the Conference on Human Factors in Computing System (CHI 92) Monterey, CA May 3-7 1992 373 380
J. NielsenV Phillips 1993 Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared Proceedings of the Conference on Human Factors in Computing System (CHI 93) Amsterdam April 24-29 1993 214 221
P. PolsomC. LewisJ. RiemanC. Wharton 1992 Cognitive walkthroughs: A method for theory-based evaluation of user interfaces International Journal of Man- Machine Studies 36 5 741 773
C. PrattM. Dubie 2008 Observing behavior using a-b-c data The Reporter 14 1 1 4
D. SloanC. MacaulayP. ForbesS. Loynton 2009 September User research in a scientific software development project Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology 423 429 British Computer Society
S. ToogoodK. Timlin 1996 The Functional Assessment of Challenging Behaviour: a Comparison of Informant-based, Experimental and Descriptive Methods Journal of Applied Research in Intellectual Disabilities 9 3 206 222

Comments

Comment on this article

[1] W. AlbertT. Tullis 2013 Measuring the user experience: collecting, analyzing, and presenting usability metrics 2nd Morgan Kaufmann Publishers Inc San Francisco, CA, USA 2013

[2] W. AlbertT. TullisD. Tedesco 2009 Beyond the usability lab: Conducting large-scale online user experience studies Morgan Kaufmann

[3] J. DorfmanV. A. ShamesJ. F. Kihlstrom 1996 Intuition, incubation, and insight: Implicit cognition in problem solving Geoffrey D. M. Underwood 1996 Implicit cognition 257 296 New York, NY, US Oxford University Press 305

[4] M. R. EblingB. E. John 2000 On the contributions of different empirical data in usability testing Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '00) Daniel BoyarskiWendy A. Kellogg ACM New York, NY, USA 289 296

[5] R.H. FrankeJ. D. Kaul 1978 The Hawthorne experiments: First statistical interpretation Am Sociol Rev 43 623 643 doi: 10.2307/2094540

[6] F. GabbertL. HopeR. P. Fisher 2009 Protecting eyewitness evidence: Examining the efficacy of a self-administered interview tool Law and human behavior, 33 4 298 307

[7] S. R. HawkB. L. Dos Santos 1991 Successful system development: The effect of situational factors on alternate user roles IEEE Transactions on Engineering Management 38 4 316 327

[8] D. M. Kamps 2002 Preventing problems in improving behavior B. AlgozzineP. Kay Preventing problem behavior 11 36 Thousands Oaks, CA Corwin Press

[9] S. Kujala 2003 User involvement: a review of the benefits and challenges Behaviour & information technology, 22 1 1 16

[10] C. MacaulayD. SloanX. JiangP. ForbesS. LoyntonJ. R. SwedlowP. Gregor 2009 Usability and user-centered design in scientific software development IEEE Software 26 1 96

[11] J. D. McKeenT. GuimaraesJ. C. Wetherbe 1994 The relationship between user participation and user satisfaction: an investigation of four contingency factors Mis Quarterly 427 451

[12] A. L. NicholsJ. K. Maner 2008 The good subject effect: Investigating participant demand characteristics Journal of General Psychology 135 151 165

[13] J. Nielsen 1994 Usability inspection methods Conference Companion on Human Factors in Computing Systems (CHI '94) Catherine Plaisant ACM New York, NY, USA 413 414

[14] J. NielsenR. Molich 1990 Heuristic evaluation of user interfaces Proc ACM CHI’90 Seattle, WA 1-5 April 249 256

[15] J. Nielsen 1992 Finding usability problems through heuristic evaluation Proceedings of the Conference on Human Factors in Computing System (CHI 92) Monterey, CA May 3-7 1992 373 380

[16] J. NielsenV Phillips 1993 Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared Proceedings of the Conference on Human Factors in Computing System (CHI 93) Amsterdam April 24-29 1993 214 221

[17] P. PolsomC. LewisJ. RiemanC. Wharton 1992 Cognitive walkthroughs: A method for theory-based evaluation of user interfaces International Journal of Man- Machine Studies 36 5 741 773

[18] C. PrattM. Dubie 2008 Observing behavior using a-b-c data The Reporter 14 1 1 4

[19] D. SloanC. MacaulayP. ForbesS. Loynton 2009 September User research in a scientific software development project Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology 423 429 British Computer Society

[20] S. ToogoodK. Timlin 1996 The Functional Assessment of Challenging Behaviour: a Comparison of Informant-based, Experimental and Descriptive Methods Journal of Applied Research in Intellectual Disabilities 9 3 206 222

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65