+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Cueing Retrospective Verbal Reports in Usability Testing Through Eye-Movement Replay

      ,   , ,

      Proceedings of HCI 2007 The 21st British HCI Group Annual Conference University of Lancaster, UK (HCI)

      British HCI Group Annual Conference

      3 - 7 September 2007

      Eye-tracking, verbal protocols, online usability testing

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          An experimental validation is presented of a novel method for usability testing that entails the playback of dynamic eyetracking data to cue the elicitation of retrospective verbal reports. Participants in our study produced: (1) think-aloud reports during an online search task, and (2) retrospective reports during another online search task, with reports being cued by the playback of either the screen capture of events or the participant’s own eye-movements. Task-completion times and response rates were recorded for all reporting methods. Fewer participants completed the search task whilst thinking aloud, indicating the reactivity of this technique. Verbal transcripts were coded for instances of usability problems. The eye-cued method identified more usability problems than the think-aloud or screen-cued methods. A significant interaction between search engine type and retrospective cue type suggests that the value of the eye-cue method for eliciting usability problems may be greatest with more complex search environments. Our results demonstrate that when cued appropriately, retrospective reports may be less reactive and more informative than other verbalisation techniques.

          Related collections

          Most cited references 14

          • Record: found
          • Abstract: not found
          • Article: not found

          Telling more than we can know: Verbal reports on mental processes.

            • Record: found
            • Abstract: not found
            • Article: not found

            Thoughts beyond words: When language overshadows insight.

              • Record: found
              • Abstract: found
              • Article: not found

              The validity of verbal protocols.

              The reactivity of a "think aloud" verbal protocol and the veridicality of different retrospective protocols were tested over four dissimilar tasks. Generating a concurrent protocol altered accuracy in two tasks, simple addition and a choice between two gambles, and generally prolonged response times. Such reactivity partially qualifies the dominant theory of protocol generation (Ericsson & Simon, 1984). Retrospective protocols yielded substantial forgetting or fabrication in all tasks, supporting the consensus on the nonveridicality of these methods. It is concluded that protocol validity should be based on an empirical check rather than on theory-based assurances.

                Author and article information

                September 2007
                September 2007
                : 1-9
                Sony Ericsson Software Applications


                Warrington, WA3 6AE, UK
                Department of Psychology

                Lancaster University

                Lancaster, LA1 4YF, UK

                Harwell Innovation Centre

                Harwell, OX11 0QG, UK
                © Nicola Eger et al. Published by BCS Learning and Development Ltd. Proceedings of HCI 2007 The 21st British HCI Group Annual Conference University of Lancaster, UK

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit

                Proceedings of HCI 2007 The 21st British HCI Group Annual Conference University of Lancaster, UK
                Lancaster, UK
                3 - 7 September 2007
                Electronic Workshops in Computing (eWiC)
                British HCI Group Annual Conference
                Product Information: 1477-9358BCS Learning & Development
                Self URI (journal page):
                Electronic Workshops in Computing


                Comment on this article