Blog
About
Search
Advanced search

913
views
49
references
 Top references
0
cited by
 Most cited
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
1 collections
    4
    shares
      43
      similar
       All similar
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Measuring User Experience in Conversational Interfaces: A Comparison of Six Questionnaires

      Author(s): 1 , 1 , 1

      Publication date (Print):

      Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference (HCI)

      Conference theme: Human Computer Interaction Conference

      Conference date: 4 - 6 July 2018

      Keywords: User experience, conversational agents, dialogue systems, voice interfaces, evaluation, standardised questionnaires

      Read this article at

      ScienceOpen
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          User experience (UX) has become an important aspect in the evaluation of interactive systems. In parallel, conversational interfaces have been increasingly used in many work and everyday settings. Although there have been various methods developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This study reviews the six main questionnaires for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure various UX dimensions. We found that (i) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (ii) two questionnaires assessed affect, and one assessed frustration dimension; and, (iii) enchantment, playfulness and motivation dimensions have not been covered sufficiently by any questionnaires. We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.

          Related collections

          Most cited references 49

          • Record: found
          • Abstract: found
          • Article: not found

          Measuring emotion: the Self-Assessment Manikin and the Semantic Differential.

           Judy M Bradley,  Nicholas Lang (1994)
          The Self-Assessment Manikin (SAM) is a non-verbal pictorial assessment technique that directly measures the pleasure, arousal, and dominance associated with a person's affective reaction to a wide variety of stimuli. In this experiment, we compare reports of affective experience obtained using SAM, which requires only three simple judgments, to the Semantic Differential scale devised by Mehrabian and Russell (An approach to environmental psychology, 1974) which requires 18 different ratings. Subjective reports were measured to a series of pictures that varied in both affective valence and intensity. Correlations across the two rating methods were high both for reports of experienced pleasure and felt arousal. Differences obtained in the dominance dimension of the two instruments suggest that SAM may better track the personal response to an affective stimulus. SAM is an inexpensive, easy method for quickly assessing reports of affective response in many contexts.
          Article 0 comments Cited 397 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            What is user engagement? A conceptual framework for defining user engagement with technology

             Heather L. O'Brien,  Elaine G. Toms (2008)
            Article 0 comments Cited 115 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              The dimensionality and correlates of flow in human-computer interactions

               Jane Webster,  Linda Treviño,  Lisa Ryan (1993)
              Article 0 comments Cited 94 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Contributors
                A. Baki Kocabalil
                Liliana Laranjo
                Enrico Coiera
                Conference
                Publication date: July 2018
                Publication date (Print): July 2018
                Pages: 1-12
                Affiliations
                [1 ] Australian Institute of Health

                Innovation, Macquarie University
                Article
                DOI: 10.14236/ewic/HCI2018.21
                Copyright statement: © Kocaballi et al. Published by BCS Learning and Development Ltd. Proceedings of British HCI 2018. Belfast, UK.
                License:

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

                Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference
                Conference acronym: HCI
                Conference number: 32
                Conference location: Belfast, UK
                Conference date: 4 - 6 July 2018
                Conference sponsor: Electronic Workshops in Computing (eWiC)
                Conference theme: Human Computer Interaction Conference
                Product
                Product Information: 1477-9358BCS Learning & Development
                Self URI (journal page): https://ewic.bcs.org/
                Categories
                Series title: Electronic Workshops in Computing

                Data availability:

                ScienceOpen disciplines: Applied computer science, Computer science, Security & Cryptology, Graphics & Multimedia design, General computer science, Human-computer-interaction

                Keywords: standardised questionnaires, evaluation, voice interfaces, dialogue systems, conversational agents, User experience

                Comments

                Comment on this article