2,898
views
0
recommends
+1 Recommend
1 collections
    0
    shares

      Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Towards Real-Life Adoption of Conversational Interfaces: Exploring the Challenges in Designing Chatbots That Live up to User Expectations

      Published
      proceedings-article
      , ,
      34th British HCI Conference (HCI2021)
      Post-pandemic HCI – Living Digitally
      20th - 21st July 2021
      Conversational User Interfaces, Educational Chatbots, Expectations–Experience Gap, Real-life, Case Study
      Bookmark

            Abstract

            Chatbots are increasingly popular, but state-of-the-art chatbots still struggle to meet user expectations, limiting their application in many domains. The factors affecting use have been studied extensively in laboratory contexts, resulting in context-independent requirements. However, user expectations and experiences of chat interfaces are affected by the context of use. Research efforts measuring experiences with chat interfaces need to shift from studies in controlled laboratory settings to studies in real-life settings in various domains. This paper explores this field of study by reporting on a small-scale real-life case study on the gap between expectations and experiences with an educational chatbot. More case studies in the wild, such as this one, could contribute to a deeper understanding of factors affecting acceptance and real use. We propose the use of the CIMO logic across these studies to build upon previous results.

            Content

            Author and article information

            Contributors
            Conference
            July 2021
            July 2021
            : 306-311
            Affiliations
            [0001]Drillster BV

            De Meern, The Netherlands
            [0002]Avans University of Applied Sciences

            Breda, The Netherlands
            [0003]HU University of Applied Sciences

            Utrecht, The Netherlands
            Article
            10.14236/ewic/HCI2021.33
            fa9dec92-eed9-4ac0-93e7-4500da6ae38b
            © Zandt et al. Published by BCS Learning & Development Ltd. Proceedings of the BCS 34th British HCI Conference 2021, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            34th British HCI Conference
            HCI2021
            34
            London, UK
            20th - 21st July 2021
            Electronic Workshops in Computing (eWiC)
            Post-pandemic HCI – Living Digitally
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2021.33
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Real-life,Conversational User Interfaces,Expectations–Experience Gap,Case Study,Educational Chatbots

            REFERENCES

            1. and (2012) Thematic Analysis. In: et al. (eds). APA handbooks in psychology. APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological. Washington: American Psychological Association.

            2. et al. (2019) Can Direct Address Affect Engagement with Chatbots Embodied in Physical Spaces? In: and (eds). CUI ’19: Proceedings of the 1st International Conference on Conversational User Interfaces, Dublin, August 2019. New York: ACM. 1–3.

            3. (1989) Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13 (3), 319–340.

            4. , and (2008) Developing Design Propositions through Research Synthesis. Organisational studies, 29(3). 393–413.

            5. Drillster BV (2019) The Science Behind Drillster. Available from https://drillster.com/wp-content/uploads/2019/08/White-Paper-_The-science-behind-Drillster_.pdf (24 October 2020).

            6. , and (2019) Inquisitive mind: a conversational news companion. In: and (eds). CUI ’19: Proceedings of the 1st International Conference on Conversational User Interfaces, Dublin, August 2019. New York: ACM. 1–3.

            7. (1885) Über das Gedächtnis. Untersuchungen zur experimentellen Psychologie. Leipzich: Decker & Humblot.

            8. et al. (2019) Progressivity for voice interface design. Proceedings of the 1st International Conference on Conversational User Interfaces. Dublin, August 2019. New York: ACM. 1–8.

            9. , and (2018a) What Makes Users Trust a Chatbot for Customer Service? An Exploratory Interview Study. In: et al. (eds). Internet Science. INSCI 2018. Lecture Notes in Computer Science, vol 11193. Cham: Springer.

            10. , and (2018b) Different Chatbots for Different Purposes: Towards a Typology of Chatbots to Understand Interaction Design. In: et al. (eds). Internet Science. INSCI 2018. Lecture Notes in Computer Science, vol 11551. Cham: Springer.

            11. (2019) Face-to-Face Conversation: Why Embodiment Matters for Conversational User Interfaces. In: and (eds). CUI ’19: Proceedings of the 1st International Conference on Conversational User Interfaces, Dublin, August 2019. New York: ACM. 1–3.

            12. (2019) From sex and therapy bots to virtual assistants and tutors: how emotional should artificially intelligent agents be? In: and (eds). CUI ’19: Proceedings of the 1st International Conference on Conversational User Interfaces, Dublin, August 2019. New York: ACM. 1–3.

            13. , and (2008) Construction and Evaluation of a User Experience Questionnaire. In: (eds). HCI and Usability for Education and Work. USAB 2008. Lecture Notes in Computer Science, vol 5298. Heidelberg: Springer. 63–76.

            14. (1972) So lernt man lernen. Der Weg zum Erfolg. Freiburg im Breisgau: Herder.

            15. and (2016) “Like Having a Really Bad PA”: The Gulf Between User Expectation and Experience of Conversational Agents. In: and (eds). CHI ’16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. San Jose, May 2016. New York: ACM. 5286–5297.

            16. (1967) A Memory Schedule. Modern Language Journal, 51 (2). 73–75.

            17. et al. (2020) Systematic Review: Trust-Building Factors and Implications for Conversational Agent Design. International Journal of Human-Computer Interaction, 37 (4). 1–16.

            18. , and (2001) Conversational Agents for Advanced Learning: Applications and Research. Proceedings of BotShow 2001 Conference. Fontainebleau, 2001. Fontainebleau: INSEAD. 1–7.

            19. (2018) UEQ_Plus_Data_Analysis_Tool.xlsx. Available from http://ueqplus.ueqresearch.org/Material/UEQ_Plus_Data_Analysis _Tool.xlsx (18 December 2020).

            20. and (2019) Handbook for the modular extension of the User Experience Questionnaire. Available from http://ueqplus.ueqresearch.org/Material/UEQ+_Handbook_V1.pdf (18 December 2020).

            21. et al. (2011) Action Design Research. MIS Quarterly, 35 (1). 37–56.

            22. et al. (2019) A Practical Take on Theory in HCI. White paper.

            23. and (2000) A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science, 46 (2). 186–204.

            24. et al. (2003) User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27 (3). 425–478.

            25. and (2020) (Non-)Interacting with conversational agents: perceptions and motivations of using chatbots and voice assistants. Proceedings of the Conference on Mensch und Computer, Magdeburg, September 2020. New York: ACM. 321–331.

            26. (1966) ELIZA–a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9 (1). 36–45.

            Comments

            Comment on this article