Designing Chatbots for Crises: A Case Study Contrasting Potential and Reality

Chatbots are becoming ubiquitous technologies, and their popularity and adoption are rapidly spreading. The potential of chatbots in engaging people with digital services is fully recognised. However, the reputation of this technology with regards to usefulness and real impact remains rather questionable. Studies that evaluate how people perceive and utilise chatbots are generally lacking. During the last Kenyan elections, we deployed a chatbot on Facebook Messenger to help people submit reports of violence and misconduct experienced in the polling stations. Even though the chatbot was visited by more than 3,000 times, there was a clear mismatch between the users’ perception of the technology and its design. In this paper, we analyse the user interactions and content generated through this application, and discuss the challenges and directions for designing more effective chatbots.

Content

Author and article information

Contributors

Lara S. G. Piccolo

Shadrock Roberts

Anna Iosif

Harith Alani

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 1-10

Affiliations

[1 ] The Open University

Milton Keynes, UK

[2 ] Ushahidi

Nairobi, Kenya

Article

DOI: 10.14236/ewic/HCI2018.56

SO-VID: 83f8ae78-f0be-4c0d-ac55-c9ff98dd7eb3

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 32

Conference location: Belfast, UK

Conference date: 4 - 6 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Human Computer Interaction Conference

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.56

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

N. BerenteC. Salge 2017 Is that social bot behaving unethically? Communications of the ACM 60 9 29 31
A. ChamberlainA. CrabtreeT. RoddenM. JonesY. Rogers 2012 Research in the wild: Understanding ‟in the wild‟ approaches to design and development In Proceedings of the Designing Interactive Systems Conference DIS ‟12, New York, NY, USA 795 796 ACM
K. Collins 2017 Facebook Messenger bots help monitor Kenya election violence http://www.cnet.com/news/facebook-messenger-bots-help-monitor-kenya-election-violence/
R. Dale 2016 The return of the chatbots Natural Language Engineering 22 5 811817
Facebook (2018, Jan) Messenger platform design best practices. http://developers.facebook.com/docs/messenger-platform/introduction/ general-best-practices
A. FolstadP. B. Brandtzaeg 2017 June Chatbots and the new world of HCI. interactions 24 4 38 42
R. HarperU.K. Cambridge 2008 Being Human: Human-Computer Interaction in the Year 2020 Microsoft Research 2008
HeidelbergK. RashidA. Das 2018 Build Better Chatbots: A Complete Guide to Getting Started with Chatbots Apress
P. Hyman 2014 January ‟Peace technologies‟ enable eyewitness reporting when disasters strike Communications of the ACM 57 (1), 27 29
M. LouwerseA. GraesserS. LuH. Mitchell 2005 Social cues in animated conversational agents Applied Cognitive Psychology 19 6 693 704
K. MorrisseyJ. Kirakowski M. Kurosu 2013 „Realness‟ in chatbots: Establishing quantifiable criteria Human-Computer Interaction. Interaction Modalities and Techniques 87 96 Springer Berlin
N. M. RadziwillM. C. Benton 2017 Evaluating quality of chatbots and intelligent conversational agents CoRR abs/1704.04579
S. RobertsT. Doyle 2017 Understanding Crowdsourcing and Volunteer Engagement 121 134. John Wiley and Sons, Inc
Y. RogersH. SharpJ. Preece 2011 Interaction Design: Beyond Human -Computer Interaction (3rd ed.). Wiley Publishing
UN (2008, 02) United nations high commissioner for human rights -report from ohchr fact-finding mission to kenya 6 28 february 2008
C. WhartonJ. RiemanC. LewisP. Polson 1994 Usability inspection methods Chapter The Cognitive Walkthrough Method: A Practitioner‟s Guide 105 140 New York, NY, USA John Wiley & Sons, Inc
J. Zamora 2017 I‟m sorry, dave, i‟m afraid i can‟t do that: Chatbot perception and expectations In Proceedings of the 5th International Conference on Human Agent Interaction, HAI ‟17, New York, NY, USA 253 260. ACM
A. Zhou 2017 Mar 6 ways bots are positively changing the world www.forbes.com/ sites/adelynzhou/2017/02/16/see-six-ways-bots-are-positively-changing\-the-world/

Comments

Comment on this article

[1] N. BerenteC. Salge 2017 Is that social bot behaving unethically? Communications of the ACM 60 9 29 31

[2] A. ChamberlainA. CrabtreeT. RoddenM. JonesY. Rogers 2012 Research in the wild: Understanding ‟in the wild‟ approaches to design and development In Proceedings of the Designing Interactive Systems Conference DIS ‟12, New York, NY, USA 795 796 ACM

[3] K. Collins 2017 Facebook Messenger bots help monitor Kenya election violence http://www.cnet.com/news/facebook-messenger-bots-help-monitor-kenya-election-violence/

[4] R. Dale 2016 The return of the chatbots Natural Language Engineering 22 5 811817

[5] Facebook (2018, Jan) Messenger platform design best practices. http://developers.facebook.com/docs/messenger-platform/introduction/ general-best-practices

[6] A. FolstadP. B. Brandtzaeg 2017 June Chatbots and the new world of HCI. interactions 24 4 38 42

[7] R. HarperU.K. Cambridge 2008 Being Human: Human-Computer Interaction in the Year 2020 Microsoft Research 2008

[8] HeidelbergK. RashidA. Das 2018 Build Better Chatbots: A Complete Guide to Getting Started with Chatbots Apress

[9] P. Hyman 2014 January ‟Peace technologies‟ enable eyewitness reporting when disasters strike Communications of the ACM 57 (1), 27 29

[10] M. LouwerseA. GraesserS. LuH. Mitchell 2005 Social cues in animated conversational agents Applied Cognitive Psychology 19 6 693 704

[11] K. MorrisseyJ. Kirakowski M. Kurosu 2013 „Realness‟ in chatbots: Establishing quantifiable criteria Human-Computer Interaction. Interaction Modalities and Techniques 87 96 Springer Berlin

[12] N. M. RadziwillM. C. Benton 2017 Evaluating quality of chatbots and intelligent conversational agents CoRR abs/1704.04579

[13] S. RobertsT. Doyle 2017 Understanding Crowdsourcing and Volunteer Engagement 121 134. John Wiley and Sons, Inc

[14] Y. RogersH. SharpJ. Preece 2011 Interaction Design: Beyond Human -Computer Interaction (3rd ed.). Wiley Publishing

[15] UN (2008, 02) United nations high commissioner for human rights -report from ohchr fact-finding mission to kenya 6 28 february 2008

[16] C. WhartonJ. RiemanC. LewisP. Polson 1994 Usability inspection methods Chapter The Cognitive Walkthrough Method: A Practitioner‟s Guide 105 140 New York, NY, USA John Wiley & Sons, Inc

[17] J. Zamora 2017 I‟m sorry, dave, i‟m afraid i can‟t do that: Chatbot perception and expectations In Proceedings of the 5th International Conference on Human Agent Interaction, HAI ‟17, New York, NY, USA 253 260. ACM

[18] A. Zhou 2017 Mar 6 ways bots are positively changing the world www.forbes.com/ sites/adelynzhou/2017/02/16/see-six-ways-bots-are-positively-changing\-the-world/

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65