A new metric scale for measuring trust towards holographic intelligent agents

Trust is an essential attitude in social relationships, but it also mediates our approach to certain technology. The definition of interpersonal trust, however, is too wide to expound our understanding of how trust impedes such interaction with technology, and the lack of an applicable quantifiable model in particular presents an obstacle to our quest of building reliable, trusted, and intelligent holographic agents. In this paper, we therefore develop a novel metric scale to measure trust. We identify, select, and refine over a hundred items related to trust, check their precision and validity with the help of a judges panel, and select polarising items that are able to bring out the distinctive characteristics regarding people’s trust towards intelligent agents. Our findings indicate that an assessment of trust involves looking at the user’s belief about the agent’s competence, integrity, benevolence, and compassion, which drive the attitude of trust, influenced by the user’s general propensity to trust. Trust then drives intention to engage and ultimately engagement, which, if successful, results in the establishment of a trust relationship with the agent. While we propose an item-response scale for measuring this model of trust, we also add our deliberations on how elements of it could be replaced with alternative means that possibly offer more immediacy than self-inspection, discussing in particular alternatives for measuring elements of compassion, competence, and social relationships.

Content

Author and article information

Contributors

Xinyu Huang

Fridolin Wild

Conference

Publication date (Print): July 2021

Pages: 1-8

Affiliations

[0001]Institute of Educational Technology, The Open University, UK

Article

DOI: 10.14236/ewic/HCI2021-W2.2

SO-VID: bfbeb384-066b-490a-808e-ea1790462ea3

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: 34th British HCI Workshop and Doctoral Consortium

Conference acronym: HCI2021-WDC

Conference number: 34

Conference location: London, UK

Conference date: 20th - 21st July 2021

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Post-pandemic HCI – Living Digitally

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2021-W2.2

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

T Holz 2011 MiRA—Mixed Reality Agents International Journal of Human-Computer Studies 69 4 251 268
A.G CampbellJ.W StaffordT HolzG O’Hare 2013 Why, when and how to use augmented reality agents (AuRAs) Virtual Reality 18 2 139 159
X HuangF WildD Whitelock 2021 Design Dimensions for Holographic Intelligent Agents: A Comparative Analysis AIED’2021 Netherlands
M.M PinxterenM PluymaekersJ Lemmink 2020 Human-like communication in conversational agents: a literature review and research agenda Journal of Service Management 31 203 225
R. C MayerJ. H DavisF. D Schoorman 1995 An integrative model of organizational trust Academy of Management Review 20 709 734
C NassJ SteuerE. R Tauber 1994 Computers are social actors CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New York, NY ACM
J.P Dobel 1999 Public Integrity John Hopkins University Press Baltimore
P PhilipL DupuyM AuriacombeF SerreÉ.D SevinA SauteraudJ Micoulaud-Franchi 2020 Trust and acceptance of a virtual psychiatric interview between embodied conversational agents and outpatients NPJ Digital Medicine 3
R Borum 2010 The Science of Interpersonal Trust Mental Health Law & Policy Faculty Publications 574
S SousaD LamasP Dias 2014 A Model for Human-Computer Trust: Contributions Towards Leveraging User Engagement Learning and Collaboration Technologies, Greece June 22 128 137 Springer Cham
G JonesJ George 1998 The experience and evolution of trust: Implications for cooperation and teamwork Academy of Management Review 23 531 546
P KulmsS Kopp 2018 A Social Cognition Perspective on Human–Computer Trust: The Effect of Perceived Warmth and Competence on Trust in Decision-Making With Computers Frontiers Digit. Humanit 5 14
J. J LeeW. B KnoxJ. B WormwoodC BreazealD DeSteno 2013 Computationally modeling interpersonal trust Front. Psychol 4 893
J LeeK.A See 2004 Trust in Automation: Designing for Appropriate Reliance Human factors and Ergonomics society 46 1 50 80
K GuptaR HajikaY.S PaiA DuenserM LochnerM Billinghurst 2019 In AI We Trust: Investigating the Relationship between Biosignals, Trust and Cognitive Load in VR 25th ACM Symposium on Virtual Reality Software and Technology New York November 12 1 10
J Pimentel 2010 A note on the usage of Likert Scaling for research data analysis USM R&D Journal 18 109 112
N HannaD Richards 2019 Speech Act Theory as an Evaluation Tool for Human–Agent Communication Algorithms 12 79
P.J Lang 1980 Behavioural treatment and bio-behavioural assessment: computer applications, Technology in mental health care delivery systems 119 137
M.M BradleyP.J Lang 1994 Measuring emotion: The self-assessment Manikin and the semantic differential Journal of Behaviour Therapy and Experimental Psychiatry 25 49 59
W.M.K Trochim 2021 Likert Scaling, Conjoint.ly online at https://conjointly.com/kb/likert-scaling/ 06/2021
A Bryman 2012 Social Research Methods OUP Oxford
A. B KocaballiL LaranjoE Coiera 2019 Understanding and measuringuser experience in conversational interfaces Interacting with Computers 31 192 207
J. S RobertsJ. E LaughlinD. H Wedell 1999 Validity issues in the Likert and Thurstone approaches to attitude measurement Educational and Psychological Measurement 59 211 233
R Core Team 2021 R: A language and environment for statistical computing R Foundation for Statistical Computing Vienna, Austria URL https://www.R-project.org/
R Watts 2020 An Interactive, Holographic 3D Model using IBM Watson
J UrbanoA.P RochaE Oliveira 2013 The impact of benevolence in computational trust Agreement Technologies Beijing August 1 210 224 Springer, Heidelberg
A Ray 2018 Compassionate Artificial Intelligence: Frameworks and Algorithms Compassionate AI Lab
J. A ColquittB. A ScottJ. A LePine 2007 Trust, trustworthiness, and trust propensity: a meta-analytic test of their unique relationships with risk taking and job performance J. Appl. Psychol 92 909 927
K MarinaccioS KohnR ParasuramanE. J De Visser 2015 A Framework for Rebuilding Trust in Social Automation Across Health-Care Domains Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care 4 201 205
T KimH Song 2021 How should intelligent agents apologize to restore trust?: The interaction effect between anthropomorphism and apology attribution on trust repair Telematics and Informatics 61 101595
C McLeod 2020 Trust Zalta The Stanford Encyclopedia of Philosophy (substantive revision Mon Aug 10, 2020) Stanford University ISSN 1095-5054
F Wild 2016 Learning Theory and Algorithmic Quality Characteristics Learning Analytics in R with SNA, LSA, and MPIA Springer 23 44
P HagerA Gonczi 2009 What is competence? Medical teacher 18 15 18
S LeggM Hutter 2006 A formal measure of machine intelligence In Proc. 15th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn’06) Ghent 73 80
S.T FiskeA.J CuddyP Glick 2007 Universal dimensions of social cognition: warmth and competence Trends in Cognitive Sciences 11 2 77 83
T SingerO Klimecki 2014 Empathy and compassion Current Biology 24 875 878
P Gilbert 2014 The origins and nature of compassion focused therapy Br. J. Clin. Psychol 53 6 41
T RebedeaM DascaluS Trausan-MatuD BanicaA GartnerC ChiruD Mihaila 2010 Overview and Preliminary Results of Using PolyCAFe for Collaboration Analysis and Feedback Generation Wolpers, Kirschner, Scheffel, Lindstaedt, Dimitrova (Eds.): Sustaining TEL: From Innovation to Learning and Practice. EC-TEL 2010. Lecture Notes in Computer Science, vol 6383. Springer, Berlin, Heidelberg https://doi.org/10.1007/978-3-642-16020-2_31

Comments

Comment on this article

[1] T Holz 2011 MiRA—Mixed Reality Agents International Journal of Human-Computer Studies 69 4 251 268

[2] A.G CampbellJ.W StaffordT HolzG O’Hare 2013 Why, when and how to use augmented reality agents (AuRAs) Virtual Reality 18 2 139 159

[3] X HuangF WildD Whitelock 2021 Design Dimensions for Holographic Intelligent Agents: A Comparative Analysis AIED’2021 Netherlands

[4] M.M PinxterenM PluymaekersJ Lemmink 2020 Human-like communication in conversational agents: a literature review and research agenda Journal of Service Management 31 203 225

[5] R. C MayerJ. H DavisF. D Schoorman 1995 An integrative model of organizational trust Academy of Management Review 20 709 734

[6] C NassJ SteuerE. R Tauber 1994 Computers are social actors CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New York, NY ACM

[7] J.P Dobel 1999 Public Integrity John Hopkins University Press Baltimore

[8] P PhilipL DupuyM AuriacombeF SerreÉ.D SevinA SauteraudJ Micoulaud-Franchi 2020 Trust and acceptance of a virtual psychiatric interview between embodied conversational agents and outpatients NPJ Digital Medicine 3

[9] R Borum 2010 The Science of Interpersonal Trust Mental Health Law & Policy Faculty Publications 574

[10] S SousaD LamasP Dias 2014 A Model for Human-Computer Trust: Contributions Towards Leveraging User Engagement Learning and Collaboration Technologies, Greece June 22 128 137 Springer Cham

[11] G JonesJ George 1998 The experience and evolution of trust: Implications for cooperation and teamwork Academy of Management Review 23 531 546

[12] P KulmsS Kopp 2018 A Social Cognition Perspective on Human–Computer Trust: The Effect of Perceived Warmth and Competence on Trust in Decision-Making With Computers Frontiers Digit. Humanit 5 14

[13] J. J LeeW. B KnoxJ. B WormwoodC BreazealD DeSteno 2013 Computationally modeling interpersonal trust Front. Psychol 4 893

[14] J LeeK.A See 2004 Trust in Automation: Designing for Appropriate Reliance Human factors and Ergonomics society 46 1 50 80

[15] K GuptaR HajikaY.S PaiA DuenserM LochnerM Billinghurst 2019 In AI We Trust: Investigating the Relationship between Biosignals, Trust and Cognitive Load in VR 25th ACM Symposium on Virtual Reality Software and Technology New York November 12 1 10

[16] J Pimentel 2010 A note on the usage of Likert Scaling for research data analysis USM R&D Journal 18 109 112

[17] N HannaD Richards 2019 Speech Act Theory as an Evaluation Tool for Human–Agent Communication Algorithms 12 79

[18] P.J Lang 1980 Behavioural treatment and bio-behavioural assessment: computer applications, Technology in mental health care delivery systems 119 137

[19] M.M BradleyP.J Lang 1994 Measuring emotion: The self-assessment Manikin and the semantic differential Journal of Behaviour Therapy and Experimental Psychiatry 25 49 59

[20] W.M.K Trochim 2021 Likert Scaling, Conjoint.ly online at https://conjointly.com/kb/likert-scaling/ 06/2021

[21] A Bryman 2012 Social Research Methods OUP Oxford

[22] A. B KocaballiL LaranjoE Coiera 2019 Understanding and measuringuser experience in conversational interfaces Interacting with Computers 31 192 207

[23] J. S RobertsJ. E LaughlinD. H Wedell 1999 Validity issues in the Likert and Thurstone approaches to attitude measurement Educational and Psychological Measurement 59 211 233

[24] R Core Team 2021 R: A language and environment for statistical computing R Foundation for Statistical Computing Vienna, Austria URL https://www.R-project.org/

[25] R Watts 2020 An Interactive, Holographic 3D Model using IBM Watson

[26] J UrbanoA.P RochaE Oliveira 2013 The impact of benevolence in computational trust Agreement Technologies Beijing August 1 210 224 Springer, Heidelberg

[27] A Ray 2018 Compassionate Artificial Intelligence: Frameworks and Algorithms Compassionate AI Lab

[28] J. A ColquittB. A ScottJ. A LePine 2007 Trust, trustworthiness, and trust propensity: a meta-analytic test of their unique relationships with risk taking and job performance J. Appl. Psychol 92 909 927

[29] K MarinaccioS KohnR ParasuramanE. J De Visser 2015 A Framework for Rebuilding Trust in Social Automation Across Health-Care Domains Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care 4 201 205

[30] T KimH Song 2021 How should intelligent agents apologize to restore trust?: The interaction effect between anthropomorphism and apology attribution on trust repair Telematics and Informatics 61 101595

[31] C McLeod 2020 Trust Zalta The Stanford Encyclopedia of Philosophy (substantive revision Mon Aug 10, 2020) Stanford University ISSN 1095-5054

[32] F Wild 2016 Learning Theory and Algorithmic Quality Characteristics Learning Analytics in R with SNA, LSA, and MPIA Springer 23 44

[33] P HagerA Gonczi 2009 What is competence? Medical teacher 18 15 18

[34] S LeggM Hutter 2006 A formal measure of machine intelligence In Proc. 15th Annual Machine Learning Conference of Belgium and The Netherlands (Benelearn’06) Ghent 73 80

[35] S.T FiskeA.J CuddyP Glick 2007 Universal dimensions of social cognition: warmth and competence Trends in Cognitive Sciences 11 2 77 83

[36] T SingerO Klimecki 2014 Empathy and compassion Current Biology 24 875 878

[37] P Gilbert 2014 The origins and nature of compassion focused therapy Br. J. Clin. Psychol 53 6 41

[38] T RebedeaM DascaluS Trausan-MatuD BanicaA GartnerC ChiruD Mihaila 2010 Overview and Preliminary Results of Using PolyCAFe for Collaboration Analysis and Feedback Generation Wolpers, Kirschner, Scheffel, Lindstaedt, Dimitrova (Eds.): Sustaining TEL: From Innovation to Learning and Practice. EC-TEL 2010. Lecture Notes in Computer Science, vol 6383. Springer, Berlin, Heidelberg https://doi.org/10.1007/978-3-642-16020-2_31

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org