Effects of Timing on Users’ Agency during Mixed-Initiative Interaction

We explore the role of timing in situations where a human user and semi-autonomous software can each initiate actions, building on cognitive theories of rhythmic expectation and mutual temporal adaptation during conversation. Two controlled experiments demonstrate that adjustments to the rhythm of back-and- forth interaction have significant effects on perceived agency, task performance and stress. Conclusions include design guidance that establishing a predictable rhythm of interaction is likely to be beneficial for mixed initiative systems.

Content

Author and article information

Contributors

Guo Yu

Alan Blackwell

Stephanie Wilson

Luis Anunciação

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 1-12

Affiliations

[0001]University of Cambridge

Cambridge, UK

Article

DOI: 10.14236/ewic/HCI2017.35

SO-VID: 216325a8-c433-40d8-b711-2acdfb789c75

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 31st International BCS Human Computer Interaction Conference (HCI 2017)

Conference acronym: HCI

Conference number: 31

Conference location: Sunderland, UK

Conference date: 3 - 6 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: digital make-believe, with delegates considering our expansive

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2017.35

Self URI (journal page): https://ewic.bcs.org/

References

L.H. ArnalA.-L. Giraud 2012 Cortical oscillations and sensory predictions Trends in Cognitive Sciences 16 7 390 398
M. Bratman 1999 Faces of intention: Selected essays on intention and agency Cambridge, UK Cambridge University Press
S.M. BokerJ.L. RotondoM. XuK. King 2002 Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series Psychological methods 7 3 338 355
J.K. Burgoon 1978 A communication model of personal space violations: Explication and an initial test. Human Communication Research 4 2 129 142
J.K. Burgoon 1995 Cross-cultural and intercultural applications of expectancy violations theory Intercultural Communication Theory 19 194 214
J.K. BurgoonC.A. WhiteJ.O. Greene 1997 Researching nonverbal message production: A view from interaction adaptation theory Message Production: Advances in Communication Theory 279 312
S.M. CasnerE.L. HutchinsD. Norman 2016 The challenges of partially automated driving Communications of the ACM 59 5 70 77
C. Chafe 1993 Tactile audio feedback. In: Proceedings of the International Computer Music Conference Tokyo, Japan 10-15 September 1993 International Computer Music Association: 76 79
M. ClaytonR. SagerU. Will 2005 In time with the music: The concept of entrainment and its significance for ethnomusicology European meetings in ethnomusicology 11 3 142
D. CoyleJ. MooreP.O. KristenssonP. FletcherA. Blackwell 2012 I did that! Measuring users’ experience of agency in their own actions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Austin, TX, USA 5-10 May 2012 New York, NY, USA: ACM 2025 2034
A. Cypher 1991 Eager: Programming repetitive tasks by example. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New Orleans, LA, USA 27 April 2 May 1991 New York, NY, USA: ACM 33 39
J.J. DarraghI.H. WittenM.L. James 1990 The reactive keyboard: A predictive typing aid Computer 23 11 41 49
A. DesantisC. RousselF. Waszak 2011 On the influence of causal beliefs on the feeling of agency Consciousness and Cognition 20 4 1211 1220
P. FaratinC. SierraN.R. Jennings 1998 Negotiation decision functions for autonomous agents Robotics and Autonomous Systems 24 3 159 182
C. FarrerM. BouchereauM. JeannerodN. Franck 2008 Effect of distorted visual feedback on the sense of agency Behavioural Neurology 19 1-2 53 57
M.H. FisekJ. BergerR.Z. Norman 1991 Participation in heterogeneous and homogeneous groups: A theoretical integration American Journal of Sociology 97 1 114 142
M.H. FisekJ. BergerR.Z. Norman 1995 Evaluations and the formation of expectations American Journal of Sociology 101 3 721 746
T. FujiokaL. J. TrainorE.W. LargeB. Ross 2009 Beta and gamma rhythms in human auditory cortex during musical beat processing Annals of the New York Academy of Sciences 1169 1 89 92
S.P. Gill 2012 Rhythmic synchrony and mediated interaction: towards a framework of rhythm in embodied interaction AI society 27 1 111 127
P. HaggardS. ClarkJ. Kalogeras 2002 Voluntary action and conscious awareness Nature Neuroscience 5 4 382 385
P. HaggardM. Tsakiris 2009 The experience of agency feelings, judgments, and responsibility Current Directions in Psychological Science 18 4 242 246
C. HarrisonD. TanD. Morris 2010 Skinput: Appropriating the body as an input surface. In: Proceedings of the 2010 ACM SIGCHI Conference on Human Factors in Computing Systems Atlanta, GA, USA 10-15 April 2010 New York, NY, USA: ACM 453 462
S. HawkinsI. CrossR. Ogden 2013 Communicative interaction in spontaneous music and speech Music, Language and Interaction 285 329
S.G. HartL. E. Staveland 1988 Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research Advances in psychology 52 139 183
N. HonJ.H. PohC.S. Soon 2013 Preoccupied minds feel less control: Sense of agency is modulated by cognitive load Consciousness and cognition 22 2 556 561
E. Horvitz 1999 Principles of mixed-initiative user interfaces. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems Pitts- burgh, PA, USA 15-20 May New York, NY, USA: ACM 159 166
P. IceR. CurtisP. PhillipsJ. Wells 2007 Using asynchronous audio feedback to enhance teaching presence and students’ sense of community Journal of Asynchronous Learning Networks 11 2 3 25
T. IioM. ShiomiK. ShinozawaT. AkimotoK. ShimoharaN. Hagita 2011 Investigating entrainment of people’s pointing gestures by robot’s gestures using a WOz method International Journal of Social Robotics 3 4 405 414
M.R. JonesM. Boltz 1989 Dynamic attending and responses to time Psychological Review 96 3 459 491
V., KlingsporJ. DemirisM. Kaiser (1997) Human-robot communication and machine learning Applied Artificial Intelligence 11 7 719 746
S. KnightN. SpiroI. Cross 2017 Look, listen and learn: Exploring effects of passive entrainment on social judgements of observed others Psychology of Music 45 1 99 115
B. LibetC.A. GleasonE.W. WrightD.K. Pearl 1983 Time of conscious intention to act in relation to onset of cerebral activity (readiness potential): The unconscious initiation of a freely voluntary act Brain 106 623 642
H. LimerickJ.W. MooreD. Coyle 2015 Empirical evidence for a diminished sense of agency in speech interfaces. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems Seoul, Republic of Korea 18-23 April 2015 New York, NY, USA: ACM 3967 3970
H. McCann 1998 The works of agency: On human action, will, and freedom Ithaca, NY, USA Cornell University Press
J.W. MooreP. Haggard 2008 Awareness of action: Inference and prediction Consciousness and Cognition 17 1 136 144
J.W. MooreS.S. Obhi 2012 Intentional binding and the sense of agency: a review Consciousness and cognition 21 1 546 561
C. NashA. Blackwell 2008 Realtime representation and gestural control of musical polytempi. In: Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08) Genova, Italy: 5-7 June 2008 28 33
L. NowickiW. PrinzM. GrosjeanB.H. ReppP.E. Keller 2013 Mutual adaptive timing in interpersonal action coordination Psychomusicology: Music, Mind, and Brain 23 1 6 20
A.D. Patel 2007 Music, language, and the brain Oxford, UK Oxford University Press
G. RohenkohlA.M. CravoV. WyartA.C. Nobre 2012 Temporal expectation improves the quality of sensory information The Journal of Neuroscience 32 24 8424 8428
E.A. Schegloff 1992 Repairafternextturn:Thelast structurally provided defense of intersubjectivity in conversation American Journal of Sociology 97 5 1295 1345
B. Shneiderman 1992 Designing the user interface: strategies for effective human-computer inter- action 3 MA Addison-Wesley Reading
C. StetsonX. CuiP.R. MontagueD.M. Eagleman 2006 Motor-sensory recalibration leads to an illusory reversal of action and sensation Neuron 51 5
C. SummerfieldT. Egner 2009 Expectation(and attention) in visual cognition Trends in Cognitive Sciences 13 9 403 409
V. WyartA.C. NobreC. Summerfield 2012 Dissociable prior influences of signal probability and relevance on visual contrast sensitivity Proceedings of the National Academy of Sciences 109 9 3593 3598

Comments

Comment on this article

[1] L.H. ArnalA.-L. Giraud 2012 Cortical oscillations and sensory predictions Trends in Cognitive Sciences 16 7 390 398

[2] M. Bratman 1999 Faces of intention: Selected essays on intention and agency Cambridge, UK Cambridge University Press

[3] S.M. BokerJ.L. RotondoM. XuK. King 2002 Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series Psychological methods 7 3 338 355

[4] J.K. Burgoon 1978 A communication model of personal space violations: Explication and an initial test. Human Communication Research 4 2 129 142

[5] J.K. Burgoon 1995 Cross-cultural and intercultural applications of expectancy violations theory Intercultural Communication Theory 19 194 214

[6] J.K. BurgoonC.A. WhiteJ.O. Greene 1997 Researching nonverbal message production: A view from interaction adaptation theory Message Production: Advances in Communication Theory 279 312

[7] S.M. CasnerE.L. HutchinsD. Norman 2016 The challenges of partially automated driving Communications of the ACM 59 5 70 77

[8] C. Chafe 1993 Tactile audio feedback. In: Proceedings of the International Computer Music Conference Tokyo, Japan 10-15 September 1993 International Computer Music Association: 76 79

[9] M. ClaytonR. SagerU. Will 2005 In time with the music: The concept of entrainment and its significance for ethnomusicology European meetings in ethnomusicology 11 3 142

[10] D. CoyleJ. MooreP.O. KristenssonP. FletcherA. Blackwell 2012 I did that! Measuring users’ experience of agency in their own actions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Austin, TX, USA 5-10 May 2012 New York, NY, USA: ACM 2025 2034

[11] A. Cypher 1991 Eager: Programming repetitive tasks by example. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems New Orleans, LA, USA 27 April 2 May 1991 New York, NY, USA: ACM 33 39

[12] J.J. DarraghI.H. WittenM.L. James 1990 The reactive keyboard: A predictive typing aid Computer 23 11 41 49

[13] A. DesantisC. RousselF. Waszak 2011 On the influence of causal beliefs on the feeling of agency Consciousness and Cognition 20 4 1211 1220

[14] P. FaratinC. SierraN.R. Jennings 1998 Negotiation decision functions for autonomous agents Robotics and Autonomous Systems 24 3 159 182

[15] C. FarrerM. BouchereauM. JeannerodN. Franck 2008 Effect of distorted visual feedback on the sense of agency Behavioural Neurology 19 1-2 53 57

[16] M.H. FisekJ. BergerR.Z. Norman 1991 Participation in heterogeneous and homogeneous groups: A theoretical integration American Journal of Sociology 97 1 114 142

[17] M.H. FisekJ. BergerR.Z. Norman 1995 Evaluations and the formation of expectations American Journal of Sociology 101 3 721 746

[18] T. FujiokaL. J. TrainorE.W. LargeB. Ross 2009 Beta and gamma rhythms in human auditory cortex during musical beat processing Annals of the New York Academy of Sciences 1169 1 89 92

[19] S.P. Gill 2012 Rhythmic synchrony and mediated interaction: towards a framework of rhythm in embodied interaction AI society 27 1 111 127

[20] P. HaggardS. ClarkJ. Kalogeras 2002 Voluntary action and conscious awareness Nature Neuroscience 5 4 382 385

[21] P. HaggardM. Tsakiris 2009 The experience of agency feelings, judgments, and responsibility Current Directions in Psychological Science 18 4 242 246

[22] C. HarrisonD. TanD. Morris 2010 Skinput: Appropriating the body as an input surface. In: Proceedings of the 2010 ACM SIGCHI Conference on Human Factors in Computing Systems Atlanta, GA, USA 10-15 April 2010 New York, NY, USA: ACM 453 462

[23] S. HawkinsI. CrossR. Ogden 2013 Communicative interaction in spontaneous music and speech Music, Language and Interaction 285 329

[24] S.G. HartL. E. Staveland 1988 Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research Advances in psychology 52 139 183

[25] N. HonJ.H. PohC.S. Soon 2013 Preoccupied minds feel less control: Sense of agency is modulated by cognitive load Consciousness and cognition 22 2 556 561

[26] E. Horvitz 1999 Principles of mixed-initiative user interfaces. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems Pitts- burgh, PA, USA 15-20 May New York, NY, USA: ACM 159 166

[27] P. IceR. CurtisP. PhillipsJ. Wells 2007 Using asynchronous audio feedback to enhance teaching presence and students’ sense of community Journal of Asynchronous Learning Networks 11 2 3 25

[28] T. IioM. ShiomiK. ShinozawaT. AkimotoK. ShimoharaN. Hagita 2011 Investigating entrainment of people’s pointing gestures by robot’s gestures using a WOz method International Journal of Social Robotics 3 4 405 414

[29] M.R. JonesM. Boltz 1989 Dynamic attending and responses to time Psychological Review 96 3 459 491

[30] V., KlingsporJ. DemirisM. Kaiser (1997) Human-robot communication and machine learning Applied Artificial Intelligence 11 7 719 746

[31] S. KnightN. SpiroI. Cross 2017 Look, listen and learn: Exploring effects of passive entrainment on social judgements of observed others Psychology of Music 45 1 99 115

[32] B. LibetC.A. GleasonE.W. WrightD.K. Pearl 1983 Time of conscious intention to act in relation to onset of cerebral activity (readiness potential): The unconscious initiation of a freely voluntary act Brain 106 623 642

[33] H. LimerickJ.W. MooreD. Coyle 2015 Empirical evidence for a diminished sense of agency in speech interfaces. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems Seoul, Republic of Korea 18-23 April 2015 New York, NY, USA: ACM 3967 3970

[34] H. McCann 1998 The works of agency: On human action, will, and freedom Ithaca, NY, USA Cornell University Press

[35] J.W. MooreP. Haggard 2008 Awareness of action: Inference and prediction Consciousness and Cognition 17 1 136 144

[36] J.W. MooreS.S. Obhi 2012 Intentional binding and the sense of agency: a review Consciousness and cognition 21 1 546 561

[37] C. NashA. Blackwell 2008 Realtime representation and gestural control of musical polytempi. In: Proceedings of the 2008 Conference on New Interfaces for Musical Expression (NIME08) Genova, Italy: 5-7 June 2008 28 33

[38] L. NowickiW. PrinzM. GrosjeanB.H. ReppP.E. Keller 2013 Mutual adaptive timing in interpersonal action coordination Psychomusicology: Music, Mind, and Brain 23 1 6 20

[39] A.D. Patel 2007 Music, language, and the brain Oxford, UK Oxford University Press

[40] G. RohenkohlA.M. CravoV. WyartA.C. Nobre 2012 Temporal expectation improves the quality of sensory information The Journal of Neuroscience 32 24 8424 8428

[41] E.A. Schegloff 1992 Repairafternextturn:Thelast structurally provided defense of intersubjectivity in conversation American Journal of Sociology 97 5 1295 1345

[42] B. Shneiderman 1992 Designing the user interface: strategies for effective human-computer inter- action 3 MA Addison-Wesley Reading

[43] C. StetsonX. CuiP.R. MontagueD.M. Eagleman 2006 Motor-sensory recalibration leads to an illusory reversal of action and sensation Neuron 51 5

[44] C. SummerfieldT. Egner 2009 Expectation(and attention) in visual cognition Trends in Cognitive Sciences 13 9 403 409

[45] V. WyartA.C. NobreC. Summerfield 2012 Dissociable prior influences of signal probability and relevance on visual contrast sensitivity Proceedings of the National Academy of Sciences 109 9 3593 3598

Studying business & IT? Drive your professional career forwards with BCS books - for a 20% discount click here: shop.bcs.org