Real-time Perception of Non-verbal Human Feedback in a Gaming Scenario

Social interactive robots require sophisticated perception and cognition abilities to behave and interact in a natural human-like way. The precise perception of behaviour of an interaction partner plays a crucial role in social robotics. Human feedback is one of the most important behavioural cues of human during conversation. Feedback is a signal that human sends to the interlocutor, consciously or unconsciously, that can convey four communicative functions. These functions: continuation, perception, understanding, and acceptance of the message, activate or inhibits different behaviours of human. Past systems focused mainly on the scripted scenarios irrespective of interaction partners feedback behaviour. This paper proposes a system for perception and interpretation of the interaction partner feedback from non-verbal cues. The proposed work has been tested in a 20 question gaming scenario in real time and it has shown promising results.

Content

Author and article information

Contributors

Salah Al-Darraji: http://rrlab.cs.uni-kl.de

Zafar Zuhair: http://rrlab.cs.uni-kl.de

Karsten Berns: http://rrlab.cs.uni-kl.de

Conference

Publication date: July 2016

Publication date (Print): July 2016

Pages: 1-10

Affiliations

[0001]Robotics Research Lab.

Department of Computer Science

University of Kaiserslautern

Gottlieb-Daimler-Str. 48 67663 Kaiserslautern, Germany

Article

DOI: 10.14236/ewic/HCI2016.27

SO-VID: ed4cda1a-1e56-4af1-9526-c3286f02b536

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 30th International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 30

Conference location: Bournemouth University, Poole, UK

Conference date: 11 - 15 July 2016

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Fusion

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2016.27

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

N. AkalinP. UluerH. Kose 2014 Non-verbal communication with a social robot peer: Towards robot assisted interactive sign language tutoring Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on 1122 1127
S. Al-DarrajiK. BernsA. Rodić 2016 Action Units Based Facial Expression Recognition Using Deep Learning Advances in Robot Design and Intelligent Control: Proceedings of the 25th International Conference on Robotics in Alpe-Adria-Danube Region (RAAD) Belgrade, Serbia
J. Allwood 2001 Cooperation and flexibility in multimodal communication H. BuntR.-J. Beun Cooperative Multimodal Communication 2155 Lecture Notes in Computer Science 113 124 Springer Berlin Heidelberg
L. Cerrato 2007 Investigating Communicative Feedback Phenomena across Languages and Modalities PhD thesis, KTH Computer Science and Communication, Department of Speech, Music and Hearing Stockholm, Sweden
C. ChaoJ. LeeM. BegumA. L. Thomaz 2011 Simon plays simon says: The timing of turn-taking in an imitation game RO-MAN, 2011 IEEE 235 240
J. DeLamaterD. MyersJ. Collett Social Psychology Westview Press
J. HanN. CampbellK. JokinenG. Wilcock 2012 Investigating the use of non-verbal cues in human-robot interaction with a nao robot Cognitive Infocommunications (CogInfoCom), 2012 IEEE 3rd International Conference on 679 683
A. JevticG. DoisyY. ParmetY Edan 2015 Comparison of interaction modalities for mobile indoor robot guidance: Direct physical interaction, person following, and pointing control IEEE Transactions on Human-Machine Systems 45 6 653 663
P. Noller 2006 The SAGE Handbook of Nonverbal Communication, chapter Nonverbal Communication in Close Relationships 403 421 SAGE Publications
M. ProetzschT. LukschK. Berns 2010a Development of complex robotic systems using the behaviour-based control architecture iB2C Robotics and Autonomous Systems 58 1 46 67 doi: 10.1016/j.robot.2009.07.027
M. Proetzsch F. ZimmermannR. EschbachJ. KloosK. Berns 2010b A systematic testing approach for autonomous mobile robots using domain-specific languages R. DillmannJ. BeyererU. HanebeckT. Schultz KI2010: Advances in Artificial Intelligence volume 6359 of Lecture Notes in Computer Science 317 324 Springer Berlin/Heidelberg
S. SalehK. Berns 2015 Nonverbal communication with a humanoid robot via head gestures PETRA ’15: Proceedings of the 8th International Conference on PErvasive Technologies Related to Assistive Environments Corfu, Greece ACM ISBN 978-1-4503-3452-5/15/07
S. SalehA. KicktonJ. HirthK. Berns 2013 Robust perception of an interaction partner using depth information Proceeding of the International Conference on Advances in Computer-Human Interactions (ACHI) Nice, France
H. S. YoonS. Y Chi 2006 Visual processing of rock, scissors, paper game for human robot interaction In SICE-ICASE 2006 International Joint Conference 326 329
Z. ZafarK. Berns 2016 Recognizing Hand Gestures for Human-Robot Interaction Proceedings of the 9th International Conference on Advances in Computer-Human Interactions (ACHI) Venice, Italy

Comments

Comment on this article

[1] N. AkalinP. UluerH. Kose 2014 Non-verbal communication with a social robot peer: Towards robot assisted interactive sign language tutoring Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on 1122 1127

[2] S. Al-DarrajiK. BernsA. Rodić 2016 Action Units Based Facial Expression Recognition Using Deep Learning Advances in Robot Design and Intelligent Control: Proceedings of the 25th International Conference on Robotics in Alpe-Adria-Danube Region (RAAD) Belgrade, Serbia

[3] J. Allwood 2001 Cooperation and flexibility in multimodal communication H. BuntR.-J. Beun Cooperative Multimodal Communication 2155 Lecture Notes in Computer Science 113 124 Springer Berlin Heidelberg

[4] L. Cerrato 2007 Investigating Communicative Feedback Phenomena across Languages and Modalities PhD thesis, KTH Computer Science and Communication, Department of Speech, Music and Hearing Stockholm, Sweden

[5] C. ChaoJ. LeeM. BegumA. L. Thomaz 2011 Simon plays simon says: The timing of turn-taking in an imitation game RO-MAN, 2011 IEEE 235 240

[6] J. DeLamaterD. MyersJ. Collett Social Psychology Westview Press

[7] J. HanN. CampbellK. JokinenG. Wilcock 2012 Investigating the use of non-verbal cues in human-robot interaction with a nao robot Cognitive Infocommunications (CogInfoCom), 2012 IEEE 3rd International Conference on 679 683

[8] A. JevticG. DoisyY. ParmetY Edan 2015 Comparison of interaction modalities for mobile indoor robot guidance: Direct physical interaction, person following, and pointing control IEEE Transactions on Human-Machine Systems 45 6 653 663

[9] P. Noller 2006 The SAGE Handbook of Nonverbal Communication, chapter Nonverbal Communication in Close Relationships 403 421 SAGE Publications

[10] M. ProetzschT. LukschK. Berns 2010a Development of complex robotic systems using the behaviour-based control architecture iB2C Robotics and Autonomous Systems 58 1 46 67 doi: 10.1016/j.robot.2009.07.027

[11] M. Proetzsch F. ZimmermannR. EschbachJ. KloosK. Berns 2010b A systematic testing approach for autonomous mobile robots using domain-specific languages R. DillmannJ. BeyererU. HanebeckT. Schultz KI2010: Advances in Artificial Intelligence volume 6359 of Lecture Notes in Computer Science 317 324 Springer Berlin/Heidelberg

[12] S. SalehK. Berns 2015 Nonverbal communication with a humanoid robot via head gestures PETRA ’15: Proceedings of the 8th International Conference on PErvasive Technologies Related to Assistive Environments Corfu, Greece ACM ISBN 978-1-4503-3452-5/15/07

[13] S. SalehA. KicktonJ. HirthK. Berns 2013 Robust perception of an interaction partner using depth information Proceeding of the International Conference on Advances in Computer-Human Interactions (ACHI) Nice, France

[14] H. S. YoonS. Y Chi 2006 Visual processing of rock, scissors, paper game for human robot interaction In SICE-ICASE 2006 International Joint Conference 326 329

[15] Z. ZafarK. Berns 2016 Recognizing Hand Gestures for Human-Robot Interaction Proceedings of the 9th International Conference on Advances in Computer-Human Interactions (ACHI) Venice, Italy

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65