Augmented Reality Enhanced Human Robot Interaction for Social Robots as e-Learning Companions

The recent advancements in technology are bringing both Augmented Reality (AR) and social robotics into classrooms to enhance interpersonal communications alongside students’ motivation and engagement in learning. This paper presents a systematic overview of the applications of AR technology in educational settings whilst exploring the role of social robots and discussing the possible utilisations and integrations of the two technologies to facilitate student learning. In this work in progress paper, the authors have considered factors such as the novelty, learning types, advantages and challenges of utilising AR alongside social robots in the classrooms. The authors have identified current gaps of applying AR technology effectively in e-Learning context, which need to be addressed in future research when integrating with social robots into the learning environment to enhance Human Robot Interaction (HRI) based e-Learning companions. The authors also have proposed and developed two different integrated solution frameworks, which aim to bridge the paradigms of supporting robots as social actors. These AR technologies based frameworks promote enhanced interactions among students, social robots and the learning materials in an integrated e-Learning environment.


INTRODUCTION
This work in progress paper discusses the role of social robots as learning companions for facilitating students learning process by enhancing their learning motivations and engagements.In particular, the effect of social robot learning companionship in optimising student cognitive capacity with the integration and contribution of Augmented Reality (AR) technology is envisioned and proposed.AR can be defined as a technology which overlays virtual objects/augmented components into a real-life environment.These augmented components simulate existence within the real world often meshing themselves to real life objects (Azuma et al., 2001).
The human and robot interactive learning process enhanced by AR technology within a learning companionship context aim to reduce the intrinsic and extraneous loads and increase the germane load to optimise the overall cognitive capacity (Sweller, 1988).The correlation between this optimised cognitive capacity and the improvements in learning motivations and engagement is the main focus of the ongoing research work.
The main contributions of the ongoing work are in these key aspects: 1) Identify the major challenges of integrating AR and social robot technologies to effectively inspire and motivate students learning and engagement in e-Learning context 2) Analyse the gaps of existing technical solutions and the new requirements to tackle the challenges effectively 3) Propose integrated solution frameworks to tackle the challenges 4) Develop software and hardware prototypes to demonstrate the feasibility of the frameworks 5) Recommend future theoretical research and practical development directions for this novel and promising interdisciplinary exploration

2
Motivations are decided by a learner's needs to gain new knowledge and skills.The intrinsic motivation to achieve autonomy and competence could be elicited in the context of having learning comparisons to learn together (Hoskins and Newstead, 2009).The measurement of this motivation could be done with some wellestablished framework and scales (Vallerand et al., 1992), (Biggs, 1999), and (Biggs et al, 2001).
The extent to which of a student engagement during a learning process is influenced by the degree of his/her enjoyment rather than simple involvement or participation in the learning knowledge (Trowler, 2010).This enjoyment is a kind of intrinsic motivation to learn.The balance of the challenges presented during the learning process and grown skills to deal with these challenges decides the enjoyment of the The balance is the basis of a flow/optimal experience (Csikszentmihalyi, 1992) enjoyed when fully engaging in the learning process.Usually students' engagements occur at three dimensions: behavioural engagement, emotional engagement, and cognitive engagement (Morton, 2009).The engagement level is decided by interest, convenience and expectation.In the case of learning together with a companion, the emotional engagement could be reflected in interest, enjoyment and healthy competition and as part of social interactions.The level of student engagement on each dimension could be measured based on active acknowledgement, longer attention span and active participations (Morton, 2009).From the individual student learning perspective, student engagement could be characterised by the student's attention in learning, interest in learning, involvement in learning, active participation in learning, and student-centeredness (Trowler, 2010).

THE ROLE OF A SOCIAL ROBOT
Robotics technology has been explored and utilised in broad educational contexts and applications throughout the long history of the general adoptions of Information and Communication Technology (ICT) and Artificial Intelligence (AI) in these domains (Mubin et al., 2013).The subjects of teaching and learning ranges from adult foreign language study, social skills training to educational entertainments for children such edutainment robotics (Vallerand et al., 1992).The role of a robot in these educational contexts and applications could be categorised by as either teaching material, an innovative teacher/tutor, or an intelligent learning companion.
In recent years, as the rapid development of intelligent humanoid robot technologies, various experiments and applications have been explored to use these robots as educational companions with a focus of 'learning together' with students, rather than as the passive teaching materials or as innovative teachers (Lupetti, 2016;Kanda et al., 2004;Movellan et al., 2005;Tanaka et al., 2017;You et al., 2006;Han et al., 2008;Saerbeck et al., 2010;Lee et al., 2011;Castellano et al., 2011).
The essence of a robot as a learning companion is 'learning together' with the students, offering intelligent assistance to facilitate the learning process, and inspiring students' motivations and engagements for effective learning outcomes (Tanaka et al., 2015;Noyes, 2017).This companionship inspires motivation and engagement for the student.For example, in the case that the large effect size of the positive mood induction using a positive body language with humanoid robots demonstrates the potential power to induce positive emotions (Whale, 2017), which broaden attention, thinking, and action and to build physical, intellectual, and social resources.

HRI AND SHARED PERCEPTION ENHANCED BY AUGMENTED REALITY
Augmented Reality is a proven technology that is able to improve students' learning activeness and motivations when learning abstract concepts, such as the control structures in a computer programming (Fredrickson, 1998), or even complex environmental issues in socioscientific reasoning scenarios (Mesia et al., 2016).
The power of Augmented Reality technology in facilitating the intuitive perception and interaction between students and the learning materials can be transformed into an interactive learning companionship between a social robot and students when they need to perceive a common learning context in which the learning togetherness is achieved.The effectiveness of a shared perception and efficient interactions, in turn, affect the cognitive capacity of students during the learning process.
According to the cognitive capacity theory (Sweller, 1988), every learning activity uses cognitive capacity in three ways: 1) Intrinsic load (difficulty level, depends on student) 2) Extraneous load (irrelevant obstacles to deciphering) 3) Germane load (encoding/retrieval for long-term learning) In the context of an Augmented Reality enhanced interactive learning companionship between a social robot and students, our work so far has been focusing on increasing the probability of learning in term of enhancing motivations and engagements with this enhancement.
The two hypotheses of the investigation are 1) The existence of benefits for using Augmented Reality and enhanced social robots to improve undergraduates learning motivation and engagement.
2) Augmented Reality enhanced social robots cause a high-quality learning motivation and engagement.
Regarding the optimisation of the overall cognitive capacity during the learning process, as an interaction between social robot and a student, there are two aspects to focus on: (i) Reduce extraneous load by: • Avoiding presenting too much info to same part of working memory of a student • Removing the need to switch attention from one place to another • Reducing redundancy in information presentation (ii) Increase germane load by: • Introducing prior knowledge, active learning, and comparisons • Encouraging integration across complementary modalities in student and robot interactions Although AR provides students many advantages in educational settings, research has suggested that the implementation of AR itself poses a challenge which may be difficult for students to utilize in an effective manner.Chang et al (2014) found that usability is the most important influencing factor which affects educational effectiveness, thus a well thought out design is necessary (Munzo-Cristobal et al., 2015).Another related crucial issue that must be considered with leveraging AR into the learning environment is cognitive overload (Dunleavy et al., 2009).Cheng and Tsai (2013) suggested that students may experience cognitive overload in an AR learning environment due to the amount of material and complexity of tasks.To avoid cognitive overload, it must be ensured that the tasks are well defined, non-complex and materials be made readily available.Finally, the success of the design of a particular e-Learning application with AR and social robots depends on incorporating subject domain knowledge into the system design process.

INTEGRATED SOLUTION FRAMEWORKS
As social robots are now creating interpersonal relationships in everyday scenarios, including moving towards higher education (Heo and Kim, 2003;Li et al., 2016).The benefit of displacing technologies are convenient to both students and teachers, as well as being beneficial on a cost-time basis (Allen and Seaman, 2010) so long as the robot's instructions are perceived credible.
The success of robot use in the classroom will depend on the degree to which the students find the instructions credible and are able to learn from them as student perceptions of credibility can have a tremendous impact on the effectiveness of the instructor.(Frymier and Thompson, 1999) Whilst learning is a critical outcome closely associated with credibility, (Schrodt et al., 2009) when bridging the paradigm supporting that robots are social actors.AR technologies promote greater interaction between students (Kamarainen et al. 2013) and this technology bridges the paradigm between students and the learning material used.
AR was first introduced as a learning tool within the American armed forces as a training simulator for aviation gunners and proved successful which would suggest with further development the technology could be ported over to other training situations (Caudell and Mizell, 1992).With recent technological advancements AR has become a cheap tool which no longer requires extensive head mounted displays (HMD) and thus it has become a popular tool within education ranging from key stage levels (Chiang et al., 2014;Kerwalla et., al 2006) to that of university (Ferrer-Torregrosa et al., 2015).
The research progress within this paper has presented a possibility in which AR could be provisioned within an educational environment alongside social robots.With this the author proposes that a system be built upon further which would allow the inbuilt hardware components of the social robot 'Pepper' by Softbank Robotics to interact with students as a learning companion.
With the Pepper robot, one simple way to provide an AR environment would be through using the camera and tablet on the body of the robot to present an augmented perception of the real world which has been fed through the camera onto the tablets viewport.The other more advanced way to achieve an AR perception would be to use a thirdparty AR headset to perceive the common learning context generated and projected by the social robot for the students.With this approach, the user would view a 3D object which is to be projected from the tablet creating a new medium, in which learning information is ascertained and understood.An example of this approach would be a social robot acting as a learning companion, interacting with a student whilst the student can then visualise the learning materials being conveyed in a 3D augmented environment via the AR headset.A rendering of this can be seen in Fig. 1 whereas Pepper robot displays a QR code on the tablet, and the student can then view augmented objects through the use of an AR headset or a mobile handset.
Based on the two frameworks, a set of e-Learning software applications specially developed for the Pepper robot will be used for user evaluation experiments.
Computer Science degree undergrads will participate in these experiments to investigate and test 'existence' and 'high quality' hypotheses proposed in the Section 3.

CONCLUSIONS AND FUTURE WORK
Although there is limited previous research published in regards to the combination of AR, educational studies and social robots.The author has noticed an increase in the number of research studies on these topics as AR and social have become more affordable and robust.It is likely that future research will focus on the implementation and interaction of these technologies within educational settings as to better validate the usefulness and practicality.
While AR alongside social robots can potentially support effective learning and teaching throughout key stages levels to that of a university level course, however, varying studies have demonstrated conflicting opinions in regards to the cognitive load which is placed onto a student.The impact on cognitive load has been associated with the amount of materials and complex instructions provided although this may stem from inadequate technological expertise, poor user experience design or lack of direction within the classroom and thus will not reflect on the sole usability of social robots alongside AR.
The authors have proposed that an integrated solution framework be implemented as to better support the integration of social robots alongside AR within an educational environment.The social robot would act as both a tutor and a companion, operating independently as to interact students.To ensure that the integration is deemed successful the social robot must be perceived credible by the students and the faculty as to vend clear and precise, non-complex instructions.
We note the integrated solution framework outlined in this paper may change as further studies and implementations reveal the need for additional/different functionality.In particular, the following gaps in research have been identified in regards to the combination of AR and social robots in an educational context.It is the authors' recommendations that the following points be used as a guide to future research undertaken: (I) Additional research could be directed towards the integration of social robots and learning environments over an extensive period of time.
(II) More studies aimed at understanding multisensory experiences alongside social robots and AR within learning environments in regards to haptics.
(III) In order for a social robot to participate in natural modes of communications with human beings and be successfully integrated and accepted within society in a broader sense, they should be designed to have the ability of recognize a person's affective states (emotions, moods, and attitudes), so that they could respond appropriately during the e-Learning process with students.
(IV) As the effects of cognitive overload have not been directly linked to the use of AR and social robots, it is recommended that the criteria in which this is experienced be further researched when implementing social robots into the classroom.
(V) As robots can be seen as a novelty, further research would be required to see if results are consistent once the 'novelty' has worn off as to determine the usefulness of implementing such a system within an educational environment.
(VI) As new emerging technologies may be difficult to use, especially if the user has not encountered them previously it is important that further research be conducted into the usability of such devices and the user's preferences and opinions.

Fig. 1 :
Fig.1: Using Pepper Robot's Built-in Tablet and AR Headset for Providing Augmented Reality