Addressing Motivation in an E-Learning Scenario by applying Narration and Choice

Self-directed learning is a critical requirement in adult education, which stresses the importance of appropriate learning environments, to support this process. Moreover, motivation of learners is critical for the acquisition of knowledge in a self-directed manner. Gamification, theoretically framed by self-determination theory, provides a framework to address learners’ three basic psychological needs for competence, autonomy and relatedness and might therefore benefit motivation in the long term. We conducted a theory-based approach for an experimental comparison, which was pre-registered. To this end, we implemented a learning environment with gamified adaptions including a narrative, a pedagogical agent and allowing choice in the course of the learning environment to address the three basic psychological needs as proposed by self-determination theory, with a focus on autonomy and relatedness. In a user study, we compared the gamified learning environment with a control version to examine the effect of the gamification elements on need satisfaction and situational motivation. Our results show that the need for autonomy was significantly better addressed within the presented gamified learning environment, but no other significant positive effects on motivational, cognitive or behavioural outcomes can be determined.


INTRODUCTION
The relevance of lifelong learning, especially in a self-directed manner, has been increasing in relevance in recent years, especially during the Covid-19 pandemic. Acquiring knowledge and skills in this manner requires a high degree of selfmotivation and focused attention (Fahnoe & Mishra, 2013). Therefore, motivation is considered a prerequisite for learning (van Roy & Zaman, 2017). To voluntarily engage in any form of self-directed learning, the learner has to be self-motivated, ideally based on autonomous motivation, which means that the relevance and importance of the respective behaviour is accepted by the learner (Burton, Lydon, D'Alessandro, & Koestner, 2006). This autonomous motivation can either be intrinsic motivation, which describes the motivation to engage with a task because the execution of the task in itself is pleasurable, or so called identified motivation (Deci & Ryan, 2011).
To address this challenge to the learner, which is relevant in online courses and in mobile learning applications (Christensen et al., 2013) but also extends to learning in schools and adult learning (Park, 2009), researchers and practitioners try to promote autonomous motivation. This process can be considered as one of the most relevant concepts in learning (Vallerand et al., 1992) based on its positive relationship to learning performance (Law, Lee, & Yu, 2010) and related constructs (Froiland & Worrell, 2016). As a theoretical basis for this approach, self-determination theory has been applied in learning contexts for several decades, proposing the satisfaction of the basic psychological needs autonomy, competence and relevance, and their potential beneficial impact on the motivational state of learners. The theory especially promotes the benefits of autonomy-supportive classrooms, and the internalization of extrinsic motivation due to autonomy support (Deci & Ryan, 2011). Internalization only occurs in settings in which the needs for competence, autonomy and relatedness are satisfied, resulting in a shift to more autonomous motivation (Deci & Ryan, 2011).
This approach in theory as well as in its aim has been adopted in the area of gamification research in relation to education (Seaborn & Fels, 2015), in which technology plays a major role and presents a new perspective to address the three basic psychological needs of learners. As shown by Sailer, Hense, Mayr, and Mandl (2017) distinct gamification mechanisms might be suitable to address specific needs. If all needs of the learner are satisfied, this should result in internalisation of motivation (Deci & Ryan, 2011) and ideally be already reflected in short-term changes in motivation, i.e. situational motivation, which reflects the motivation in relation to the current interaction (Guay, Vallerand, & Blanchard, 2000). Because course work is often linear in nature and online courses often require learners to work on their own and therefore lacking the relationship to peers in this context, we specifically address the gamification mechanisms of choice and non-competitive relatedness, as opposed to two-edged approaches such as leaderboards (Seaborn & Fels, 2015). In more detail, our study is focused on the initial effects on need satisfaction and motivation if the gamification elements of meaningful choice, narrative and a relatable pedagogical agent are introduced into an e-learning environment. This should result in a technology-supported learning experience, which persuades the learner to actively engage with the learning content by affecting need satisfaction as well as situational motivation through gamification.

Motivation in Learning
Autonomous motivation is a prerequisite in learning scenarios (van Roy & Zaman, 2017), however in self-directed learning this is even more critical, because the learner has to control his or her learning behaviour in pursuit of their learning goals (Winne & Hadwin, 2010). The prevalence of self-directed learning scenarios has been steadily increasing reflected in MOOCs and a diverse range of elearning applications, but also in a shift to online teaching in reaction to the current pandemic situation. All these approaches aim to enhance motivation and stress the active role of the learner within the learning process (Abar & Loken, 2010;Efklides, 2011;Mega, Ronconi, & De Beni, 2014). But even though the idea is to establish the learner as an independent initiator of the learning process (Otto, Perels, & Schmitz, 2011), there is often no clear theory-based approach in digital applications.
However, some well established theories in learning such as the self-determination theory have also been considered in relationship to technologysupported learning (Seaborn & Fels, 2015). Selfdetermination theory proposes that motivation is connected to basic psychological needs, namely the need for competence, autonomy and relatedness (Deci & Ryan, 2011). Satisfaction of these three psychological need in learning settings can support the learner in developing a more autonomous motivation Tsigilis & Theodosiou, 2003;Vallerand et al., 1992). In more detail, a continuum of motivation is proposed, ranging from extrinsic to intrinsic motivation, with identified and intrinsic motivation being considered the forms of autonomous motivation. To enhance autonomous motivation in a learning scenario, the three basic needs have to be addressed to result in more autonomous motivation (Deci, Schwartz, Sheinman, & Ryan, 1981), this process is termed as internalization and integration of extrinsic motivation (Deci & Ryan, 2011).
In most systems in self-directed learning, autonomous motivation has to be established quickly, because otherwise the learner might not continue with the learning environment. We therefore focus on situational motivation, which refers to the current interaction, but can also be depicted on the continuum from extrinsic to intrinsic motivation (Guay et al., 2000). This requires adequate satisfaction of the basic psychological needs, which should result in internalisation of situational motivation, resulting in less external and more autonomous motivation (Deci & Ryan, 2008).
However, as shown in reviews by Seaborn and Fels (2015) and Nacke and Deterding (2017), even though gamification is used, its application as well as research on the topic is often not based on theory. The integration of a gamification element does not necessarily result in enhanced motivation, the learning environment as a whole has to provide meaningful addition to the learning process (Dicheva et al., 2015). Self-determination theory has been used as a theoretical framework to provide such meaningful additions in learning environments (Sailer et al., 2017;Winne & Hadwin, 2010). As outlined in a meta-analysis by Sailer and Homner (2019) gamification in general appears to have a positive effect in learning, but the contribution of specific aspects in gamification on motivational, behavioural and especially cognitive learning outcomes is not yet conclusively answered.
There have been claims in several studies based on self-determination theory to have found links between specific gamification elements to each of the three basic psychological needs in respect to motivational outcomes (see Table 1).
The meaningful integration of gamification elements based on self-determination theory into learning environments appears to result in noticeable changes in the perceived need satisfaction. However, in the studies linking gamification and need satisfaction, e.g. Sailer et al. (2017), the demonstrated effect is not extended to motivation. In the studies, which directly measure motivation e.g. Barata, Gama, Jorge, and Goncalves (2013), the focus is on global intrinsic motivation, which is measured by the Interest/Enjoyment of the Intrinsic Motivation Inventory (Ryan, Mims, & Koestner, 1983). The situational motivation, referencing the current interaction, it not measured and addressed.

Autonomy
Choice of task order (Barata et al., 2013) Contextual embedding (Sailer et al., 2017) Competence Visual tracking of progress (Dichev, Dicheva, Angelova, & Agre, 2014;Sailer, Hense, Mandl, & Klevers, 2014) Positive feedback (Dichev et al., 2014;Sailer et al., 2017) Relatedness Interaction with virtual characters (Rowe, Shores, Mott, & Lester, 2011) Narrative with a relevant role for the user (Rowe et al., 2011;Sailer et al., 2017) However, if internalisation of motivation has to occur in the short-term for a learner to continue with a selfdirected learning scenario, an effect on situational motivation has to be demonstrated (Schaper et al., 2021). Even though the effect of game fiction, referring to the inclusion of a fictional storyline, in gamified learning on motivational and cognitive outcomes was not initially supported in the metaanalysis by Sailer and Homner (2019), the analysis presents several aspects, which should be considered for investigation of this factor to potentially discover such an effect. Foremost, an effect would be expected in respect to selfdetermination theory as outlined by Rigby and Ryan (2011), but also based on the work of Rowe et al. (2011) and Sailer et al. (2017), especially because a positive effect of game fiction could be demonstrated on behavioural outcomes. Because behavioural measures are often measured as aspect of the interaction with a learning environment, the use of a measure to relate to motivational aspects during the interaction, such as situational motivation (Guay et al., 2000) might be more suitable to find an effect on motivational measures. Furthermore, the effect of narrative in learning environments is usually not examined in experimental designs (Sailer & Homner, 2019), which stresses the need to address this factor in a more controlled setting.

Additional Constructs
For a more comprehensive view on our learning environment, we also considered the concepts of user experience and engagement. User experience can be classified in three subtypes, ergonomic quality, hedonic quality and appealingness of a product, as proposed by Hassenzahl (2001), and describes different quality aspects worth considering for designing an e-learning application (Gordillo, Barra, Aguirre, & Quemada, 2014).
In relation to this, engagement refers to the quality of a user experience in terms of the user's active emotional and cognitive involvement in a learning context (O'Brien, Cairns, & Hall, 2018). Engagement can be considered as critical for effective knowledge acquisition (Finn & Zimmer, 2012;Rodgers, 2008) and learning outcomes in general (Beer, Clark, & Jones, 2010). Due to their significant relevance in the development of an e-learning environment, both constructs were additionally considered.

CONTRIBUTION
In the present study, we address need satisfaction, by adding meaningful choice and a relatable narrative as well as an agent, with the overarching aim to affect situational motivation through internalisation. We use an experimental as opposed to a quasi-experimental study design and preregistered our design, measures and hypotheses (see 9. Pre-Registration) to counteract the problem of insufficient methodological rigor outlined by Sailer and Homner (2019) as well as Seaborn and Fels (2015).
By addressing the needs for competence, autonomy and relatedness by integration of a narrative as well as choice and a relatable agent we plan to increase need satisfaction. In particular:  H1a: Need satisfaction for autonomy is higher in the gamified condition.  H1b: Need satisfaction for competence is higher in the gamified condition.  H1c: Need satisfaction for relatedness is higher in the gamified condition.
Satisfying the basic psychological needs should result in an internalisation of motivation in the gamified learning scenario. In particular:  H2a: Intrinsic motivation is higher in the gamification condition.  H2b: Identified regulation is higher in the gamification condition.  H2c: Extrinsic motivation is lower in the gamification condition.
Even though the focus of the study was set on need satisfaction and situational motivation, we assessed several additional measures for additional insights into the perception of the learning scenario, namely knowledge acquisition, engagement, user experience, and acceptance of the learning scenario, which should also be positively affected by the gamification mechanisms.
 H3: Engagement is higher in the gamification condition.  H4: Knowledge is higher in the gamification condition.  H5: User Experience is higher in the gamification condition.  H6: Acceptance for the e-learning is higher in the gamification condition.

LEARNING ENVIRONMENT
Two versions of a learning environment were designed and implemented. A basic version serves as the control group, the experimental condition uses the same learning environment with additional integrated gamification elements. They were implemented with the Python-based open-source software Ren'Py (Rothamel, 2021), a visual novel engine. The final gamified version and control version were published as a browser application and are freely accessible. Both learning environments feature a visual user-interface and voice recordings.

Learning Material
The learning scenario was set up as an introduction to Finnish language. To this end, the learning material was divided into four independent lessons. The topics for the lessons were: (1) Greetings, saying goodbye and getting to know someone, (2) useful expressions for everyday life, (3) Finnish numbers and their composition and (4) personal pronouns and regular verb endings.
Learners received an overview of the lessons after a short introduction to the learning scenario. Each lesson consisted of a learning section providing the content of the respective lesson with additional summary tables, audio presentation of newly introduced words and a subsequent test section with five single-choice questions based on the content of the lesson. Users received feedback with a detailed explanation for the correct solution. After all four lessons were completed, a final test took place. It consisted of four questions from each of the previous lessons. Two questions were repetitions from the respective test section and two questions were newly introduced. During the final test, users only received feedback on the correctness of their answer.

User Interface
All the assets of the user interface were hand-drawn, therefore unknown to the participants and set up to simulate a classroom setting. Because the learning environments are focused on the transfer of knowledge through a virtual pedagogical agent, the user interface resembles a personal interaction with another student. The information are displayed via text boxes and audio and makes use of tables to summarize content (see Figure 1). The audio button, which appears whenever a new word is introduced, can be used indefinitely until the user chooses to advance the dialogue.

Gamified Adaptions
The gamified learning environment included the following gamification mechanics: First, at the beginning of the learning environment, the user is asked for their name, which is referenced several times afterwards in direct speech. This feature should address the need for autonomy and social relatedness (Sailer et al., 2017) as the user takes on an integral and significant role in the learning environment.
Second, while in the control condition the order of the lessons is fixed, in the gamified condition the user is able to choose the order in which the lessons are presented, therefore satisfying their need for autonomy (Naul & Liu, 2020;Sailer et al., 2017).
Third, the need for competence as well as social relatedness of users is supposed to be addressed through empathetic feedback via the facial expressions and gestures of the pedagogical agent Sailer et al., 2017). To this end, four static states of the agent were implemented, expressed by body language: (1) approving, (2) embarrassed, (3) explaining and (4) neutral (see Figure 2). The positively formulated feedback for correct as well as incorrect answers is enhanced by the emphatic reaction of the virtual agent.
Fourth, the gamified condition has an implemented story and the agent has a personalised conversation with the user while also referring to them as 'you'. The setting of the story is a language tandem session in which the pedagogical agent teaches the user basic knowledge of the Finnish language. For that matter, the agent is introduced as a native Finnish speaker that spends some time abroad at a German university. In the control condition, the agent does not address the user on a personal level and remains only in the neutral state with neutral text without referencing the implemented story.

Participants
Initially, 47 students of blinded university participated in the online experiment. Seven had to be excluded due to incomplete data, resulting in a final sample of N = 40 participants (26 female, 13 male, 1 diverse) with a mean age of 21.73 (SD = 1.63), randomly allocated to both conditions (Ngamification = 19, Ncontrol = 21). Students received ungraded partial course credit for participation as part of their curriculum. The participants were instructed not to take part in the study if they had prior knowledge in Finnish.

Measures
We assessed need satisfaction for competence, autonomy and relatedness with the questionnaire by Sheldon and Filak (2008). Because the main focus of the study was on the effect of the manipulation on need satisfaction, i.e. motivational outcomes, we integrated a newly developed additional questionnaire to assess these measures (Wolf, Hohm, Künzl., & Hurtienne, unpublished). This questionnaire is based on the work of Desmet and Fokkinga (2020) and was added as a further indicator for need satisfaction, especially relatedness and autonomy, which are sometimes inconsistent (Hassenzahl, 2001). Situational intrinsic, identified, and extrinsic motivation as well as amotivation were measured with the Situational Motivation Scale (Guay et al., 2000). The performance during the learning phase and the final test at the end of the learning environment was tracked for each participant. As additional measures we included engagement measured with the UES-SF (O'Brien et al., 2018), the User Experience Questionnaire (Laugwitz, Held, & Schrepp, 2008) and assessed the acceptance of the learning environment (Kreidl, 2011). The number of items and Cronbach's alpha as indicator for internal consistency of the respective scales, for all questionnaires are presented in Table 2. Finally, participants had to give qualitative feedback regarding positive as well as negative aspects of the learning scenario to serve as indicators for errors and potential improvements for future work.  Sheldon and Filak (2008) b (Wolf et al., unpublished)

Procedure
The study was an online experiment and with one session, which had to be completed in one sitting, but at a time freely chosen by the participants. Participants were instructed to have sound enabled and work on the study without interruptions in an undisturbed environment. Before starting the study, they were informed about its content and filled in a consent form and reported demographic data.
Based on their randomly allocated condition participants received a link to one of the two independently hosted learning environments. At the end of their respective learning environment, the users were asked to take a screenshot of their encrypted performance, presented as a random string to transfer their results to the survey tool. Subsequently, they completed all questionnaires including qualitative feedback as well as an independently saved e-mail address to allocate course credit. Participation in the study took around 30 minutes.

RESULTS
All analyses were conducted with SPSS 25 and JASP (JASP-Team, 2021) and alpha was set at .05.
Due to the small sample size, we used Mann-Whitney U tests for group comparisons. For the tests of hypotheses, we used one-sided tests as stated in the preregistration. All descriptive values are shown in Table 3.  Sheldon and Filak (2008) b (Wolf et al., unpublished)

Quantitative Data
In regard to H1a we found significant higher need satisfaction of autonomy based on Sheldon and Filak (2008)  Concerning H2a-H2c we found no significant higher values in situational intrinsic motivation (U = 168, p = .200) or identified motivation (U = 234, p = .829) as well as no significantly lower values in extrinsic motivation (U = 184, p = .668) in the gamified condition relative to the control condition.
Regarding H3, there was no significantly higher engagement in the gamified condition relative to the control condition (U = 207.50, p = .591).
For knowledge acquisition, four of the participants failed to transfer their encrypted results into their questionnaire, resulting in a reduced sample (Ngamification = 15, Ncontrol = 21).
We compared the mean percent of correctly solved tasks across all lessons between both conditions for H4, resulting in no significant difference between the control condition (M = 80, SD = 12) and the gamified condition (M = 72, SD = 18, U = 194, p = .885).
There was also no significant difference in the performance during the final test between the control condition (M = 79, SD = 13) relative to the gamified condition (M = 73, SD = 16, U = 189, p = .850).
For H5, we found no significant differences in the user experience between the control condition relative to the gamified condition (U = 185.50, p = .357).
Concerning H6, there was no significant differences in the acceptance of the learning environment between the control condition and the gamified condition (U = 182.50, p = .327).

Qualitative Data
The qualitative feedback was very diverse, potentially because positive and negative feedback were mandatory. Concerning errors within the learning environments, there was one case in which an undisclosed error message was present for one participant in the control condition. Three participants in the gamification condition and one in the control condition indicated, that they would also like to receive feedback on the correct solution in case of errors within the final test. Three participants in each condition noted that a back button would be helpful to re-read instructions or to undo an unintentional click, which advanced the user within the learning environment.

DISCUSSION
Concerning our pre-registered hypotheses, we found significantly better need satisfaction of autonomy in the gamified condition with the questionnaire of Sheldon and Filak (2008), but this was not supported by the additional questionnaire by Wolf et al. (unpublished), which partly supports H1a. There were no significant differences regarding the need for relatedness and competence, we therefore reject H1b and H1c. The non-significant differences regarding intrinsic, identified and extrinsic situational motivation indicate that no discernible internalisation of motivation occurred in the gamified condition, we therefore reject H2a-H2c. There was also no significant difference concerning the cognitive outcomes, with a similar performance of participants in both conditions in the tests within the lessons and the final quiz, we therefore reject H4. Furthermore, we found no indication of a significant effect of the gamified conditions regarding engagement, user experience or acceptance, therefore rejecting H3, H5 and H6.
Overall when considering motivational, cognitive and behavioural outcomes of the learning environments (Sailer & Homner, 2019), there is no indication that there was a discernible benefit on motivational measures, with the exception of better autonomy need satisfaction in the gamified condition. This in in line with the conclusions drawn by Sailer and Homner (2019) regarding the effect of game fiction on motivation, but extends this conclusion by demonstrating that failing to find this effect appears to be no result of quasi-experimental studies, but is also not present in controlled experiments. Our aim to test if the use of a measure for situational motivation would reflect the effect of gamification on behavioural measures (Sailer & Homner, 2019) was not supported, stressing the need for the additional assessment of behavioural data. This indicates that even though there appears to be a positive effect on the need for autonomy due to the narrative and/or additional choice within our gamified condition, this is not sufficient to extend a significant influence on the three outcome types. There was also no indication that such a short interaction with an online learning environment would result in internalisation of situational motivation as reported by Schaper et al. (2021), even though an effect of gamification in form of narrative and choice on the need for autonomy was found.
The difference in results using two different questionnaire measures also raises the problem if both measures refer to the same construct. The items by Sheldon and Filak (2008) are well established, but show an extremely low internal consistency on the relatedness scale. For competence and autonomy, both questionnaires show similar consistency.
The learning environment was well received in terms of user experience and concerning its acceptance and participants showed moderate to good performance, indicating no problems with ceiling effects. There were some suggestions for improvement in the qualitative data, but overall it could be expected to be well received in an applied content, which is supported by participants suggesting this use in the comments. However, the benefit and potential differentiation of the gamified adaptions does not result in an overall difference.

Limitations and Future Work
The descriptive values and the qualitative feedback indicate that the learning environment could be improved. This includes the gamified aspects, which should ideally result in better outcomes (Rigby & Ryan, 2011). Based on our results it appears adequate to track behavioural measures in a subsequent study, because their effect has metaanalytic support (Sailer & Homner, 2019) and our results indicate that the use of situational motivation measures were not significantly influenced by our manipulation.
Based on the mostly non-significant differences between both conditions it would be of interest to provide the participants with an overview of the respective other condition to see if the additional narration and choice is actually preferred, even if it is not reflected in most the measures used. Critically, if there is still a preference for either condition it would be relevant for gamification in theory as well as in applied settings to find a suitable indicator, which does not rely on direct comparison. This could also require a more prolonged interaction with a more extensive learning environment, which was rather brief in the current study. In addition, a larger sample would be more suitable to find effects.

CONCLUSION
In this study, we investigated the effect of narration and choice on need satisfaction, situational motivation as well as knowledge acquisition, engagement, user experience and acceptance in an e-learning scenario. Even though we found that the need for autonomy was significantly better addressed within the gamified learning environment, there were no other significant positive effects on motivational, cognitive or behavioural outcomes. The contribution fills a gap in gamification research outlined in the meta-analysis by Sailer and Homner (2019), to conduct a controlled experiment to test the influence on game fiction on the three outcome types, however the results indicate that our approach did not result in a positive effect. More controlled research on gamification in digital learning is needed to investigate its actual benefit and also to identify measures, which consistently reflect the potential benefits.