University of Birmingham Investigating the interaction between personalities and the benefit of gamification

Many studies have confirmed the benefit of gamification on learners’ motivation. However, gamification may also demotivate some learners, or learners may focus on the gamification elements instead of the learning content. Some researchers have recommended building learner models that can be used to adapt gamification elements based on learners’ personalities. Building such a model requires a strong understanding of the relationship between gamification and personality. Existing empirical work has focused on measuring knowledge gain and learner preference. These findings may not be reliable because the analyses are based on learners who complete the study and because they rely on self-report from learners. This preliminary study explores a different approach by allowing learners to drop out at any time and then uses the number of students left as a proxy for motivation and engagement. Survival analysis is used to analyse the data. The results confirm the benefits of gamification and provide some pointers to how this varies with personality.


INTRODUCTION
Over the past few years, the popularity of online learning systems has increased, and much research has been devoted to the improvement of these systems. Recent research has explored the adaptation of learning systems to make them more suitable and enjoyable for learners. Adaptation refers to the process of tailoring something to meet users' needs [11]. Accordingly, there have been several attempts to design learning systems based on the learners' characteristics, for instance, their individual skills, knowledge, affective states, and personalities [4], [8]. In one example, Filippidis and Tsoukalas [29] adapted the instructional design of a learning system based on learners' learning styles. The researchers concluded that the performance and satisfaction of learners improved as a result of using this adaptive system. However, Dichev et al. [24] argue that motivation is a crucial factor that must be considered to ensure successful learning outcomes. Moreover, Carini et al. [15] state that if learners' motivation and engagement levels increase, it may serve as a positive predictor of their achievement. Motivating and engaging learners in an online learning system can be a significant challenge. Some researchers have suggested the incorporation of gamification elements into instructional design by using points, badges, and the like to enhance learners' motivation and engagement levels. Many studies have confirmed the positive effects of such gamification elements [10], [50]. Nevertheless, the problem with these elements is that some learners find them tedious and annoying (particularly in long-term courses) [30], and other learners may focus on collecting points and badges instead of concentrating on the educational content [27]. Theoretical work predicts that gamification has different impacts on individuals with different personalities [52]. Some learners benefit from the gamification of the course, whereas others are negatively affected. Thus, it is recommended that a learner model be built and used to adapt gamification elements based on the different personalities of the learners [52]. To do so, it is important to understand this relationship. Previous research has attempted to investigate the relationship between users' personalities and gamification elements [36], [19]. Most of these studies showed that individuals with different personalities favoured different game elements. For example, extroverts preferred leaderboards, whereas introverts preferred physical rewards, such as gifts and key rings. However, in all these studies, the learners were required to complete the entire experiment, after which they were required to respond to a self-report questionnaire that asked them to specify their preferred gamification elements. In some studies, learners were asked to take a test to measure the extent of the knowledge gained throughout the course. However, the results 2 obtained through the above studies may be unreliable and may not provide a good measure of motivation. Moreover, these studies did not consider the data on learners who dropped out part way through the experiment. It may be crucial to factor in these dropouts and to determine whether their dropping out was affected by the gamification elements. Thus, as a more effective way to measure motivation and engagement, we measure the length of time that learners actually use the system. We hypothesise that learners who are more motivated and engaged will use the online learning system for a longer duration, which may in turn enhance their performance. Therefore, we conduct a preliminary experiment to investigate how different individuals with different personalities are motivated by game elements. We do this to build a learner model that can be used to adapt gamification elements based on learners' personalities. The hypothesis is that conscientious learners will not receive any benefit from gamification whilst other personalities, such as extrovert learners will. Further, neurotic learners will be demotivated by the gamification elements. However, the results do not reveal a negative effect for any personality dimension, the learners with different personalities are found to interact differently with the gamification elements. Some types of learner are observed to be highly motivated by these elements, whereas others are not affected at all. This result could be explained by the fact that a limited number of gamification elements are used (namely, points, badges, and leaderboards), and these may not have had as obvious a cost on learners as would the incorporation of, for instance, social features. In addition, only a few learners have extreme scores on any of the specific personality dimensions. Therefore, future experiments should be conducted with a long-term course and with more learners to ensure that different personality dimensions are better represented. In addition, adding higher cost gamification elements would be valuable.
In this work we use a more ecologically realistic methodology and method of analysis to confirm that the value of gamification does vary with personality. These techniques could be usefully applied to other areas.
The rest of this paper is structured as follows. The next section presents an overview of relevant topics, such as gamification, motivation, engagement, and personality. The following section describes the design of the experiment. Finally, the last section presents the results of the experiment and provides recommendations for future work.

Background
The demand for online learning has dramatically increased because of new technologies. Online learning is described by Ally [2] as the ability to access materials via the Internet for the purpose of interacting with instructors, lessons, and other learners. The aim of this interaction is for learners to gain knowledge and to enhance their experiences.
Many studies emphasise the benefits of using an online learning system. Learners have access to their study materials at any given time, from any location, and without extra costs related to accommodation or travel. However, learners in an online learning system may be demotivated because they may feel isolated. In addition, there is a delay in the responses that they receive from their instructors and other learners. Moreover, an online learning system considers all learners as essentially the same. Their differences in terms of preference, personality, and learning style are not usually taken into account [1].

Motivation and Engagement
Motivation is an important area to researchers in psychology, computer science, and business. It is defined by Sailer et al. [46] as a psychological process that initiates and directs goal-oriented behaviours. Dichev et al. [24] described motivation as the extent of one's efforts to accomplish a specific objective. They also defined motivational design as the process of organising resources and methods to enhance motivation in a learning environment or in other environments. According to Dichev et al. [24], motivation is a crucial factor that needs to be enhanced to ensure learners' success. With the emergence of online learning systems, the focus of earlier research was on learners' achievement and how to enhance their performance upon completion of the online course. Lately, however, more studies have concentrated on the importance of learners' motivation and their engagement when interacting with the online learning system [22], [16]. Carini et al. [15] stated that if learners' motivation and engagement are increased, these may serve as positive predictors of learners' achievement and performance. If learners engage more with the system, they will use it for a longer time, which in turn can enhance their performance.
Self-determination theory is one of the most popular theories used to explain motivation, particularly in the field of education [24]. This theory was first proposed by Ryan and Deci [45]. They stated that humans continually and actively gain experience and expertise when they seek challenges. This theory categorises motivation into two main types: intrinsic and extrinsic. For Stannett et al. [50], intrinsic motivation is more important than extrinsic motivation. They mentioned the Foursquare application as evidence of this. They suggested that most users stop using this application because they cannot connect it to any particular goal or aim. However, many other researchers have argued that for an application to be effective, both extrinsic and intrinsic types of motivation must be considered [45].

Intrinsic motivation
Intrinsic motivation occurs when an activity matches a user's goals. Therefore, users are satisfied when they engage in an activity. Three factors must be considered to motivate users intrinsically: autonomy, competence, and relatedness [50]. Autonomy can be achieved by aligning tasks with users' values. Users should be able to say, 'I am in control. I am doing things that follow my values.' Competence can be achieved when users feel that they are working towards their own goals and objectives. Users should be able to say, 'Yes, I am doing it. I am getting better.' Relatedness can be achieved when users feel that they are part of a group that has the same goals and interests.

Extrinsic motivation
In this type of motivation, users are motivated when they are provided with an external element. These elements can be used to encourage users to perform activities. For example, learners can be given physical rewards for completing a specific task [45]. From another point of view, intrinsic and extrinsic motivation are complementary. Users can be motivated intrinsically and extrinsically on different scales. However, in the case of extrinsic motivation, learners will complete the task only if they are offered external rewards. Thus, learners may lose their ability to change their behaviour on their own.

Gamification
The concept of gamification has been increasingly used in marketing and business to attract customers. Lately, gamification has been widely used in learning and education. Robson et al. [43] believe that gamification will be extensively used in numerous applications. Gamification is defined in different ways according to the area involved. For example, in marketing, it is defined by Bunchball [12] as the integration of a game dynamic into a site, service or community to engage participants. In education, it is described by Caponetto et al. [14] as the process of using game thinking and game mechanics to solve problems. Lee and Hammer [39] provided a similar definition of gamification in education. It is the use of game mechanics, game dynamics and a game framework to promote a desired behaviour. Most of these definitions have different phrasings but are similar in meaning. In these definitions, the focus is on changing learners' behaviours by motivating them with the use of gamification elements to enable better engagement. Most researchers focus on one aspect of the definitions, the use of game elements such as points and rewards in a non-game context (e.g., the learning environment) to enhance the motivation and engagement of learners.
Robson et al. [43] showed that gamification is not a complete game. It includes specific elements, such as badges, rewards, levels, avatars, time constraints and leaderboards [10], [21]. In gamification, users achieve their goals in a game environment. For example, users who are concerned with their fitness and with sports can use a gamified application to motivate themselves. They can earn rewards and badges when they complete a specific exercise, or they can compete with their friends by publishing their levels or scores on social media, as seen in the Nike+ application [16]. This application awards users with points and badges as they walk more steps. Users can also use their points to compete with others in the leaderboard or they can publish their results on social media, such as: Twitter [16] [50].
In the same way, gamification is used in online courses to increase learners' engagement. According to Stott and Neustaedter [51], using gamification in learning can increase students' achievements. Students imagine that they are in a game, and thus they are less likely to fail or fear failure. In addition, by using levels and progress in gamified learning applications, students can start from one point until they stop or fail. An important feature that can be provided by gamification is rapid feedback. In addition, gamification supports and improves learners' social skills. This benefit is possible because of the interaction between learners, which can be either cooperative or competitive [24]. Recently, many studies have attempted to examine the effect of gamification on learners. For example, Cheong et al. [16] asked 76 students to use QuickQuiz, an application designed to motivate learners. After four weeks, the participants were asked to complete a self-report questionnaire about their engagement. Of the learners, 77% confirmed that they were more motivated because of gamification elements and that they enjoyed using gamification elements. Barata et al. [6] asked 52 learners to complete a questionnaire about their engagement after using two versions of a learning application, one with gamification elements and the other without. The results indicate that most of the learners enjoyed the gamification elements. Although positive effects of gamification were shown by some studies, other researchers claim that there is not any significant difference in satisfaction between learners assigned to a gamified online learning system and those using a non-gamified system. It is claimed by Merry et al. [42] that the level of learners' knowledge and their motivation in a gamified learning system does not show any improvement compared to those in traditional learning systems. Moreover, other studies argue that gamification elements can be a boring or annoying technique for some learners. In addition, others have agreed that the effects of gamification might only apply in shortterm courses [30]. To address these issues, Tondello et al. [52] suggested designing and adapting gamification elements on the basis of learners' attributes, such as: learners' affective states and personalities.

Personality
Previous research has confirmed that personality has a major influence on individuals' behaviour, such as their academic performance and choice of job [20]. Personality can be described as a set of characteristics and psychological factors that are used to control how individuals feel, think and interact with others. These characteristics can be identified as personality traits. Personality traits provide a deep understanding of personality and comprise all aspects of individuals and how they interact with the outside world [33]. There are three common models of personality that are widely used: Eysenck's theory of personality, the Myers-Briggs Type Indicator, and the Big Five personality traits [26], [38]. In this paper, the focus is on the Big Five personality model as it is one of the most popular and has been widely used in similar research.

The Big Five personality model
A common way to analyse users' personalities is through the five-factor model of personality known as the Big Five model [32]. This model divides users' traits into five categories: conscientiousness, agreeableness, extraversion, neuroticism, and openness to experience. The first dimension of the Big Five personality model that has universal agreement is extraversion. Individuals who are extraverted are usually active, talkative, social, and assertive. Hogan and Hogan [34] stated that this type of personality can be divided into two components: ambition (having initiative and ambition) and sociability (being sociable, exhibitionist and expressive). The second dimension is neuroticism or emotional instability. The common traits associated with this personality are being anxious, depressed, angry, embarrassed, emotional, worried and insecure. The third dimension is agreeableness, which some researchers refer to as likeability or friendliness. The common traits of this dimension are being flexible, trusting, cooperative, forgiving and softhearted. The fourth dimension is called conscientiousness or conscience. It is also labelled by other researchers as conformity or dependability. The traits associated with this personality are being careful, responsible, organised, and prepared. Furthermore, others state that this dimension of personality is associated with the traits of being hardworking, achievementoriented and persevering. This can explain the amount of research that has been done to examine the correlation between conscientiousness and academic achievement and job performance. The fifth personality trait is the most complex. It is openness to experience, which has been expressed by other researchers as being intellectual. The traits associated with this personality are being imaginative, cultured, curious and broad-minded [7], [44]. The summary of the traits associated with each dimension are shown in Table 1.

Big Five instruments
Many instruments have been created for the purpose of assessing the Big Five personality traits. The most popular instruments that are cited in the research are the NEO Five Factor Inventory (NEO-FFI) and the Big Five Inventory (BFI). There are many versions of each of these instruments. For the version of the NEO-FFI that was developed by Costa and MacCrae [20], there is a version called the NEO-PI that contains 181 self-reported questions and another version with 240 questions. However, the lengths of these two versions make it difficult to use them in research. Most learners will select random answers to finish all of the questions. Thus, the 60-question NEO-FFI version was developed. This instrument has since been modified into several shorter instruments, each containing 10 questions [20], [5]. However, most of these short instruments suffer from reliability problems. Moreover, because the NEO-FFI is not free, many researchers have focused instead on a free version of the BFI that was developed by John et al. [37]. This tool has fewer questions (numbering 46). In addition, this tool has a higher reliability score than the NEO-FFI. There are several versions of the BFI, in many languages, that are suitable for a variety of ages. For example, there are versions that target adults and others that are appropriate for children or parents [37]. 5

Adaptation
Recent research has shown that learners' performance is much better when they are taught individually than when they are in a classroom with other learners [23], [31]. This is because the instructor can better understand the most effective way to deliver the information to the student and to ensure that he or she engages with the learning content. However, this becomes more difficult in an online learning system in which the instructor and learners are physically separated. However, it is claimed by many researchers that the 'one-sizefits-all' approach is difficult to apply in online learning systems. Consequently, many researchers have shifted to adapt learning technologies and contexts to meet users' needs.
Adaptation is a procedure for tailoring something to satisfy users' needs and wants [11]. In terms of learning, Magoulas et al. [41] explained adaptation as the process of organising the learning environment to accommodate differences among learners. Feigh et al. [28] described adaptation in technology as a human-machine system that can change its behaviours based on learners' requirements. Researchers have applied this adaptation of technology to many disciplines, such as learning, e-commerce, health care, and marketing. One method of adaptation was illustrated in [47], where it is argued that the design of the learning contents based on the mood and the emotion of leaners can help to improve the learners' performance. This can be accomplished by designing the learning system based on the affective states and moods of learners. Adaptation based on emotional measures enhances the performance of the learners. However, measuring and recognising the emotions of learners requires advanced tools that may be expensive and difficult to use [47]. Thus, researchers have shifted to adapting educational content based on learners' knowledge and learning styles. For example, Beyer and Davis [8] point out the importance of analysing the educational content to make it suitable for each learner. Again, the consideration of the knowledge level and experience of learners is a crucial and useful technique for enhancing learners' performance, but it is difficult to design specific material for each individual learner. Adapting educational content and instructional design based on learners' learning styles can enhance the learners' performance and satisfaction, as shown by Franzoni et al. [31] and confirmed by Alshammari et al. [4]. Researchers argued that adaptation can enhance learners' performance and satisfaction and the perceived usability of online learning systems. However, some studies highlight the instability of individual learning styles, since they can change according to the affective state of the learners and their knowledge levels [3], though Shoda and Mischel [48] argued that learners' personalities are considered a stable characteristic. Therefore, it may be better to focus on adapting educational content and determining whether personality has any effect on learners' performance and achievement. For instance, conscientious learners are considered to be dependable and responsible, and thus they may have good class attendance and high academic averages [13]. Bidjerano and Dai [9] showed that personality might be responsible for persistence and consistency in processing information. Users who are more open to experience usually prefer to have a deep understanding of information and are capable of elaborate processing. However, learners who are more agreeable and extraverted are likely to prefer peer learning and collaborative work. Extraverted learners often encounter difficulties when using problem-solving strategies. They sometimes cannot solve or overcome problems easily because of a lack of concentration.
Gamification has been used to enhance learners' motivation recently. However, some research show that it is more effective if the design is based on the learners themselves [52]. For example, the elements of gamification can be personalised on the basis of learners' moods or affective states. However, these states are not stable and learners will be in different moods on different days [47]. In addition, assessing the moods and the affective states of learners is difficult. To address this, adapting gamification elements to learners' personalities has been suggested [52]. For example, research has argued that extraverted learners prefer social elements (such as chats and leaderboards) as a way to motivate them, whereas introvert learners are demotivated by these elements. A few recent studies have attempted to personalise gamification elements on the basis of personality [18]. One of these studies was that of Jia et al. [36], who examined learners' preferred elements. They asked users to complete a personality test and a test to report their favourite element and whether they found this element enjoyable and helpful. This experiment was based on a self-report questionnaire through which users filled the personality test and then indicated their preferred gamification elements. Another experiment was conducted by Codish and Ravid [18], who focused on only one dimension of personality, that of extraversion. In their experiment, they assessed users' personalities by administering the FFI questionnaire. A total of 133 undergraduate students participated and took a gamified course. At the end of the term, the learners were asked about the gamification elements they preferred. After that, Codish and Ravid [19] used a paper prototype to define the relationship between gamification elements and all personalities. The results of all of these experiments confirmed that the extravert learners preferred points, badges, and leaderboards, whereas the introvert and neurotic learners were more motivated by rewards. The experiments showed that highly conscientious users were motivated by level and progress elements. However, these experiments may not reliable since they were based on learners' self-report questionnaire responses. Most of the relevant studies that have attempted to understand the relationship between gamification elements and personality dimensions forced participants to complete the entire experiment, which conflicts with the main aim of gamification which is enhancing learners' motivations. Furthermore, these works excluded the dropout learners from the analysis. This could significantly bias the results since these learners might be the ones in which we are most interested. Some studies also measured the extent of knowledge gained by learners at the end of the experiment as a proxy for their motivation levels, while other studies used a self-report questionnaire to ask learners about their preferred game elements. However, using the level of knowledge gained and self-reported data may be unreliable. Therefore, in this research, we will measure learners' motivation in a different way by providing gamified and non-gamified versions of a learning system to two groups of learners and giving them the freedom to drop out at any time. We hypothesise that learners who are more motivated and engaged will use the learning system for a longer duration.

Method
In this study, we used a new approach to measure learners' motivation and engagement. Instead of analysing the results obtained from the self-report questionnaire from learners who completed the experiment, we used a more objective approach by analysing the number of learners who dropped out at each stage of the experiment. We used the drop out rate as a proxy for motivation and engagement. We hypothesise that learners who are more motivated would use the learning website for a longer duration, which may enhance their performance. The main aim of this study is to measure the motivation and engagement of learners. Therefore, we did not measure learners' knowledge.
To do so, we built two identical versions of a learning website. One version included gamification elements, such as points, badges, and a leaderboard. The other version lacked these elements. The objective of the website was to teach the learners Microsoft Excel. The course was divided into 15 lessons, which started with the introductory topic and progressed into advanced topics.
To run the experiment, we asked 197 learners from different high schools in Saudi Arabia to participate in the experiment. The participants were between 16 and 18 years old (109 girls and 88 boys). Before commencing, ethical approval was granted by the school. In addition, a consent form was sent to the students' parents to obtain their approval, explain the purpose of the experiment and to confirm that all the data collected would be anonymous and secure. The students and their parents were informed that they had the freedom to drop out of the experiment at any time.
At school, the learners were asked to complete a registration form that contained questions regarding their demographic information, a BFI personality test and a test related to the topic. Then, the learners were divided into two groups having approximately the same number of learners and balanced by gender, age, knowledge level and personality. Following registration, the learners were free to use the website whenever and wherever they wished. Learners could also stop using the website whenever they wanted. As the course progressed, we anticipated that several learners would drop out at various stages; we refer to these as drop out learners (DoL) in our study. After six weeks, most of the learners had either dropped out or completed the experiment.
In this experiment, we hypothesised that learners assigned to the gamified version would be more motivated and use the learning website for a longer period of time than learners in the non-gamified version.
Regarding personality, we only focused on learners with high or low values of each personality trait. We divided learners into high, average and low values of each personality. To do this classification, we plot the value of the learners' personalities. Then, we took learners up the high tail as a high extreme in that personality and learners under the low tail as a low extreme. Later, we compared the drop out rate of learners who had high or low scores in each personality trait in the gamified and non-gamified versions. We hypothesise that learners with the most extreme personalities would vary more in their motivation by the gamification elements. Learners who are highly conscientious would not benefit from the gamification elements, and their progress in the two versions would be almost the same. And learners who are highly neurotic would be demotivated from the gamification since they would find it annoying and boring. Other personalities will benefit from gamification.

Results
Unlike other research that has ignored the participants who dropped out during the experiment, we care more about these DoLs. We will use the drop-out rate as a proxy for motivation. 7 We hypothesise that learners who are more motivated will use the learner website for a longer time. Thus, we need a special type of analysis that considers the time spent on the system and the status of the learner as the dependent variables. This type of analysis is called a survival analysis, which is a common statistic used in, e.g., medicine, biological sciences, and engineering. Survival analysis can be defined as a set of methods to analyse the participants' time spent in the experiment, from entering the experiment until the event of interest occurs [35]. This event can be death, disease or dropping out. Survival analysis uses the information provided from learners who completed the course and from the learners who dropped out, which is called censored data. There are two main concepts in a survival analysis: survival and hazard functions. The survival function provides the probability of learners' survival at each timepoint, while the hazard function provides the probability that an event will occur [17].
Survival analysis can be used to compare the survival between two groups. One of the most common methods is using the Kaplan-Meier graph, which is used to plot the survival distribution for two groups. The issue with this test is that it is used to compare the cumulative survival distribution between two groups at arbitrary chosen points and does not present the differences in both groups at all times. For determining continual differences between two groups, many researchers have used the log-rank test. This is based on the p-value obtained from chi-square tests. However, the problem with these two non-parametric methods is that both are used with categorical data, and they are less effective if applied with continuous data. Furthermore, these tests only show if there is a difference between the two groups and do not indicate how much of a difference there is. Therefore, most researchers use a specific type of regression that addresses these issues. This method is called the Cox proportional hazard model [53]. The Cox hazard model is used to evaluate the effect of specific factors on the rate of a particular event happening (death, drop-out) which is called the hazard rate. This model analyses the relationship between the hazard function and the predictors or the treatment. By assuming a nonlinear relationship between the hazard function and the predictors, the Cox hazard allows for the isolation of the treatment's effect from other variables. The Cox hazard can be expressed by a hazard function, which can be calculated using software such as R or SAS using the following formula [49]: :are the coefficients used to determine the effect size of covariates.
(t): This term is referred to as the baseline hazard. Which indicates the value of hazard at time 0.
The Cox hazard can indicate if there is any difference between groups, since this technique calculates the p-value using three different tests. In addition, the Cox hazard shows if the hazard is increased or decreased by examining the sign of the regression covariate, while defining the variation between the two groups can be done by examining the value of exp( ) which is called the hazard ratio (HR). The value of HR can be interpreted as follow [49] [53]:  HR= 0, there is no effect.  HR <1, the hazard is decreased.  HR >1, the hazard is increased.
In this research, we plot the cumulative survival function for the overall learners in the gamified and the non-gamified version using a Kaplan-Meier graph to understand the data. In this analysis, the digital time refers to the course lessons and the event of interest is dropping out. Figure 1 shows the plots of the overall learners' cumulative drop out in the gamified and the non-gamified versions. Then, we use the Cox hazard to determine if there was any significant difference, as well as the size of the difference. Figure 2 shows the result from the Cox model after applying a built-in function in R that calculates the Cox model. The p-value from three different tests is less than 0.05, which indicates there is a significant difference between DoL in the gamified and the non-gamified versions. The value of coef refers to the regression covariate. If the coef value is positive, then the drop-out rate is high in the second group, and the drop-out rate is low if the coef is negative. The result from our analysis is 0.6636, which shows the second group (non-gamified version) has a higher drop-out risk than the first group. The result from exp(coef) refers to the hazard rate (HR), which shows how much the two groups are different. In our analysis, the result of HR is 1.94, which means the drop-out rate in the second group is almost twice as much as in the first group.
The Cox hazard result shows the positive effect from gamification on the overall learners' enhanced motivation. However, we must thoroughly understand which personality is motivated more by the gamification elements and which is not. Therefore, we applied a logistic regression to our data. We assumed that the five personality traits are the independent variables, and the dependent variable is a binary value that is 0 if the learner is a drop-out or 1 if the learner completes the course. We found that the personalities affected by gamification elements are conscientiousness, extroversion and neuroticism. However, we could not use the logistic regression to predict which version is the best for each personality because of the Censored data.
Regarding the effect of gamification on personality, we hypothesise that highly conscientious learners will be less motivated by gamification than low conscientiousness learners. While highly extroverted learners will be highly motivated by gamification. However. highly neurotic learners will be demotivated with these elements. To classify each personality into high, average, and low, we  plotted our data and then took the high tail as the learners with a high specific trait and the low tail as the learners with a low specific trait. The people in the middle are the learners who are considered average regarding a specific personality trait. We are usually only concerned with learners in the high and low tails since those are the learners with extreme personalities, and we expect that those people will be highly affected. Figure 3 shows the classification of the conscientious personality. Learners with a conscientiousness score from 0 to 1 are considered to have low conscientiousness, learners who have a value bigger than 1 and less than or equal to 3.8 are considered average, and learners who have a personality value bigger than 3.8 are considered to have high conscientiousness. We performed the same method with the other personality traits. Later, we compared the survival distribution for the highly conscientious in the gamified and the non-gamified version, and we performed the same analysis for the low conscientiousness learners. Figure 4 shows the Kaplan-Meier graph for the high conscientiousness learners in the gamified and the non-gamified versions. Then, we applied the Cox hazard to find the difference between the drop out rate of highly conscientiousness learners in both versions. Figure  5 shows the results from the Cox hazard. The pvalue from the likelihood test is 0.14, which shows there is no significant difference between the dropout rate of highly conscientious learners in the gamified and the non-gamified versions. The result from the regression covariate is 0.4, which means that the drop-out rate is higher in the non-gamified version.    between the survival in the two versions. The coef and exp(coef) values indicate that the drop-out rate is twice as high in the non-gamified version as in the gamified version.

Figure 7: The result from the Cox regression when it is applied to the low conscientious learners
The Cox proportional hazard is applied to measure the difference between the highly extravert learners in the gamified and the non-gamified version. We found there is a significant difference for these types of learners between the two versions. The drop-out rate for the highly extravert learners is almost a third higher in the non-gamified version. While the low extravert learners in the gamified had no significant benefit from gamification.
For the high neuroticism and low neuroticism learners, there is a significant difference between the drop-out rates in the gamified and the nongamified versions. Both these types of learners receive a significant benefit from gamification. Which conflicts with our hypothesis that highly neurotic learners will be demotivated by gamification elements. However, this may because of the limited number of gamification elements and the interaction between the neuroticism and other dimensions of personality, such as extraversion. Table 2 shows a summary of the result obtained from applying the cox hazard model.

DISCUSSION
The results confirm the positive effect of gamification in enhancing the motivation and the engagement for overall learners. However, previous research has indicated that gamification elements have a negative effect, particularly in long-term courses [30]. These gamification elements may be annoying and tedious to some learners, whereas other learners might become more excited by these elements and concentrate on collecting points, while losing focus on the course content. This type of learner will tend to stop using the learning system if the game elements are removed [27], [50]. In this experiment, we did not observe any negative affect. Highly conscientious learners do not receive any benefit from gamification, while other personalities, such as highly extrovert and highly neurotic learners, get a significant benefit from gamification. These results may be explained by several factors. First, this work involved only a small number of learners with extreme personalities. Most of the learners in this experiment were in the average personality range. Second, the interaction between personalities may affect the results. This was clearly shown when we tried to measure the correlation between personalities. We found a high correlation in our sample between the neurotic and extroverted personalities, which may explain the positive effect from gamification on neurotic learners. Furthermore, in this experiment, there were a limited number of gamification elements, which reduced the possibility of a negative impact. Displaying points and badges at the top corner of the page does not involve any additional costs to learners, in contrast to other aspects such as social elements. For example, collaborating and competing with others requires that learners invest additional time, which might prompt individuals with a certain personality dimension to drop out. Furthermore, collaboration and competition elements among individuals with extroverted personalities may distract them from the actual goal of the learning system. For neurotic learners who care about others' opinions and are prone to anxiety, such elements will be demotivating and cause them to drop out of the learning system [51]. In contrast, these elements will show a significant positive effect among extroverted learners who care less about their progress than about interacting with others [19].
This work focuses on the learners' motivation and on the length of time the learners use the online learning system. However, the experiment does not consider the performance of the learners. Thus, it may be helpful to return to the school after finishing the experiment to conduct a post-test to measure whether the learners' learning is affected by the gamification elements. Future work should build a solid understanding of the relationship between game elements and learners' personalities. This understanding is required for the construction of a dynamic model that will offer the most effective game elements for specific personality dimensions and that will avoid gamification for other personality dimensions. In addition, future studies should consider including more learners to ensure that the groups contain more learners with more extreme personality traits. Furthermore, social elements should be incorporated since (as previously suggested) they can have a significant impact on learners. For example, learners could be provided with a way to engage in discussions to help each other out or to post their scores on social media in order to introduce some competition.

CONCLUSION
Several researchers have recommended gamification as a technique to enhance learners' motivation and engagement in online learning systems. However, it has been suggested that the use of gamification elements, such as points, badges, and social elements may have a negative effect on some learners, who might find these aspects annoying and tedious. Others may become so preoccupied with the gamification elements that they may lose focus on the course content. In order to overcome these negative effects, previous research has recommended that gamification elements be customised based on learners' personalities [52]. Thus, it is important to examine the relationship between gamification and learners' personalities. Theoretical models point to the various effects of gamification on learners with different personalities [18], [19], [36]. Most of these previous works required participants to use a gamified learning system, after which those who finished the experiment were required to complete a questionnaire to determine the game element they found most motivating. However, failing to consider learners who drop out from the experiment (or forcing completion) could have biased the results. It is important to identify the reasons they withdrew their participation, which may relate to the gamification elements. In addition, using a self-report questionnaire may yield unreliable results. Therefore, to overcome the issues of the previous studies, this research has investigated a new approach to measure learners' motivation. We used the number of learners who dropped out from the experiment as a proxy for motivation and engagement. We hypothesised that learners who are highly motivated would use the learning system for a longer time. The focus of this research is to measure the motivation and the engagement of learners. Therefore, the knowledge gained, and the learners' performance would not be measured.
To do so, we built a learning website to teach Microsoft Excel in two identical versions, one with gamification elements and the other without. Then, we divided 197 learners among the two versions (balancing the number of learners, gender, personality and knowledge). Learners were free to use the website whenever they wanted and drop out from the experiment at any time. We used the learners' drop out rate as a proxy for the learners' motivation and engagement. To analyse our data, we needed a special type of analysis that included learners who stayed and those who dropped out from the experiment. This analysis is called survival analysis. The most common method used with continuous data is the Cox proportional-hazard model.
The results obtained from the cox proportionalhazard model confirmed that learners assigned to the gamified version used the website for longer than learners in the non-gamified version. This means the motivation of learners in the gamified version is much better than learners in the nongamified version. Further, the results point to the various effects of gamification on different personalities. Highly conscientious learners are shown to have little benefit from gamification compared with extroverted and neurotic learners, who have a significant benefit from gamification. This result may be attributed to the small number of learners with extreme values for a specific personality trait. In addition, the interaction between personalities may affect the results. This was clearly shown when we tried to measure the correlation among personalities. We found a high correlation in our sample between the neurotic and extroverted personalities, which may explain the positive effect from gamification on neurotic learners. Moreover, this experiment employed a limited number of gamification elements, namely, points, badges, and leader boards, and did not include any social elements. Social elements are expected to have a significant impact on learners, especially extroverts, who prefer to interact with others, and neurotic learners, who focus excessively on others' opinions.
Thus, further research is required with a larger number of learners to ensure a variety of personalities. In addition, we need to examine the effect of other game elements such as social elements. This can help determine the elements best suited to each personality dimension.