Prototyping Games-based Environments for Learning C++ Programming

An implementation of a design for a game-based virtual learning environment is described, which is then tested using groups of first year students learning C++ programming. The focus of the testing is to determine the viability of a multi-level, multi-player gaming environment as a learning environment, to determine the issues in developing and presenting learning material within the environment, and to try to establish the importance of the level of game play as a motivating factor to enhance learning. Our aim was also to examine the length of time required before students are capable of using the environment for learning, and to what extent they help each other during the game, since MMORPG environments are known for their ability to create online communities, in which mutual help cultures amongst players are emergent.


INTRODUCTION
One challenge facing educators today is that young people in learning situations are not always eager to do difficult things.Two major choices that are regularly then chosen are either to force them or (an alternative temptation when profit is at stake) lower the requirements for a pass mark.However, in the gaming industry neither of these options are viable, people cannot be forced to buy and play the games, and, in general, players do not want the short and easy option.For educators, this raises an interesting question: "How do game designers manage to get new players to learn their games which are often long, complex and difficult, and even pay for the privilege?"An answer provided by Gee [1] is: "The answer, I believe, is this: the designers of many good games have hit on profoundly good methods of getting people to learn and to enjoy learning."He goes on to claim that "Under the right conditions, learning, like sex, is biologically motivating and pleasurable for humans".This may be regarded as an extreme view, but it is an observable fact that at times learning can be a pleasurable experience.The notion of getting students hooked on learning, rather than seeing it as something they have to force themselves through, is every educator's dream.The question that needs answering is:" How do we do it?" In designing games for learning a major concern is to ensure that the game is used and reused by the participants, an aim we have in common with the commercial game producers.From the many studies of learning that have been done, from Rousseau to Piaget, we know the benefits of revisiting the same learning material on a number of occasions, reinforcement, and drill and practice models.The effect of repeated exposure to the same information again and again greatly increases the likelihood that the information is retained and aids in understanding and learning.
Deep learning requires the commitment of time and effort.The gaming industry have invested heavily into getting players to commit the necessary time and effort to master their games, the challenge for educationalists is to tap into the same feelings, getting students to devote the same time and energy to learning as they do to playing games in the evenings.
The purpose of this experiment is to establish if we put learning content into a game, or design a game specifically for a learning purpose, what works best, what are the costs associated for us as educators, and what is the threshold for game play to retain motivation, what works and what does not work?An implementation of a design for a game-based virtual learning environment is described, which is then tested using groups of first year students learning C++ programming.The underpinning pedagogic model utilised in the development of the prototype is the constructivist model, which the authors support, on the basis of a considerable body of research, as the model of choice for independent learning development.The project described here is focused on the creation of a prototype based on a previously published design [2] for a games based virtual learning environment.
The prototype presented here is based on an earlier prototype for presenting methods in Java utilising a method of layering the material on top of an existing game.The Java game is fully documented and described in [3]

MODEL FOR LEARNING
What we know from learning theory is that if we repeatedly expose students to learning content they are more likely to retain information and thereby improve their learning and understanding.What we know from games is that they are motivating and engaging, and we are especially interested in using MMORPG as we are also looking into socialisation and reuse as a fundamental feature of these games.A key concept for game players is that they will keep on trying until they succeed, they will keep on revisiting the same game elements and information again and again to get to the point where they are able to progress.This is dissimilar to the common learning situation where the students tend to shy away from areas where they have encountered failure and / or problems.In the game a failure is seen as a challenge and players will keep on going because they believe that eventually they will be successful.Even after succeeding they may go back and redo to gain greater skill in the problem areas.This key difference in user perspective is a major motivation for using games for learning.
The prototype described here is one of a series of four games that are being developed, looking at 2 main areas, games designed for game play (type 1 and 2) and games designed specifically for learning (type 3 and 4): Type 1.Take an existing game and layer the learning material on top.Existing gameplay elements and outcomes are unaffected.Type 2. Take an existing game and embed the learning material into the game.Make the learning material part of the game, by changing gameplay elements and outcomes to incorporate the material.Type 3. Creating a built-for-purpose game where the game is built around the learning material and nothing else, and the learning material content and context are explicit.Type 4. Creating a built-for-purpose game where the learning material is abstracted out of context and the game is built around these abstracted elements.
What we know from our previous survey [4] is that games designed explicitly for the purpose of learning have been successfully created.However, the effort required in creating such games and the specialist skills required makes it infeasible for an academic with no special games creation background to create such games.It is also the case that even these built-for-purpose games often fail to engage or fail in other aspects, but the most important issue is that they are also non-reusable.By this we mean that a game built entirely around a body of learning material cannot be easily changed to accommodate an alternative body of learning material, in fact to accommodate new learning material a whole new game has to be developed.Therefore, in looking at games for learning it is important to consider the opportunity to reuse the game elements and easily adapt them to new learning material.Through reuse we can keep the cost of creating games for learning down, while also engendering some confidence that a game that has successfully engaged students in one learning context is more likely to be successful when reused for another.
As we say above, we know designed-for-purpose games can be successful, although they are expensive, but even many of these expensive developments fail because successfully designing game play is not an exact science.The costs associated with games are enormous, the current industry standard for a major platform game is development teams of 60-120 people, a development time of 2 years, and a cost of 20-30 million pounds.Therefore, designing games specifically for purpose where the game play is built around the learning content is not cost effective, unless it is going to be used for a very large group of students on a worldwide basis.
The alternative to design-for-purpose is to utilise existing games with successful game play, the games having been designed with game play in mind and pre-tested to confirm the game play is successful before utilising the game in a learning context.If this approach is successful, the opportunity exists to develop a suite of pre-tested games that have successful game play, and then embed a variety of learning content into these games.We are investigating two techniques for adding learning content to a game: layering the learning content onto the game, with little or no relationship with the game play; and embedding the learning content into the game, where there is an explicit relationship between the learning material and some aspects of the game play and game activity.Clearly layering is relatively easy, and less expensive, to do, whilst embedding is both more complex and more expensive.The experiment described in this paper is a game with embedded learning content, where the game was predesigned with game play in mind, and the learning content was embedded once the game was complete.

Design of the learning game
The design for the virtual learning environment is based on the utilisation of a multi-level, multi-player games based model and its inherent support for constructivist learning in a higher education environment.
In addition to the rationale already described, there are several reasons for the choice of a environment based on computer games: generally computer games require players to take an active role in the activities performed; games are built to offer a freedom for the player to explore the environment and material; the gaming format also lends itself to support the known advantages of narrative from oral traditions; and the model fits with a younger age demographic, and their interest in current trends in the entertainment industry.
The decision to adopt a game-based approach to the development of e-learning follows on from a survey of current state-of-the art support for lifelong learning that was carried out by the authors [4], a key element of that investigation being the different techniques to improve learning and retention through engagement of the learner.The decision to adopt a games-based approach is also based on several studies that have shown the significance of using computer games in education.In particular, Henderson [5] published a study showing that playing a recreational video game provided beneficial informal educative experiences, even when used only over a short period of time.This study provides support for the cognitive worth of playing games, and it also contradicts the popular opinion by confirming that recreational video games are a significant cognitive artefact of youth culture.Finally, it also demonstrated the viability of stimulated recall as a methodological tool in game research.

Active learners and deep knowledge
In designing games for learning a major concern is to ensure that the game is used and reused by the participants.As identified earlier, the effect of repeated exposure to learning content will achieve the desired effect, increased retention and improvements in understanding and learning.This experiment therefore does not attempt to test the learning outcomes of the students to conclude that learning takes place, but rather to demonstrate that the approach increases the exposure of the participants to the learning material and activities with that material.There may be a desire to test learning outcomes later to establish how effective the different techniques are, but that is not the focus in this experiment.
Thus the purpose of this experiment is to establish: The theoretical argument for constructivism is that deep knowledge and long-lasting knowledge is more likely to arise from constructivist learning environments.The perceived benefits of a constructivist learning environment include absorption and synthesis of facts, linking the knowledge of facts with understanding of other knowledge domains, the enhancement of collaborative/cooperative skills and time.One of the key points of constructivism is to get the students involved in the process, and give the tutor the role of facilitating and supporting learning.
Experience has shown that most students will, after some initial misgivings, elect to follow the active learning route and actively be involved in the learning process.The rationale for this is that it is simply more enjoyable.
Students learn more, and enjoy themselves more when they are actively involved, rather than just passive listeners.The presentation of material in books, papers, multimedia presentation, traditional lectures or web can all encourage the student to take an active role, use the lab for assignments or even just imagine the activities.However there are few forms of learning that, like games, entice, encourage and even force the student into activity.Computer games are built to activate and keep a player active with problems and tasks, through the whole period of play.An important aim here is that, in addition to keeping the player active, they also ensure the tasks consistently offer a challenging degree of difficulty.
Csikszentmihalyi [6] introduced the concept of flow, through a study of people involved in activities such as rock climbing, chess and dance in 1975.He describes flow as a state of complete absorption or engagement in an activity and refers to the optimal experience.According to flow theory, flow can occur when an activity challenges an individual enough to encourage playful, exploratory behaviours, without the activity being beyond the individual's reach.Flow has been shown to have a positive impact on learning, see Galarneau and Kiili [7,8].
Many people are advocating taking the hard work and discipline out of learning.However, we would argue that this is not the answer.Rather, what we should be doing is finding tasks that will harness the passion of the student to the hard work needed to master difficult material.
Therefore, one main aim for us as designers of learning material is to design content in such a way as to allow different students to obtain the state of flow, irrespective of their different knowledge and abilities.

SOCIAL RELATIONS IN GAMES
Social relations between students during learning are an integral part of the authors' philosophy for learning.Manninen [9] has shown that the communicative aspect of current multiplayer games is enabled by a relatively limited set of interaction forms.The communication between players in the prototypes described in this paper is mainly via IRC.But the avatars do also have a limited set of emotions they can express within the environment.
Studies by Kolo and Baur [10] have shown that many players not only connect to a online game in order to play but also to stay in contact with the fellow players, many players also connect to fellow players via messaging systems during game play.They engage via their characters in various social interactions from trading or fighting to entertaining other characters.Many players regularly meet the same characters online and address a relatively fixed group of playing partners.Kolo and Baur [10] have also shown that knowing and meeting people in an online environment or game triggers frequent playing and not the other way round.
Students meeting face to face after a game session will continue to discuss the stories that have unfolded in the game environment, and they will naturally continue to discuss other topics that were discussed during game play.This offers the opportunity both to enhance and extend the learning opportunity offered within the game through social interaction, and to address a key concept of constuctivism, that learning is a social activity.More information on this topic can be found in the previously published design for a learning environment by the authors.[2]

ASSESSMENT IN A GAMES BASED VIRTUAL LEARNING ENVIRONMENT
The focus of assessment has traditionally been purely on the final exam, resulting in smart students spending a lot of effort on trying to guess the exam questions and thereby making this the focus for the learning rather than the specified course content.Students will follow this approach if the assessment tasks let them, so their focus will often be to get by with low-level learning strategies.
A learning system based on a constructivist model gives different guidelines for the development of assessment.Firstly the assessment must be designed into the system so that it is an integral part of the learning process.The importance of integrating the assessment, rather than simply adding it at the end, is well documented, see [11].Additionally the assessment must focus on verifying that the desired learning has taken place, and not check that the student has learned only the desired facts.The use of a gaming system in education is not enough, the educators need to change the way they think about assessment, as Gee [1] notes: "Game-like learning cannot be incorporated into our schools unless those schools make radical changes.As long as they stay skill-and-drill testing centers preparing most kids with "basic skills" for service jobs, while allowing a few rich kids, with no real ability, to apply their knowledge to get A's for passing factbased tests, they will resist such learning.Worse, they will co-opt any attempts to use such learning into their own agenda.Jeopardy and Trivial Pursuit will be the games of choice in school."

HCI Educators 2009
Playing with our Education

DEVELOPMENT OF THE PROTOTYPE
In our design for a game-based learning environment the subject area is divided in topics and subtopics, which are then modelled as levels within the game.The different levels will contain multiple quests each representing some areas of content, or topics, that the student should learn.These quests will model the learning material.The quest form is borrowed from existing computer games formats, see [12].The quest format is built on the simple principle of setting some task for the player and then rewarding them on the successful completion of that task.If the player does not succeed the option is to go back and try again.Successful completion of each of these quests will provide the students with conceptual information relative to the topic of study.The aim of the game is to complete all the levels.Progression in the game is controlled via a separate part of the system dedicated to assessment, which has been previously been published by the authors in [13].
Constructivism promotes social and communication skills by creating a learning environment that emphasises collaboration and exchange of ideas.This social community is naturally present in lectures and tutorials.The authors view is not that the only learning that can occur is face to face, but rather that social activities are an important and significant part of the process.One of the challenges in online learning is to create this feeling of belonging to a community.To encourage and facilitate this important social aspect of learning and encourage students to assist each other, the players helping others in the game are to be rewarded according to rules built into the game.
During a quest the player can be given any amount of help, the aim of the game is to learn, not to determine if the students are capable of learning on their own.The outcome of quests does not influence the judgement of the outcome of the assessment process, but it may be used as a means to control access to the assessment.This separation of assessment and quests, and the summative and formative assessment, is in order to allow the student to receive any help necessary from the game environment, fellow students or instructors during game-play.
The use of online games setups as a basis for a learning environment is not new.There have been numerous trials of this in the form of simulations, MUDs and adventure games [14,15] During the last few years multiple virtual worlds have been developed for general communication between users from different parts of the world.Some of these are environments where people can meet and chat, while other environments give the users the possibility to contribute and create worlds and virtual objects by themselves.
The prototype developed for the experiment described in this paper is an online multi-user game where all the different players are present in the same virtual environment.The prototype is based on the Torque game engine from Garage Games 1 , in addition we have utilised the Torque MMO Kit 2 initially created by Prairie Games Inc, but now maintained and developed by a group of users under an open source licence.The Torque MMO Kit requires several servers, SVN, IRQ and HTTP, C++ and Python development environments in addition to the Torque 1 Garage Games Inc, http://www.garagegames.com/.
games engine and its suite of tools.The inclusion of tools for creating jpeg and 3D-models are also useful for creating artwork that fits with the theme of the environment.We did not opt for creating our own models, just altering models that were available to us.The effort required for creating artwork, both 2d and 3d, is substantial and was not a viable option for us in the scope of this prototype.
The effort going into this prototype is focused on embedding the learning material within the quests, not just layering the learning material on top, as was the case with a previous prototype described in [3].

PROTOTYPE ENVIRONMENT
The main question we sought to answer was: Is a games-based environment a viable way to present learning material, and what level of embedding of that material is necessary within the game?Particularly, we were interested to investigate what effect different levels of embedding of the learning material within the game context had on the player's perceptions of game-play and learning.
The aim of the prototypes was also to gain information on: The prototype was created specifically to be run as an addition to a first year course for Computer games students in C++ programming at University of Abertay in Dundee, Scotland The lecturer for the module had expressed an interest in new ways of presenting material and giving the student the possibility to assess their own progression, and an extra opportunity to add content to their portfolio.The same prototype was also made available to students in Norway.
The students testing the prototype were a mix of students from University of Abertay in Dundee, Scotland and Buskerud University College, Norway.Both Universities offered lab facilities where the game-based VLE client was installed, and in addition all the students had the option to download the client to their own laptops or home computers.A high percentage of the Norwegian students opted for the latter option.The same server was used for all the trials, located in Norway.

Design of the environment
The physical appearance of the environment used for the C++ was not the main issue and it was decided not to make it overly complex and just focus on a limited playing area and design a suitable range of quests.The virtual environment for C++ is built inside a valley and is made up of three levels: The initial level is the grassy floor of the valley; a plateau is level two; and finally a great building in the middle of the valley is level three.The whole environment is populated by around 120 NPCs (Non-Playing Characters), nearly all of them friendly, and in this environment there are no weapons or armour.Learning from the trials of the first Java prototype [3] where the students complained that too much time was taken up by moving, we positioned the NPCs so that the distance needed to move was smaller, and, in addition, we increased the movement speed of the player avatar.
The playing area is divided into several minor areas, and the students were free to move at will all over the playing area, so these subdivisions were created to aid the students in navigation.
A map of the environment is shown in figure 1.The minor areas shown on this map are: Starting point (figure 2), Village, two farms, an Aztec pyramid, a single farm, a plateau which is level two, and the great hall which is level three.Within each of these areas there was one quest, requiring the students to prove their knowledge in C++.The knowledge students are required to demonstrate may be obtained from helper NPCs within the game environment, or from any other source of their choice.Within each area there are a number of non player characters (NPCs).These NPCs can all be interacted with, and come in the three categories: helpers, quest givers and quizzers.
Helpers are NPCs that will help students move through the environment, they have implemented a simple adaptive frequently asked questions (FAQ) facility and will attempt to tailor their help to where the student are in the progression through the quests.
Figure 3: starting on a quest Quizzers are NPCs that have question on themes related to different areas within the C++ programming language.These questions are in a multiple choice format.When interacting with the NPC it will present the question, as seen in figure 8, the student will only have the possibility to attempt each question once.If the student turns away or chooses not answer, they have the option of coming back and answer another time.A correct answer is rewarded by an item.If a question is answered incorrectly the question is no longer available.The items are all tokens demonstrating knowledge within certain areas of C++ programming.
The C++ environment was also created without the traditional MMORPG elements of fighting, armour and weapons.The aim of the C++ environment is to progress through the quests and be rewarded by gaining knowledge and money.The players finally spend the money on t-shirts and other items at the merchant, and can show these off to the other players.The different forms of interaction available in the prototype are multiple IRC channels for communication and trading for exchange of items.

Learning content in the C++ environment
The C++ environment is focused around completing different quests.Feedback from the first Java prototype was that the student swanted quests that are directly linked to knowledge of programming.The quests for level one are connected to expressions, loops and logic choice (if) in C++.Students must demonstrate the required knowledge in a topic area by finding NPCs and answering their questions in the same way as the Java version.The difference from the first prototype is that when working on loops the student will try to find NPCs with questions linked to loops, as there will be NPCs available locally to help them with knowledge on loops, and the replies from the NPCs to an incorrect answer are meaningful and designed to make the student understand why their answer was wrong.The rewards for answering questions correctly are tokens, which can be used to prove your knowledge in particular areas.When a required number of these tokens has been collected they can be exchanged for a proof of knowledge in an area, 5 loop tokens are exchanged for a monetary reward and the "key of loops".Collection of all three keys from level one is required to gain access to level two.
In addition to the keys that are collected during the quests, the student must also collect identifier tokens.These identifier tokens are later used to name variables, parameters, functions, methods and classes.
Level two has two new keys that the students must acquire: I/O and functions.The key of I/O follows the same pattern as the previous keys, but functions require the creation of function headings from a description.The same goes for classes in level three.Students are required to craft a heading for a function that takes a specified set of parameters, e.g.create a function that prints a specified string a specified number of times.The student is required to first create a String parameter, an integer parameter, and then combine them with a function identifier to create the function heading, and then give the created item to the NPC that gave the quest, to complete the task.
Completion of all quests is required to access the final assessment.The final assessment of the environment is to create a rather complex class for storing transmitted messages and receive feedback on it by a tutor.The marking of this final assessment was completed off-line manually.After the student had produced a solution to the final assessment it was sent by the environment via e-mail to a tutor for marking, although there was also an option for the student to send it directly, if they included their username and sent it from the e-mail registered in the environment.
As in the first prototype, the knowledge that is required to successfully answer to the questions, both those of the NPCs and the final assessment could be found within the environment, but could equally well be obtained by any other source, text book, internet, fellow student online or offline.

Trials
A group of about 45 students in the first year computer games development course at University of Abertay Dundee(UAD), and 60 students in a first year Engineering course at Buskerud University College(HiBu), doing a module on C++ programming were offered the possibility to test out the environment.The environment was introduced to the students in a standard 2 hour tutorial.The students were given a 10 minutes introduction to the environment and how to navigate through it.The students were also given a pre-questionnaire that took between 1 and 5 minutes to fill in.For the rest of the two hour slot the students were allowed to try out the environment.Both the lecturer of the C++ module and the developer of the environment were present to answer any questions the students had.
A total of 39 students showed up for the tutorial at UAD and 26 students at HiBu, and they all filled in the pre-questionnaire.The UAD students played the game during the tutorial, none of the students left during the trial and most seemed to enjoy themselves.The HiBu students had other assignments and were encouraged to download the game at home.From observing the students we noted that not all the students utilised the environment as planned, many seemed very familiar with MMORPG style games and did their best to break it.
At the completion of the module, some weeks after the initial session, all the students in the group were e-mailed a questionnaire and kindly asked to answer it, several reminders later only a total of 11 UAD and 2 HiBu students handed in the questionnaire.

Results
The questionnaire used for the C++ students in Scotland and Norway was the same, except the Norwegian students were given a Norwegian translated version.A selection of the results are reproduced here.
In summary, the results show us: Scottish students rank their knowledge on C++ as average, they enjoy programming, enjoy computer games even more, and have a limited experience with computer games designed for learning (figure 13).Norwegians students (blue) rate their knowledge on programming below average, they do not enjoy programming particularly, although they have experience with computer games they do not enjoy them as much as their Scottish colleagues (in red), in particular their experience with MMORPGs is low (figure 6).We also gathered information on when the students logged in to the environment, so that we were not entirely dependent on the information from the questionnaires.The usage statistics show that only a small number of the Scottish students used the environment again after the initial two-hour session.Of the Norwegian students there were about 10 that actually used the environment and they all logged in over a period of two to three weeks on numerous occasions.

CONCLUSION
The prototype was developed to provide an engaging game environment, in which we could investigate the value of such environments for learning, and particularly consider the impact of different levels of embedding of learning material within the game-play itself.
We have been considering placing learning material within games, so that the games environment becomes the vehicle for delivering the learning material.We have investigated several ways to create games for learning: layering on top [3]; embedding (presented here); creating the game from scratch with just the learning material and no other game content [16]; and developing a game where the learning material is abstracted from the original context and the game is based on the abstracted elements.
The findings from this experiment indicate that the students liked the game, the game context, and found it a useful way to present learning material.The students claimed they found the game engaging, and we did observe a small amount of reuse, not as much as we would like, but some students did return to the game to play on.This is encouraging but demonstrates that there is a need to extend the game further to determine whether or not embedding really can generate the level of reuse we require to demonstrate the effectiveness of the approach.We need to determine whether the game was not engaging enough, or if there was not enough content to keep the students returning to the game.
In our previous experiment we looked at layering and found that with layering there was practically no reuse.We have looked into designing games specifically for purpose and found some reuse in those, but the problem with that approach is both cost and the requirement for expert knowledge, and we believe these eliminate this approach from consideration.
The evidence we have from this experiment is that embedding learning content into a predesigned game does achieve both the engagement we desire as well as some elements of reuse.We have not yet seen evidence of socialisation, which we would wish

HCI Educators 2009 Playing with our Education
to see to demonstrate learning in a social context.Extending this game further will allow us to conduct a further experiment with larger number of students to address both reuse and socialisation.
What we can conclude from this is that the potential exists, if we can demonstrate conclusively that this approach will result in the requisite levels of reuse and socialisation.We would then be in a position to utilise an embedding methodology to place learning content into pre-designed games, and could then invite games experts to design a suite of games of different types for different target audiences.We could also use the methodological approach we have developed in our experimental models to design an authoring tool to embed learning content into those pre-designed games, and evaluate its use by a range of academic staff.

Figure 1 :
Figure 1: Map of the playing area

Figure 5 :
Figure 5: Progressing to the next quest

Figure 6 :
Figure 6: Previous experience with MMORPG.The post questionnaire was only received from 11 students in Scotland and 2 students in Norway.The questions and results of the post questionnaire for all 13 students are combined.The students were asked how much time they spent learning the environment before they could use it.The results for the students are shown below in Figure7.

Figure 7 :Figure 8 :Figure 9 :Figure 10 :
Figure 7: Time spent learning the environment If we put learning content into a game, or design a game for learning purpose, what works best, what are the costs associated for us as educators, and what is the threshold on game play, what works and what does not work?