Rehabilitating Experience : Designing an Aesthetic and Movement-based Game for Physical Therapy

The quality of aesthetic and movement experience is important to a physical therapy patient’s motivation for recovery. The physical therapy patient’s experience of moving is often restricted to their personal recovery plan; however, we are interested in whether connecting to a personal expressive experience and use of external motivation can support the quality of the recovery experience. We ask the question, ‘how can interactive art games support a patient’s recovery process by focusing on quality of movement performance, enjoyment and engagement?’ The recent rise of gameplay with virtual and augmented reality, high resolution graphics and believable characters is now easily available to the consumer. A variety of games designed to support therapeutic environments and rehabilitation exist, yet the area of movement and aesthetic experience is under researched. In our investigation of colour psychology, frameworks for movement quality, and implementation of gaming elements in therapeutic environments, we explore trends as to what succeeds or doesn’t to make an engaging and productive therapy game. Supernova is an exploratory project to investigate the intersection of motivation in physical therapy, experience-focused design, and digital interaction.


INTRODUCTION
Physical Therapy achieves better movement goals (often from injury) by using exercises that are targeted to a specific anatomical action.While patients are personally engaged in their recovery process, there is often a lack of external motivation to support their experience.Interactive technologies such as virtual reality have been explored in the domain of physical therapy, however these studies continue to be few and lack exploration in aesthetic experience.In therapy, we believe the patient's mindset must be handled just as attentively as their impairment.If the patient is in pain, not motivated, or even a combination of the two, the result of their therapy will not be as successful.
"The reason why embodied gaming works has such a wow-factor is because it plays into our tendency to perceive the world as something with which to be engaged.Using the controller of a Rift or Vive means becoming that much more immersed in a reality that invites you to act in it" (Peeters 2016, p.1). Peeters' quote indicates that incorporating the body's agency in game design will have a much stronger impact on playing in virtual worlds.This level of agency can support the meaningful use of play in many other areas, including physical therapy.This level of agency also provides an opportunity to add principles of visual art, such as color theory, to create environments that subliminally trigger emotions and mental states to not only make a believable space, but help patients get over mental blocks in their recovery.Small visual cues that artists use to convey emotion are an aspect of meaningful gameplay are often overlooked due to a focus on creating and fine tuning the technology.
The quality of a physical therapy patient's holistic experience, and particularly their movement experience can also be imperative to their recovery.While the experience of moving in traditional therapy is often restricted to the physical actions determined for the patient's personal needs, the addition of external motivation such as interactive movement games can support the quality of the recovery experience.We ask the question, 'how can aesthetic and interactive art games support a patient's recovery process by focusing on quality of colour theory, movement performance, enjoyment and engagement?'This project highlights two gaps in current game and health research to support patient experience: 1) there is a lack of tailor-made games that support the patient's experience of moving and 2) there is a lack of qualitative data around patient experience to identify parameters of motivation and engagement.This paper is a survey of research in interactive technology and therapeutic situations and describes our game prototype for upper limb rehabilitation.

MOTIVATION
One of the authors was involved in a horseback riding accident as a child that resulted in many surgeries and physical therapy, which has continued into their adult life with no end in sight.The personal experience of trauma through surgeries, and then again in the structure of their physical therapy plan, developed an aversion to try to regain full results early in the process that has made commitment to rehabilitation even more difficult as an adult.Similar experiences were voiced by many acquaintances and peers when discussed in professional and academic environments that prompted an inquiry to how the rehabilitation process could be more fun, individualised and motivating.
To begin investigating this approach to rehabilitation we approached a professional physical therapist and discussed the challenges and rewards that many patients experience during their therapy.The therapist stated that motivation to do their exercises and get better results varied by person, but that home maintenance was especially difficult.Children often have difficulty engaging in set 'exercises', so he became used to making up games on the spot such as 'don't step on the alligators' when walking on a balance beam.However, they did not have games or alternative approaches to the therapy regime pre-designed.The therapist was most interested in patients regaining their daily functionality, exercises are tied directly to daily actions (examples given include: twisting around to give a child snack in the back seat; reaching the arm high to get something off a shelf; the ability to sit on a toilet; the ability to walk with stability and balance).Within this functionality there is minimal attention given to the exploration or performance of the exercises.Controlled range of motion metrics and satisfactory task completion are used as markers in place of more qualitative aspects of speed, grace, smoothness, or dynamics of movement.
Historically, Irmgard Bartenieff studied with Rudolf Laban to learn and implement the Laban Movement Analysis system in her physical therapy practice to rehabilitate polio patients by activating their pelvis use through movement and developing patterns of body connectivity (Bartenieff & Lewis 1980).While some physical therapists may provide minimal attention to the quality of the movement being performed in exercises, we believe that this will be a motivational factor that has potential to contribute to the proper execution of exercises as well.

BACKGROUND ON THE EXPERIENCES OF EXISTING SYSTEMS
While there have been prior projects that explore the use of digital games in therapeutic situations, the findings show promise yet have been limited to available technology.Based on research done with stroke victims, burn patients, and children with decreased motor function participating in VR therapeutic exercises (Wille, et al. 2009), there are research areas that can be explored more deeply such as improved graphics, use of colour theory and movement quality.We have compared a variety of games designed for therapeutic uses and describe the systems in more detail in the following paragraphs (see Table 1).Merians, et al. (2002) studied stroke victims using VR to improve their hand dexterity, using three patients recovering from a stroke they had three to five years before.Each patient went through approximately 90 minutes of these VR exercises five days a week for two weeks.Schneider and Hood (2007) studied the psychological effects that VR-based environments had on patients going through chemotherapy.The study found that patients who use the VR experienced a perceptively shorter chemotherapy session due to the distraction of being in a different place virtually.Robillard, et. al. (2003), found immersing people with phobias into virtual situations that made them confront their fear without actually confronting their fear for logistical reasons was very effective (p.468).The environment they were immersed in was able to be made for phobias that would be difficult to be exposed to in the real world and completely controlled for the sake of the patient.

Existing Therapy Based Games
There are two examples of commercial games used in a therapeutic or medical setting.The Woodland Wiggle installation by Chris O'Shea (O'Shea 2013) located at the Royal London Hospital provides the children in the hospital a fun distraction using motion capture and interactive media.The GestureTek company has created a system like the Xbox Kinect setup for patients to play games such as bowling, drumming, or playing a soccer goalie.While the technical setup works well, the graphics appear dated and seem to take away a certain enjoyable aspect to the game.

Colour Theory and Aesthetics
Colour theory, specifically colour psychology, is the broad study of colour relationships and the role they can play in affecting the mind.One of the most easily recognised uses of colour theory is the creation of logos.Depending on the company and what they are selling, the graphic designer making the logo will skillfully use colours that trigger an emotion response from the viewer to associate the company with a certain mood or feeling.Looking at the chart we have created using popular colour psychology in logo design that stems from the study of colour theory that early twentieth century painter Wassily Kandinsky did (Figure 1), many companies that involve nature will use mainly green within their logos.Companies that want to build trust with their customers (financial institutions, car manufacturers, computer companies) will use the colour blue because it gives the sense of dependability and strength to the consumer.1997), needing just the right amount of continual challenge to maintain engagement without becoming overwhelming.We are using the game "Flow" designed by Jenova Chen (who was inspired by Csikszentmihalyi) as our inspiration for an engagement goal in Supernova.

Current PT Procedures for Children
Our target audience is children and young adults, to emphasise the playful elements that we find unique to gameplay.However, we have heard anecdotally from many adults and senior citizens that they wish they had a more playful way to do their physical therapy, because it would help motivation beyond 'getting better'.While we are currently designing for children we are creating our movement databases with many shapes and sizes of people moving so that our games are accessible to many populations.
We work with a professional physical therapist whose clients tend to be members of the public who had a minor injury that they are recovering from.This includes children, young adults, adults and senior citizens who need to regain daily functionality of their body for basic tasks.While some athletes come through this practice that may require more elaborate regimes, most patients are doing basic exercises to regain strength and flexibility after an injury.This practice does not work with patients who have had serious or debilitating injuries.
Our physical therapist illustrated many exercises for us, which all have a functional goal such as being able to twist to hand a toy to a child in the back seat.Speed, force or more qualitative aspects of movement are not explored in this scenario.All patients do repetitions of their exercises at the studio with the physical therapist, and are also required to do their exercises at home on their own.Patients may add weights to their limbs to make the performance of their exercises more difficult.
Games are often made up on the spot to help children engage in their exercises, such as telling children not to step on the alligators when walking on a balance beam.However, games are not devised ahead of time or developed for specific exercises which could allow for more complex engagement by patients.
In collaboration with our physical therapist, we have chosen four arm actions that are used to manoeuvre through Supernova.These actions include: flick/ swat, raised extension/flexion, squatting and press/push (see Table 2).

How a PT Session is Run with Supernova
We are designing the Supernova game to be both a studio session with your physical therapist as well as an at-home accessible game.This enables the physical therapist to give instruction and feedback on the patient's performance as well as to create a personal narrative for the patient, related to their injury and/or personality, that will support the motivational factors in Supernova at home.Our mixed-reality approach to therapy is supportive to the patient's motivation while continuing to be based in the physical needs of the patient.
The patient will turn on their device for playing the game (Xbox with Kinect camera or laptop with Kinect camera), begin the game application, and will see the space environment.The patient can choose whether to perform all the actions simultaneously in the game, or to work on them one at a time.If simultaneously, the patient is constantly moving forward through the space environment, flicking asteroids out of the way, ducking underneath planets and throwing stars at objects.The patient can slow down time by pressing forward.To win, the patient needs to have performed so many actions (as determined by their recovery plan).Rewards are given as twinkling rocket ship animations.To 'level up' a patient adds physical weights to their arms, making the completion of the game more difficult.
These actions have been chosen as beginning actions because of their prevalence in cases seen by our physical therapist.We plan to create more varied exercise scenarios in future versions, that can be tailored to the individual patient's needs.

TECHNICAL SETUP AND AESTHETICS
Our setup uses a single Microsoft Kinect to capture the player's movements when they are standing within the predetermined record space.The 3D space world is built in Unity and projected on the wall in front of the player, reflecting the impact of their actions on the environment.

Movement Interaction
Patient's movement is tracked by using a Kinect camera and the GestureBuilder software as part of the Microsoft SDK.GestureBuilder is used to create a detection solution that is built from training data, which is then implemented in Unity to track movement.Physical therapy is mostly focused on the correct functional performance of a movement, however more performative qualities are of interest from anecdotal discussions with patients who see movement quality as important to motivation.The pursuit of detection by movement quality has been explored (see Section 3: Background) yet is limited.
We are exploring movement quality by using the Laban Movement Analysis framework, which is excellent for deconstructing movement into a set of 'parameters'.In prior work, we have found the LMA system to hold more opportunities than other frameworks (such as Benesh) because of its explicit nature that is useful for camera detection.
While we are working on the detection of LMA movement qualities separate from the body action with this system and hope to implement it in the future, we are currently using a 'Wizard of Oz' method to allude to the performative quality of motion.We do this by focusing the actual detection on one of the three elements of a movement quality (space, time, weight) while instructing the player to attend to all the required elements (this way the feedback is close to the preferred quality).We have implemented three different qualities of action for upper arm exercises that take place: flick, press and float.These qualities are layered onto body actions of swatting something away, pressing the environment away, floating an arm up to grab an object and a fourth option to 'float' while doing a squat to duck under something in a sneaky manner.Flick' gestures are quick, light and indirect, which we track by calculating whether the movement is fast and changes direction frequently.'Float' gestures are sustained, light and indirect, which we track by calculating whether the movement is slow and changes direction frequently.'Press' gestures are sustained, strong and direct, which we track by calculating whether the movement is slow and does not change direction frequently.

Space Environment
Space is considered to be the 'final frontier', an expansive and endless place of exploration and wonder.We decided to use a space environment for this specific game because it provides a curious and open playground within which leveraging fantasies of science fiction for engagement.In the photos of nebulas taken by telescopic satellites, colourful gas clouds form striking shapes that spark the imagination to associate them to objects like butterflies and horses.Because these clouds are invisible in reality, scientists assign colours to each element found in the clouds.Since they are initially a blank slate, assigning specific colour palettes to artist created nebula clouds can give the image its own emotion and personality.We first used Photoshop brushes based on forms found in photographs taken of space to create our own nebula images with varying emotional qualities.We used aspects of Kandinsky's colour scale to design our own peaceful and energetic (Figure 2) versions of space environments.Two dimensional images were created with many layers, which are extracted and made transparent.We then used these hand-created images to manipulate the colouring in a space environment in Unity.Colours were applied by sorting each type of celestial body (cloud, galaxy, etc.) into groups based on colour.Then we control which mood is dominated the landscape by varying the size, quantity and colour families of each visual element.The aesthetic is crafted by making choices about relationships between colours.While colour theory suggests that cool colours give a sense of relaxation and peace, and saturated warm colours fire you up, we found that a 70% warm to 30% cool ratio creates a more realistic and visually appealing combination of colour.If warm is used 100% without any cool contrast it is overwhelming and becomes hostile (see Figure 3).Even in realistic images, snow covered landscapes include a tint of warmth for contrast.

Gameplay
Supernova challenges players to use the correct quality of movement as they complete specific tasks in space.If playing the game with exercises one at a time, the player can lean to one side to advance to the next exercise.Players are required to swat away asteroids by using quick, light and indirect 'flick' gestures as they are flying through space.Sustained, light and indirect 'float' gestures are used when squatting to duck underneath a passing planet, as well as to reach behind the head to grab and throw a star at an oncoming object.Sustained, strong and direct 'press' gestures are used to slow down time.The player is continuously moving forward in space and navigating objects as they encounter them.When the player wins a scene, they are rewarded with a fun visual animation of a rocket ships and stars.This experience of attending to movement quality in an interactive game challenges patients to think differently about the way they move, and the level of control and responsibility they have with respect to their own bodies in recovery.

EXPLORATORY PILOT STUDY
To test our exploratory phase, we performed a small pilot study with five young adults (18-22 years old) as a first step before deploying on actual patients.The participants included one male and four female university dance students who have all had some experience with physical therapy.Participants each had 10 minutes of gameplay exploring the different exercises and then participated in a focus group to discuss their experience.
From the researcher's observation, each participant took a different movement approach to the game.Some used very effortful, expressive gestures while some only focused on the correct execution of an exercise.Some tried to be very unmotivated while playing.While this might seem to pose a challenge to a game based on movement aesthetic, we found it very useful to consider new game elements that would affect the interaction.For example, having varying speeds so the patient is constantly shifting how they approach the movement.
Participant comments relating to motivation in the focus group include p5 stating that: "This is great distraction.I had two shoulder injuries that would keep me awake.Even now, my upper body is not there [strong enough] and I'm scared to go there.But if the approach [to physical therapy] was made in a more appealing manner I wouldn't curl into a ball at the thought of using my upper body now."Creating a social atmosphere was also noted as a motivating factor, because "it would push everyone to do more" (p2).
Other comments included discussing whether it was important to do the exercises separate, or whether they could all exist together in a game, such as p2 "it would be nice to move in more directions".Also, p1 "doing repetitions gets boring".However, p3 reminded the group that "sometimes people just want to be told what to do", and not have a lot of extra pieces to the task.
The overall aesthetic was discussed as "tranquil but distracting" (p5).The sound, colours and expected quality of movement worked together and it was noted that "it's interesting enough that it is distracting you from the exercise you are doing but not so interesting that you aren't thinking about how you do the movement" (p4).
These findings suggest that we are making some successful choices in the game design, but there are many aspects of the gameplay that can be tweaked.The responsiveness is not always as quick and smooth as is desired for movement feedback.
Manipulating the timing to shift may help with the perception of responsiveness.Also, an action does not need to be executed perfectly to get the points, though it does need to be close.Thus, the game in its current state will not be as supportive as an analysis tool regarding a patient's perfect execution of exercises.However, there may be alternative design criteria that can support proper execution.
Future studies include working with physical therapy patients who are children through a local clinic to explore how the current gameplay and design choices work for a younger target audience.

DISCUSSION AND CONCLUSION
This paper has focused on illustrating the design considerations present when designing aesthetic and movement based games for physical therapy patients.We have designed a game titled Supernova that challenges physical therapy patients to perform exercises while navigating through a space environment.This game leverages concepts of colour theory and quality of movement in the game design to support engagement and motivation for personal recovery.
Through a survey of game design for therapeutic applications we believe there is use for the design of serious and motivational games to support physical therapy in particular, though also in many therapeutic situations.While Supernova is still in an exploratory stage, the findings from our initial pilot study suggest that colour theory and performance of movement quality are important factors in personal motivation that are worth deeper investigation.This preliminary presentation of research, design goals and prototype explorations provides a foundation to further work in the design and deployment of a game to support rehabilitation.
One of the goals we hope to accomplish is to make an engaging and entertaining immersive therapy experience.This contribution will support research in patient's rehabilitation experiences and can further our understanding of what creates enchanting experiences.While we are at the beginning stages of this project we feel that it is important to emphasise the challenges present in the domain of physical therapy games and movement practices.
While there appear to be many advantages to a rehabilitation-focused game, there are also disadvantages.The recovery process is a very personal process, based on an individual's medical needs, motivation and personal interests.While we will never be able to design for all possible options we can target patients for specific types of injury and can build a variety of options that can support individual adjustments.A game could also complicate the rehabilitation process, by adding in more variables to attend to than needed or take more time than necessary.The design of the game will also not be able to appeal to all audiences, and may turn off patients to their rehabilitation plan.
Properly tracking exercises can also be difficult if the patient is wearing a fabric that is not seen by the camera or the calibration of the system is off.While these are all possible challenges we are exploring ways to mitigate them in future versions.We are also aware that this gamified system of addressing recovery will not be suitable for everyone.
The design and investigation of Supernova will contribute new understandings of the impact of various aesthetic qualities on engagement and gameplay experiences.Future work includes using the Perception Neuron inertia-based motion capture system to have even more detailed tracking of movement, and the Oculus Rift for less mobile exercises to provide a more immersive experience for patients in heightened injury scenarios.Future scenario ideas include popping bubbles, swinging on vines, wading through a swamp and bowling as inspirations dependent on exercises chosen.
Additionally, in future research, we anticipate discovering that certain types of therapy could only work with one type of immersion.For instance, virtual reality might only work for pain relief or phobia treatment because patients with physical balance issues dealing with a complete immersive experience using a headset and hand equipment.
In the same vein, an AR experience may not be successful for phobia and pain treatment because it not immersive enough.These theoretical situations are only preliminary examples of problems we may face when making our environments.

Table 1 :
Analysis of Existing Therapy-Oriented Games McCarthy et al. 2006).Enchantment in gameplay is defied by McCarthy et al. to mean pleasurable disorientation, where perception and attention are heightened (McCarthy et al. 2006).It states that a sense of wonder is tied to being present in the moment, without relying on our rational memory of something.Yet it also requires 'Flow' (Csikszentmihalyi (Freeman 2013 2009)ke light making objects bigger/dark objects appearing smaller and cool colours invoking calm/warm colours invoking energy are applied they can have a subconsciously tremendous impact on the mind.Examples in interactive artworks include a project by Salevati et al., exploring how colour theory can be used in addition to a generative visualisation system to effectively bring attention to viewer's emotional experience (2016).Subyen et al. explored how to use Kandinsky's colour theory to map metaphor to qualitative movement information in order to emphasise the perception of movement in a live generative visualisation system (2011).Figure 1: Colour Psychology in Logo Design and Kandinsky Colour Emotion Comparison action(Nacke et al. 2009).The kinetic camera is used in Tweetris, where two players compete to quickly create Tetris shapes with their bodies to play a whole-body form of Tetris(Freeman 2013).

Table 2 :
Actions Used in Gameplay Tied to PT Function