Magic Mirror: Exploring Designing for Transitions as They Play out in Everyday Physical Rehabilitation

In this paper we report our initial experiences in designing digital technology for facilitating various kinds of transitions that elderly citizens and therapists manage, as they go about the rehabilitation process in the clinic and beyond. Specifically we detail MagicMirror as an example sketch in designing for the transition between exercising at the rehab centre and at home; and also between minute everyday transitions while exercising itself. We invoke the notions of collaborative articulation and piggybacking on everyday artefacts and practices as the way to design for these transitions. Finally we reflect on how MagicMirror explorations contribute to ongoing discourses in designing for the experiential quality of Fluency.


INTRODUCTION
With the Wii platform introducing a more physical gaming experience the idea of using this technology, as part of physical rehabilitation seems straightforward.
Hence, experiments with 'WiiHabilitation' are gaining ground throughout the world of geriatrics and physical rehabilitation [1,4,7].The Wii platform allows us to look into the general notion of integrating play and the experiential qualities belonging to the gaming universe with the tough everyday challenges of going through physical rehabilitation.Furthermore, taking a closer look, bowling using the handheld Wii controller or snowboarding with the 'Wii fit BalanceBoard' takes what has been on the research agenda for years -the inclusion of more bodily/physical ways to interact with computersfrom the lab to the living rooms of the general public.The Wii platform has mainstreamed physical interaction technology in terms of cost and robustness.This provides us with a tremendous opportunity to pursue our overall exploration of new ways for humans to interact with digital technology not in secluded lab settings but 'out there in the wild' as part of everyday life.Taking explorations in WiiHabilitation as a starting point, we look beyond the immediate and specific features of the Wii platform to address the overall question of whether and if so how digital technology can be designed to facilitate smoother transitions, of various scales, in the rehabilitation process as experienced by therapist and citizens.Hence, see the Wii platform not only as a new instrument for training but also, at the same time, as an opportunity to introduce a new level of information sharing and exchange between citizens and therapist in the rehabilitation activities as the play out over extended periods of time and across the home and the clinic.Specifically, we believe that this provides us an opportunity to explore the role digital technology plays in supporting this process of extending the rehabilitation process beyond the clinic into the living rooms of senior citizens.
In this paper we report on our early experiences of envisaging, sketching and co-designing MagicMirror.MagicMirror is an example sketch embodying the notions of designing for transitions: one, the macroscopic transition of the recorded exercises from the clinic to home and back.The other kind of transitions are microscopic, minute transitions that the elderly patients manage, for instance, while exercising at home, with the video playing on their TV as a mere visual reference, to engaging with it more directly for control and manipulation, and back to 'unmindful' exercising.We first detail the MagicMirror scenario and discuss the notions of transitions.Building on previous work [6], we explore the notions of collaborative articulation, and piggybacking on everyday artefacts and practices in designing MagicMirror for a smoother management of these transitions.We then emphasize how the various MagicMirror sketches embody these notions and are the driving engines of our research through design.Finally we reflect on our early experiences and connect the sketches to the ongoing discourse in designing for fluency [5].

MAGICMIRROR SCENARIO
Moving beyond Wii, we employ the metaphor of the mirror, taking off from a gym, where wall-wall mirrors provide a self-referential image of ones progress of muscle building.Here the MagicMirror takes multiple forms at various stages and situations: a big screen at the rehab centre and a TV at home, for instance.With MagicMirror we explore the possibilities of tracking and video recording the exercises the citizen undergoes under the supervision of the therapist at the rehab centre.The citizen takes home this recorded video and uses it as the 'reference' exercise to monitor self while exercising at home, as it plays out on the TV.Simultaneously, the MagicMirror tracks the body movements, records and overlays it on the instructional video, thus giving the citizen a self-referential video for exercising.This recorded video is then taken back to the clinic where the citizen and therapist discuss the progress in detail, thus enabling a more collaborative rehabilitation process.The Scenario as depicted in Figure 1: Ole is a 70-year-old man living alone.He has developed knee pain due to a biking accident and is undergoing a rehab process at the rehab centre.He meets his therapist and they record the video of Ole doing the exercises at the rehab centre.The videos are carried onto Ole's cell phone to his home, where he plays them on his television, while exercising.The MagicMirror tracks his movements and overlays this feed with that of the recorded version, thus giving Ole a direct reference to exercise correctly.On his next visit to the rehab centre, the therapist looks up the recorded home videos of Ole's exercises to take a look at his progress.

3.DESIGNING FOR TRANSITIONS
As an integral part of our work, we explore the notion of transitions and designing for transitions.Within the world of rehab with MagicMirror we identified two kinds of transitions: one, the macroscopic transition managed by taking recorded exercises from the clinic to home and back, and secondly the micro transitions that happen as part of everyday transactions with the world around us.
In MagicMirror scenario we build on the previous work of designing digital technology for hand rehabilitation by Sokoler et al [6].Here they address specifically the designing of short-lived assemblies to support collaborative articulation of the rehab process between the citizens and therapists.The short-lived assemblies are formed with digitally augmented devices piggybacking on everyday artefacts and practices.With designing for both the macro and micro transitions we explore both the notions of facilitating a collaborative articulation and piggybacking on everyday artefacts, respectively.Below we detail these transitions as they play out in the world of rehabilitation with MagicMirror.

Macro Transitions: Towards a more collaborative articulation of the rehab process.
MagicMirror is part of a larger research project, where we are exploring if, and how, digital technology can facilitate a sense of continuity in the rehabilitation process beyond the rehab centre.Currently, the therapists prescribe exercises to do at home by the citizens, but don't have much information of how the citizens managed to do these exercises.At the same time, unlike while exercising with their therapists, the citizens don't have much clearer instructions during exercising at home.The MagicMirror scenario (see figure 1) then is an example of a way digital technology can facilitate rehabilitation process to go beyond the rehab centre by enabling information exchange between the rehab centre and home.As the scenario depicts, the recorded exercises at the rehab centre become the ideal exercise while exercising at home.The live tracking and overlaying of the current exercise over the ideal exercise gives a self-referential monitoring of the progress to the citizen.With this recorded home exercise available at the rehab centre, the citizen can discuss with the therapist the various aspects of the progress of the rehabilitation process.
As with the hand rehabilitation project [6], here the citizen's mobile phone is the central device carrying the recorded videos (see figure 1).As the citizen and therapists manage this transition of rehab information, they set themselves up for a more collaborative articulation of the rehab process.

Micro Transitions: Piggybacking on everyday things and practices.
The other kind of transitions is microscopic, minute transitions that the elderly citizens and therapists manage in their everyday interactions during the rehab process.Within the design space of MagicMirror, we identify two such example transitions that we imagine the therapists and the citizens manage both at the rehab centre and at home.We explore the notion of piggybacking on everyday artefacts and practices in designing for these transitions.First, we look at the transition that the citizens have to manage in shifting demands of their attention while exercising at home.
For instance, the transition exercising with the video, playing on their TV, as a mere visual reference -to engaging with the video more directly for control and manipulation, and back to 'unmindful' exercising.Other similar transitions that they manage are when someone rings the doorbell or they get a telephone call.

Figure 2: Augmented bottle as facilitation transitions between engaging with the video and other activities
In negotiating these transitions, we drew on many artefacts that the citizens use as props while exercising: a wrist band or watch, or as one therapist suggested, water bottles of 500ml.capacity. Figure 2 shows the paper sketches of augmenting a water bottle with a sleeve.The citizens hold the bottle while exercising, and as soon as they keep the bottle down and move their hands away, the video play and tracking pauses.From here, they can engage in other activities, like answering the phone or the door.Here the control over the video is integrated in the exercise practice and hence allows for a 'smoother' transition in addressing other demands in attention of the citizens.We explore the other set of transitions in the rehab centre, when the citizen and therapist engage in the rehab process.For instance, the therapist and the citizen discussing the recorded video from home; and moving from here to record a new or modified set of 'ideal' exercises.Here the stress is more on the rearrangement of the social assembly around.Figure 3 depicts such a transition from an informal discussion to a more formal recording session.
In designing for such transitions in social space, we piggyback on the big screen augmented with a strip of LED lights controlled by a phone-docking device (see figure 3).The citizens dock their mobile phone in the docking device kept near the screen, and this triggers video files to be accessible by the therapist.If they feel a need to do another fresh recording, they place the mobile phone in a 'recording' slot of the dock, which then lights up the LED strip around the screen.This signals the shift they managed from an informal discussion to a more formal recording to themselves and other people in the rehab centre.Thereby, by designing the switching on of the LED lights by placing the mobile phone in record mode, the therapists and the citizens manage a smoother rearrangement of their socio-material surroundings.

SKETCHING MAGICMIRROR
We follow a research through design approach as the core methodology for our research projects.Our work lies in the design exploration vertex of the Interaction Design Research triangle [3], with sketching as the driving engine.Inspired from Bill Buxton [2], by sketch we mean early manifestations of design ideas in various forms, not the least in software/hardware, video and photomontages, etc., that is available for the people to experience.
Taking the above paper sketches (figure 1, 2 & 3) as the starting points, we sketched the MagicMirror aspects in various forms as detailed below.

The MagicMirror
The intended technology behind the MagicMirror is motion tracking through Infrared enabled webcam, which will then be processed by an image processing software to track and overlay.This process in real-time, with the processing done by the software as the video is being captured.Our sketch of MagicMirror is still away from this technological possibility.However, we simulated the tracking and overlapping by using a combination of Adobe Premier and Flash.The exercises were captured from the Adobe Premier suite through the Mac webcam, which were then taken to flash and the body joints were tracked frame by frame (see figure 4).This video was taken to the home of the elderly citizen and played as the reference video.The new exercises were again recorded through Premier and played alongside the reference video during the exercise at home.This home video was taken back to the studio, to be tracked and aid over the reference video.Finally we took this overlaid video to the rehab centre again to simulate and enact the discussion of progress.

Figure 4:
The body joints tracked for the particular exercise.
For this initial exploration, we selected one form of exercise for back training: lifting both the hands from side of the body to the top and back.The vertical body position, and the straightness of the joints were the main aspects to be tracked in this exercise.As shown in figure 4, the MagicMirror sketch tracks the vertical position of the spine with respect to the hand movements.Also the citizen was asked to wear wrist and arm bands with IR emitting LEDs, which were tracked by a webcam ensemble, for later processing in an image processing software.This process is currently ongoing.For the purpose of this sketch, the black bands augmented the visual reference provided by the lines.

The sketches for facilitating micro transitions.
For the transitions during exercising at home, we wrapped a wristband mocking up an augmented sleeve.We planned to use the bottle to enact the situation of answering the doorbell while exercising at home.One can imagine directly how as one leaves the bottle, the video pauses and starts again when the bottle is picked up.We further plan to augment the water bottle with a sleeve integrated with pressure sensors connected to Arduino, which will be interfaced with the Image processing software to control the play / pausing of the video.At the rehab centre, we took a strip of LED and connected it to a button through Arduino Duemilanove.The button was fixed to a breadboard, which also acted as the docking device (see figure 5).As the citizen kept the mobile device on the button, the LED lights switched on and off when the button was released.

ENACTING AND EXPERIENCING MAGICMIRROR
As a core part of our research, we also employ sketching as an evoking process for co-designing with related skate-holders.With sketching, enacting, or experiencing a sketch we move towards a more collaborative exploration of the design space and possibilities thrown up by the MagicMirror.Here we highlight briefly how this process of sketching MagicMirror in-situ helped us in the collaborative exploration.
During the process of sketching and after, we involved therapists from the local rehabilitation centre at Tårnby, a suburb of Copenhagen.As an early exploration of the possibilities, we enacted a three stage cycle with a senior citizen and the therapists: recording of the reference video at the rehab centre, exercising at home using this reference video, and finally, the meeting back at the rehab centre with the therapist and citizen discussing over the home video (see figure 6).With reference to the macro transition, the therapists could immediately realise the importance of the video material, as any discrepancy was visually evident with the tracking of the body joints.The senior citizen meanwhile, was enthusiastic of seeing herself as the focus of the reference exercise.She specifically mentioned that the overlaying of her home exercise video with the reference video is really helpful, as she gets encouraged seeing to what extent she can stretch her arms.With reference to the micro transitions at home, the citizen when explained the idea of the bottle, insisted to enact the situation by using her own prop: a cardboard roll (see figure 6) that she uses while exercising at home.This clearly indicates how the sketching and enactment enabled the citizen to imagine further possibilities.She also appreciated the possibility of using an exercise prop to transit gracefully from being engaged with the video to other demands.With this, she said, she could get back to the exact position of her exercise after attending to her cat.The therapists accepted the idea of the lit LED strip on the screen while recording.They could imagine how it might help a smoother rearrangement of people, as there are number of citizens exercising in a large hall.They believed the sublime light from the LED would give a hint for others that there is a recording in progress and they can be approached once the light switches off.

5.REFLECTIONS
In this paper we described our early experiences in sketching MagicMirror with an elderly citizen and her therapists.Specifically we highlighted how the sketch embodied the notions of designing for two sets of transitions: the information exchange between the rehab centre and home, and the minute shifts the citizens and therapists manage as they go about the rehab process with MagicMirror.
In designing for these transitions we built on earlier work in hand rehabilitation [6], specifically exploring the notions of collaborative articulation, and piggybacking on everyday artefacts and practices.We also emphasized our way of researching through design, with collaborative sketching as the driving engines informing on the ongoing discourses on designing for transitions.
Löwgren presented the experience quality of 'Fluency' [5] as the degree of gracefulness with which people manage multiple demands on their attention.With increasing infusion of digitally augmented artefacts in our everyday lives, designing for a smoother transition between these multiple demands is an ongoing challenge.The MagicMirror is an example sketch embodying the notion of piggybacking on the everyday artefacts and practices to design for such transitions.
Thereby with MagicMirror we believe we move a step forward in designing for transitions and contributing to the ongoing discourse on fluency as an experiential quality.

Future work
As mentioned, this is a report in our early work.In future we plan to develop a more stable sketch with a real-time tracking and processing of the exercise videos through IR enabled webcams and image processing software.At the same time we plan to expand both on the types of exercises and also on the number of citizen participants in the codesigning phase.Finally, we plan to identify more transitions and build hardware sketches embodying the notions of designing for transitions.

Figure 3 :
Figure 3: Managing the transition from discussing to recording at the rehab centre

Figure 5 :
Figure 5: Collaborative articulation of rehab process with the Docking device sketch.

Figure 6 :
Figure 6: the Three Stages of MagicMirror Explorations -rehab centre, home and back