Creating Spatio-Temporal Contiguities Between Real and Virtual Rooms in an Assistive Living Environment

This paper gives an overview of how the role of architecture, design, and HCI is to be redeﬁned in a world of ubiquitous computing and the recent advances in display technology. It describes the future care lab and our take on research in this ﬁeld. We elaborate on how wall-sized displays can form an integral part of an ambient intelligence space that aims to assist the elderly and empower them to stay longer in their familiar surroundings before moving into nursing homes. We propose a classiﬁcation of different concepts that virtually extend a real environment and describe some possible scenarios.


INTRODUCTION
As large urban displays increasingly move into sight of our everyday city landscape, they are not limited anymore to the public places of metropoles like New York or Tokyo.With the rapid advances in display technology in terms of size and cost, displays are gaining increased presence in everyday life and in all kinds of environments, leading to the coined term of "ambient displays".While much research has been done in this field, relatively little has been done on displays of the scale of large urban screens in a home environment, specifically in "ambient assisted living" spaces.This paper focuses on the possibilities that arise when walls become displays and the role of these displays in the context of ambient assisted living and mixed reality.

space and Ambient Intelligence
A few decades ago, the function of a room was mainly defined by its static location within an arrangement of rooms.In domestic housing, the functional roles of kitchen, bathroom, living room, and bedroom are often clearly assigned by room layout and the relationships between the rooms.Moreover, the functions of the defining elements of space -floor, ceiling, walls and openings -did not exceed their primary architectural function.
However, Information and Communication Technology (ICT) is becoming more dominant in our personal living space.The concept of "Ambient Intelligence" (AmI) and its derivative "Ambient Assisted Living" (AAL) describe the intelligent integration of ICT as well as sensors into our (living) environment, thus making it aware of our presence and context, so that it is able to react to, communicate with, and support the people living within this environment (Raisinghani et al. 2004).
Until recently, windows and doors alone directly linked the inside and outside spaces.Windows have the primary function of ventilation and lighting, for example, but they also enable communication and allow vision out of as well as into the room.To allow for technical control, a wall traditionally holds a simple light switch which you use to change the lighting of a room, often in a binary manner.
With the emergence of ambient displays, the penetration of projectors into the consumer market, and especially the foreseeable availability of OLED display technology that promises thin and seamless wall-sized displays, the question of whether ambient technology should not include technology which is visually too prominent becomes obsolete.
The role of such screens is not limited to only displaying a given content.Being seamlessly integrated into space, they become architectural elements and allow for other basic functions such as lighting, as well as visually and functionally extending the room where they are located.
Intelligence in architecture is rarely explored, predominantly because most buildings are assumed to be "dumb".As the autonomy of buildings increases, new kinds of mechanisms for interaction will be necessary.

HCI, design and space
Human-computer interaction (HCI) has established itself as the sub-discipline that focuses on how best to design the interface between people and (computer) technology (Dix et al. 2003).As such, it has always had strong existing connections with cognitive psychology and graphic/industrial design disciplines.However, until recently, its major focus has been on interfaces for personal computing systems using the desktop metaphor.Only in recent years has it begun to tackle the difficult question of how to create intuitive, easy-to-use, efficient and elegant interfaces for mobile, wearable, and ubiquitous technology.
Computers have become a part of everyday life.Some are carried like jewels, being a status symbol as well as decoration.Recently, the iPhone has shown how much interface and product design can affect the success and spread of a technology (here, PDAs or smart phones) that was already previously available but overlooked by most of society.People would buy iPhones and competitor smart phones even if they intended to primarily use these devices only as a phone.Similarly, we expect that health technology will more likely be accepted by the people who need it, if the technology does not stigmatise them.We argue that the technology has to thus also be attractive to younger generations, and not be only associated with patient groups and diseases.
As AmI technology such as ambient displays become more commonly used and intrude into the personal space of a wider user group, privacy and acceptance issues also arise (R öcker and Feith 2009).

Motivation for AAL
The increasing demand in care personnel due to demographic change as well as an ongoing change in social structures may not be met in the near future.This trend as well as the wish of the elderly to stay at "home", i.e. in their familiar environment and neighbourhood rather than moving to retirement homes, is leading to the development of technological solutions to meet that demand (Riva 2003).AAL solutions not only aim to assist the elderly, but also the care personnel, for example by reducing the amount of time needed for visits.While these solutions help increase the efficiency of medical personnel, it is also understood that social contact is vital for human well-being and a protection against isolation and loneliness.In the case of the elderly, daily interaction with care personnel and doctors may be the patient's primary source of human contact.We argue that such technologies should not reduce the amount of human contact but shift or extend the patient's social contact experience.However, assistive technologies will not be used and accepted by patients without gaining a thorough understanding of individual needs and wants, ergonomics, and usability (Ziefle and Wilkowska 2010).
Future technology and especially future interfaces, which are by nature centred on the human being, are impossible to visualise and be tested without actually demonstrating how this "future" might look like.For this reason, we have built the Future Care Lab.

IN PRACTICE: FUTURE CARE LAB
Traditional technical development frequently fails to accept an inevitable fact: users do not interact with a single device in isolation (Borchers et al. 2010).Technical devices should be conceptualised as embedded into a spatial context that then presents a rich backdrop against which the use of a device is defined.This conceptualisation involves the use of technology to augment everyday objects such as furniture, everyday components such as floors and walls, and everyday rituals such as social contacting or the feeling of being comfortable at home.Additionally, technical devices should not only focus on efficiency, task-appropriateness, and a rational ease of use, but increasingly have to address the pleasure of using a device (Hassenzahl 2003), which includes the emotional component of technology.Increasingly, studies show that users desire more than the mere functioning of technology, but prefer interfaces with a high social or hedonic value.Hedonic functions provide stimulation, identity, and valuable memories (Hassenzahl 2003;Wright et al. 2005; Ziefle and Jakobs 2010).Design aspects of interactive environments have been experimentally tried in controlled conditions, usually as an "artistic" installation.Little or no research has been carried out as to the cognisant effectiveness of interactive environments.
This was the reasoning behind the development of the Future Care Lab.In order to develop and implement technologies that enable patients to live as independently as possible, we are building a lab replicating a living room.This lab concept permits the implementation of essential technology (e.g., medical technology for patients) in different parts of a home environment, and allows us to study architectural issues and design aspects against a social background, incorporating users' needs and wants, and to evaluate technology acceptance of different devices when integrated into the living space.
Our primary focus group is patients with heart disease at different stages, the severest being patients with a total artificial heart (TAH) and includes patients who are highly dependent upon technology (Klack et al. 2010).At this severe stage, direct benefits are obvious, such as minimising the need to travel to distant clinics that specialise in heart transplants for frequent checkups.
Our studies at leading heart clinics like Bad Oeynhausen (Germany) and the university clinic in L öwen (Belgium) show, that for TAH patients, only four vital parameters (namely temperature, weight, blood pressure, and blood coagulation) suffice to give an adequate view on the state of health.Up to now, these patients were frequently required to travel by plane to specialty clinics for checkups.
With medical technology integrated into a living room, doctors can remotely check on their patients.In the Future Care Lab, the weighing scale is seamlessly integrated into the floor, temperature is non-intrusively taken by a high speed thermal camera, and blood pressure and blood coagulation measured by traditional medical devices but through a data port.Having this equipment integrated into a home environment essentially saves the patients time and hassle.To not stigmatise the patient as diseased, the technology is invisibly integrated into the space.
The room also features a full wall-sized multitouch display (see Fig. 1).The display uses back-projection on special acrylics (PLEXIGLAS c 7D006 by Evonik) and an array of six short distance projectors (NEC NP600S) with a resolution of 1024×768px each, giving a total of 3072×1536px on an area of 4.8×2.4m.To enable multitouch functionality, an

EXTENSIONS
In order to systematically conceptualise and explore the different meanings of "space" and "enhancement" in combination with users' needs and wants, we are developing a testbed framework with different levels of how the real world can be extended by and merge with virtual contexts.The possible extensions are spatial, functional, social and entertaining.These extensions can overlap, be modified by time, and have different levels and sources of information.

spatial extension
Through spatial extension, the interface will open up the (real) room to other places or spaces.It thus overcomes distances and compensates for any physical constraints of the patient.Images or (live) video footage can extend the living room into the outside world with an apparent visual contiguity.Places across the world can be travelled or remembered.
To create this spatial extension, a wall-sized display shows either computer-generated worlds (virtual reality) or real world images.Milgram and Colquhoun Jr. (1999) defined the term "Mixed Reality" as the continuum that spans between reality (real environment) and virtual reality (virtual environment).This whole spectrum is imaginable with the concept of spatial extensions.Rekimoto and Saitoh (1999) introduced the benefits of "a spatially continuous work space".Here, users can interchange information between devices or screens that are physically adjacent.This concept creates spatial relationships between devices or objects, implying the need to know locative information about the devices that are to be related.
Rather than spatially relating mobile or movable devices, we propose that spatial extension relates two spaces with each other: the real space and a mixed reality space that is rendered on the screen.The mixed reality is displayed in such a way to match the perspective of the viewer, thus creating a greater sense of immersion and blurring the boundary between the world in front of the display and the world "in" the display.This creates the visual illusion of actually extending the space into the wall (see Fig. 2).Information about the precise location of the viewer is necessary and can be accomplished by head-tracking.

social extension
"Social extensions" include any application in which the interface connects the living room to other people or groups.By definition, even a telephone creates a link to another person and can thus be considered a social extension.However, in an AAL environment utilising wall-sized displays, this idea can be taken much further.Hauber et al. (2006) described the notion of space as one key criterion in emulating real face-to-face communication.A wall-sized screen can display another real environment (such as another living room), facilitating the sense of thereness or togetherness of the participants in the physically different spaces.Alternatively, a virtual, modelled world can be used to create a space for communication (like chat "rooms") .Correctness of perspective as described in "spatial extensions" can create a sense of immersion and increase the perceived sense of social nearness (immersive colocation, see Fig. 3).
Giving inhabitants the possibility to communicate naturally with people in distant locations -be it family, friends or even strangers that might share common interests or problems -helps overcome the feeling of secludedness.Social isolation and the emergence of depression is highly correlated with old age.This is explained by lack of social support and networks, and changing health (Wenger et al. 1996).Especially for chronically ill people, social network ties may help people to protect their health and to cope (Penninx et al. 1999).The facilitation of social networks in real and virtual spaces is one of the general goals of "socially aware ambient intelligence" in the ISTAG report 2009.

functional extension
Wall-sized displays and other AmI technologies can functionally extend the room, attributing new functions to it that were previously not thought of or not apparent from the room's architectural properties.When integrating medical diagnostic equipment, the room can act as an examination room or for self-monitoring.In the Future Care Lab, the interface allows users to control the devices and displays aggregated medical data.When making use of social and spatial extension, a doctor could be virtually present, and the patient's living room becomes an examination room for medical checkups.Life-sized, spatially immersive video conferencing turns a room into a "conference room" and is another example where social, spatial and functional extensions meet.

time
All the extensions described above can be subject to transitions in time.They can be either in realtime or have a time-offset.Social messages could be delayed when communicating with distant time zones.Videos or conversations from the past can be (re)played, the interface then acting as a memory for the elderly.

nature
By showing simple images of outdoor scenes, the wall-sized display acts as a window to the user's preferred "outside" world (see Fig. 4).Using head-tracking allows the experience to be

shared flat
Patients could visually link their living room to a friend's or relative's.The wall-sized screen will then display the video stream or a virtual representation of a remote room that is preferably equally-equipped with the same interface.The two co-located rooms will see into the other room, virtually sharing one wall and allowing for direct communication or giving the feeling of living together.
By matching furniture, this effect of virtual neighbourhood can be strengthened.For example, the virtual neighbours could both place the same table up to the display wall so that the table will then visually continue "into" the screen, allowing them to sit at the table together.

real size video conference
A doctor could virtually "visit" the patient, appearing life-size on the interface (see Fig. 3 and Fig. 5).
Being able to see real full-body human proportions, not available in traditional video chat applications that just show the upper part of the body on a relatively small screen, promises a more natural communication experience even for people who were previously unfamiliar with video conferencing.It is essential that the patient feels at ease when virtually "seeing" the doctor to achieve flawless communication and basic visual checkups.
In a business environment, conference rooms have used similar set-ups to create telepresence (Singh et al. 2008).

waiting room and health forum
The social interaction in a waiting room can be one important reason to visit a doctor.
Exchanging experiences on health issues with others can be a means to cope; however, as the diseases represented in a waiting room can be heterogeneous, a forum or a health-related social network might be better grounds to communicate experiences.Combining the concept of a waiting room and that of a social network, patients virtually visiting a doctor might be offered the chance to meet people with similar health backgrounds while waiting for their time slot.With each visit, the doctor will update the digital personal health record of the patient to optimise automatic meeting matching in the virtual waiting room.

CONCLUSION
Just as small screens and handheld devices can be considered decorative and thus strongly design driven, large displays can be considered architectural elements as they are becoming more and more a part of our urban and private space.When walls become displays, displays become architecture.Reciprocally, architecture influences the perception of big screens, and changes the way people interact with them as well as the space in which they are installed.
These changes, however, do not just imply new practical and theoretical conceptualisations of the perception of space and how it is designed.Acceptability is also a pertinent and significant issue.Scepticism of a technology can be overcome if design and function are universal and broad enough to also attract younger generations such as the children, or preferably grandchildren, of the elderly.
If we continue to understand the needs and wants of users through acceptability and user studies, the use of large displays as an architectural element within ambient assisted living spaces -in spatially, socially and functionally extending the environment of users -can have significant impact and positive effects on the user's medical and social health, as well as facilitate the health-care system.

Figure 1 :
Figure 1: wall sized display in future care lab Figure 2: left: "wrong" perspective, right: wall sized display showing correct perspective for current viewpoint

Figure 3 :
Figure 3: illustration, spatial and social extension

Figure 4 :
Figure 4: display acting as a window into nature

Figure 5 :
Figure 5: doctor and patient in immersive co-located communication CreatingSpatio-Temporal Contiguities Between Real and VirtualRooms in an Assistive Living Environment Kasugai • Ziefle • R öcker • Russell