Cosmetic Space

Cosmetic Space is a public participatory art installation that reconstructs exhibition visitors as 3D avatars and assigns them different roles in a generated virtual 3D community. It enables visitors to instantly model a 3D double of themselves based on 3D facial model reconstruction. The 3D doubles become digital actors in a simulation depicting an unsettling cyclical performance of ritualized social violence. The different roles in the simulation are assigned to the doubles by a story engine developed for the installation.


INTRODUCTION
How do we step into representations? Cosmetic Space is a research project about inhabiting the imageextending our physical likeness into virtual realities and simulations. The project explores new, hyper-realistic capture and 3D modelling technologies, and their effects and social implications on human relationships.

DESCRIPTION OF THE PROJECT
Cosmetic Space takes shape as a participatory simulation which enables visitors to instantly model a 3D double of themselves, based on a facial scan. These masked doubles become digital actors, whose activities and gestures are controlled by a dynamic animation system. The animated scene re-enacts so called scrubbing patrols, which occurred in Austria after its annexation to Germany in 1938, articulating how political tensions are mutated into humiliating public rituals.
The setting of Cosmetic Space is a virtual world that feeds directly from the physical reality of the exhibition space. In the first area of the installation, the avatar production room, 3D doubles are modelled from people. Upon approaching the installation, the visitor finds a virtual therapist on a screen performing a treatment on an unknown person. In an instant, the visitor's face is captured and merged onto the character being treated. The therapist pushes the newly formed double into a second room behind her when finished. The new double wanders further into the representational space, and she is seen again on screen in the next roomthe cinemawhere a community of doubles and perhaps some previous exhibition visitors are gathered.  The grey audience is attracted by the scene, but unable to decide how to deal with the situation, they remain passive. Social pressure is transmitted from the work to the real audience, as visitors follow from the animation how their digital doubles treat one another and how this affects the relationships between the people watching the action.
Every entrance of a new double to the scene creates an event in the perpetual narrative, where time has collapsed. The arc of the story occurs only by integrating new exhibition visitors into the Cosmetic Space and assigning them a role in the double community.
It is difficult to know how much of us as individuals belongs to others and to our social communities. Cosmetic Space explores how the collective mind treats individual destinies. The re-enactment of the scrubbing patrols from 1938 Vienna articulates the role of public ritual and the performance of violence in the social formation of political identity. Scrubbing patrols were a collective aggression of Nazi propaganda against unwanted citizens. Local Jews were rounded up and forced to clean the city streets, drawing a large and often cheerful crowd of Austrians, happy to observe the humiliation while being spared themselves.
Cleaning is a harmless everyday activity, normally considered boring and necessary, but it has darker meanings when tied to ideas of racial purity. Cleaning as disciplinary act is a tool for establishing psychological hierarchies between people and marking the other as polluted. In Cosmetic Space, the character design of the body doubles underlines these group dynamics. Each body is textured with contemporary signs transformed as attitudes representing the social values of the black, grey and white groups.
The software running the installation does not have any skin colour or gender recognition, or other discriminatory features. The body assignment is based entirely on the story engine and does not perform any evaluation on the audience members. We are expecting unexpected doubles.

TECHNICAL DESCRIPTION
Technically, our main challenge has been to develop an automated 3D portraiture system, which runs at interactive speeds, while producing reliable and aesthetically appropriate results forany visitor in an unsupervised setting. 3D scanning of human subjects in a live scenario is a field of active research in the computer vision community (especially for surveillance and immersive media applications), but no single technology has yet emerged to solve all of the requirements of a public interactive artwork such as ours. For this reason, over a number of months of research and development, we have produced a custom 3D scanning pipeline that is able to create a highresolution animatable face model of any visitor, in just a few seconds from a simple stereo photo. This technology provides a key element to our experience. It serves as a "portal" between the physical apparatus and the virtual environment, through which visitors are integrated into the work itself as characters, becoming actors and participants in the story.
The environmental and interaction design of our scanner draws inspiration from the world of security surveillance and commercial augmented media, considering a speculative alternative form of devices such as airport body scanners.
Another key influence has been the tintamarresque photo booth, a form of tableau or painted backdrop, which often contains holes, through which visitors can place their heads and be photographed, to appear as characters in an imaginary scene. The tintamarresque represents an aesthetic mixture of portraiture and caricature, the resulting photo becoming evidence of, or a souvenir from a trip to another life-story. In our system, however, the story is not a static snapshot, but rather a dynamic world in which the likenesses or reflections of the visitors take over their own actions and lives.  We have designed our scanning system to be modular, portable, and powerful by developing a GPU-enabled C++ software framework based on open-source libraries. Specifically, we are following cutting edge academic research in the field of 3D morphable modelsthis research allows us to create virtual masks that can be fit to the faces of the visitors by analysing facial landmarks in a pair of 2D images. The models are textured in photorealistic detail using this same image pair, resulting in a 4K texture map of the face of every participant. The mesh and texture are then passed through a post-processing stage where they are modified to our aesthetic. The entire scanning process takes just a second or twoafterwards, the newly generated and customized face model is automatically fit to a rigged avatar and animated in our networked multi-scene Unity environment. As we have developed a fully functional prototype of our scanning system, our aim now is to refine the production, and to expand the system to a multiuser scenario, in order to investigate the group dynamic of our story, and to create a participatory experience that is both personalized and socially complex.
The development plan is to take the current prototype through to post-production, in order to have the installation premiere sometime in 2019.