Fear of Flight: Expanded Presence and Inter-sensory Gesture in Multimedia Performance

Fear of Flight investigates the intersection of surround audiovisual composition and live performance to draw meaningful conclusions about the experience of presence and the role of the performer within immersive media composition. Among the theoretical issues that contributed to Fear of Flight’s development are the impact of immersion on the perception of compositional dialog, and the mechanisms by which presence may be expanded and dislocated from a performing body through mediation.


INTRODUCTION
Fear of Flight investigates the intersection of surround audiovisual composition and live performance to draw meaningful conclusions about the experience of presence and the role of the performer within immersive media composition. The production integrates abstract audiovisual narrative with a live dancer's floor and aerial performance. It also considers the sensory experience of the audience, who must adapt to their own physical situation in the surround projection environment. Real-time video capture is employed to synchronize aspects of the dancer's movement in the air with the audiovisual composition during the performance. Part of the aesthetic and technical challenge of the piece lies in the seamless integration of high-resolution, prerendered, audiovisual material with audiovisual material that is generated dynamically within the moment of performance.
Among the theoretical issues that contributed to Fear of Flight's development are the impact of immersion on the perception of compositional dialog, and the mechanisms by which presence may be expanded and dislocated from a performing body through mediation.

Motivation: Multimedia Performance
Modern multimedia performance is a hybrid practice that inherits from many different artistic traditions and modes of discourse. It is performance that makes use of media technologies in such a way as to be an integral and significant part of the production's content. Rosemary Klich and Edward Scheer attempt to tease out its defining attributes, stating that: the poetics of all forms of multimedia theatre are built on the fundamental characteristics of hybridization, audience participation, and the prioritization of the performative act over linear literary narrative (18).
They also note that: Multimedia performance, as a medium that incorporates both real and virtual, live and mediatised elements, is in a unique position to explore and investigate the effect of extensive mediatisation on human sensory perception and subjectivity (2).
Though contemporary multimedia performance might differentiate itself from past artistic traditions, it is descended from a long historical trend to investigate the affective discourse of media under multiple modes of spectatorship. In his book Digital Performance: A H istory of N ew Media i n T heater, Dance, Performance Art, and Installation, Steve Dixon situates contemporary multimedia performance within a greater narrative of hybridization, experimentation, and technology rampant in theatrical practice. He demonstrates that performance has always made recourse to new technologies and explored the creative potential between live and mediated modes of representation. In his exploration, Dixon pinpoints three periods of theatrical activity that act as direct precursors to modern multimedia performance: the Futurist movement of the early 20 th century, Fluxus, 238 Allen Kaprow's happenings, and the other mixed media and performance experiments of the 1960s avant-garde, and the aesthetic revolution in that took place in the 1990s under the new found availability of a proliferation of digital tools (87). He points to each period's aesthetic focus on the directed experience of spectatorship through multiple modes of media discourse for common ground.
The emphasis on spectatorial engagement in contemporary multimedia performance has increased focus on how mediatization shapes the reception of experience as well as ontological discourse. Sensory immersion acts to shape the spectator's sensory capabilities as well as their mode of engagement and holds them within the fixed moment of presentation. Strategies that make use of such attributes as scale, contrast, volume, and surround placement, all contribute to the immediacy of experience. They not only nail the spectator's awareness to the here and now of the performance, but also condition the spectator's senses over time to shape how they perceive the production.
In post-dramatic multimedia performance, immersive media strategies are often voiced as scenography, to contextualize the performance and provide a sense of environment.
Dramaturgy is staged. It is derived from narrative. Media, on the other hand, such as sound or image may be composed. It can be organized according to non-narrative aesthetic heuristics to encourage the production of particular affect. Multimedia performance often employs an engagement with its constituent media that falls somewhere in the middle. It can be structured according to any discourse in which it participates, or can manifest a new one. The term intermedia, refers to artwork whose aesthetic sensibility not only combines media but also necessarily intersects them, emerging from the rich terrain of their encounter. While traditionally many artists and theorists have stressed the difference between live and mediated forms of representation, contemporary multimedia performance offers a fertile intermediary for exploration. According to Klitch and Scheer: The framework provided by the notion of intermediality moves away from the theoretical polarization of the live and mediated and provides a lens through which to explore the patterns manifesting across media within the theatrical frame. (5) Here, the theatrical frame is a powerful device, able to blur distinctions in ontology and physics to create a thin veneer of congruity.
A number of questions linger as to the nature of multimedia performance. Can live performance be integrated with media into a single compositional dialog? How does performance influence inter-sensory composition? Do mediatized representations function alongside live performance in a theatrical context? This dissertation aims to engage with these questions through practice, by constructing a work around them, and by using them to analyze the result. It undertakes this effort with the ultimate goal of widening the framework through which multimedia performance is evaluated in order to clarify the resulting aesthetic dialog. What is clear is that any investigation of multimedia performance must take the experience of the spectator into consideration. To do so, the affective discourse of inter-sensory composition must be interpreted within the narrative theatrical framework, and the narrative theatrical framework's influence on the perception of affective compositional discourse must be understood. Finally, any arguments that result need to be framed against the effects of immersion and intermedia dialog on spectatorship.

Description
Fear of F light takes place in a black box theatre, partitioned into an intimate 30 x 30 foot production space by four large screens and 8 speakers. The audience sits inside the screens, on the floor around a small clearing, and is surrounded by the performance area and four constructed walls of video. The space is tightly packed, and audience members can hear and see each other's physical reactions to the performance. The production makes use of an assortment of aerial apparatus, including a silk and hoop that are positioned directly inside and above the seating area. The relatively large scale of the projected sound and image and the proximity of the performance combine with the surround composition and choreography to fully immerse the audience's senses. Part of the work's production aesthetic involves placing the audience directly inside the performance. The immersion forces the audience to actively take control of their spectatorship in order to navigate the sheer amount of sensory experience they are provided with.
The production's narrative divides into five movements that outline an abstract narrative of spiritual desire and physical transformation, phrased in the metaphorical language of birds and space travel. A woman dreams of birds, and transforms herself into a bird-woman, to join them. Her own metamorphosis complete, she discovers that she is trapped in a cage and cannot fly freely as in her fantasies. After a difficult ordeal, in which she examines her psyche to find the nature of her own desire, the woman transforms her surrounding cage into a spaceship, and enters a new world from within it.
The first movement of Fear of Flight, the things we long f or (9:04), introduces the work's thematic

Fear of Flight: Expanded presence and Inter-sensory Gesture in Multimedia Performance
Dr. Freida Abtan 239 concerns of desire and the sacred power of the unseen. A woman's dream of birds, stirs a longing within her to take flight. She examines an empty birdcage and hears the movements of an invisible flock of birds surrounding her, commanding her attention.
In the second movement, the bor ders of t he s pirit (15:03), the woman is inspired by her dream, and her own world transforms to resemble its features.
The new landscape that surrounds her is filled with magical creatures, half-women, half-bird. She tries to join the creatures, but they reject her. Eventually, the woman dons a clock of feathers and is allowed to join the flock. She flies towards a perch in the air. From her new vantage point, she sees that she is now trapped in the birdcage she once considered.
The woman confronts her situation in the third movement, what is and isn't ar e (12:20). A hybrid landscape, formed of both the natural world and her dreams, surrounds her. The woman recognizes her own form in a procession of creatures, and has a sudden insight into her situation. She places her hand on her double's heart, and white birds fly out of it, engulfing the cage and transforming it into a spaceship. Key moments from the woman's journey and the fantasies that define her are depicted in the fourth movement, gravitational pu ll (7:50), through the windows of the spaceship. The ship takes off and burns itself into a charred husk that dissolves into dust.
After a terrifying journey, the fifth movement, in outerspace w e t rust (11:04), begins in darkness.
The woman floats weightless, in a void that slowly fills with stars. Bubbles in which women lie dreaming surround her. As the woman discovers her place in the sky, she notices that she is not alone. Her movements give shape to both the matter and physical processes that surround her.

EXPANDED PRESENCE
Fear of Flight's characters are constructed through several modes of representation. They are portrayed on stage, by performers, who are present in the same temporal-spatial moment as the audience, outside of the production's theatrical discourse. They are portrayed through mediatized representations within the video projections, derived and digitally constructed from performances that took place in the past in front of a camera. Finally, they are portrayed through mediated artifacts of live performance, including shadows and real-time video feeds of the stage area that have been manipulated through image processing techniques and composited into the projected video scenography. Each mode of representation is signified to function as presence within the theatrical discourse of the production. All of them are staged.

Fixed Representation
Depictions of the cast, wearing the same or similar costumes to those worn on stage, appear in Fear of Flight's video projections during all five movements. The depictions portray the same characters as do the performers on stage, and participate in the production's narrative construction. They extend, and contribute to, the performance, functioning alongside the performers on stage even though they are subject to different barriers of representation. For example, they are not subject to gravity and cannot break the frame of their projection surface.
The actions of the depicted characters are fixed, having been created from pre-rendered video material; however, in their presentation they are resignified within Fear of Flight's theatrical discourse to participate in the same sphere of representation as the narrative. There, they occupy an equal position to the characters portrayed on stage, though they do not always belong to the same ontology. Instead, they alternate between functioning as do the characters on stage, and participating in psychological or allegorical modes of representation that are brought into the narrative through ontological syncopation.
According to Cormac Power, theatrical situations that rely on multiple modes of presence put presence i nto pl ay (9). Characters that are portrayed in the projected video are conjured through the play of their presence within Fear of Flight's theatrical discourse. In the simplest case, the video depicts a photographic referent, an original performer whose performance was captured within the video footage. The video projection, and the forms it depicts are literally present on stage as organized light that falls on a screen during the performance. The referent of the image is present, conjured through their aura in the projected video though they may have been modified or fictionalized through digital process, as is the original performance of the referent and by extension the character they portrayed in that performance. The character that is portrayed within the performance of the production by the video depiction is fictionally present when the video is projected during Fear of Flight's run.
Fear of Flight's characters are complicated through context and by the multiple modes of representation that are collapsed in the production's theatrical discourse. The same characters are portrayed within the projected video and on stage. Not only are these portrayals multiple and concurrent, but they co-exist within the same (syncopated) temporal-spatial moment, and are signified to construct a single character within the production's theatrical discourse. As a result, the portrayals conjure each other, inform each other,

Fear of Flight: Expanded presence and Inter-sensory Gesture in Multimedia Performance
Dr. Freida Abtan 240 and evoke each other. The presence of the characters is expanded through being put into play within the production's varied spheres of representation.
Multiple portrayals of the same characters co-exist within Fear of Flight's production, sometimes at once. In the second movement, the bird-women depicted in the trees, simultaneously appear onstage, portrayed by the supporting cast. Although the two flocks of bird-women are dressed differently, they have the same faces, the same narrative identity, and a similar vocabulary of movement. Later, when the characters on stage have finished their performance, their video depictions are replaced with new expressions of their identities: as dreaming women on beds of feathers.

Real-time Representation
As well as live stage performance, and fixed video depictions, Fear of F light uses artifacts of live performance to create real-time representations that function as characters, or extensions of characters, within the production's theatrical discourse. The actions of the real-time representations are determined in the moment, as they are created through processes that mediate live performance. When their subject is visible, the mediation is obvious: any motion in the real-time representation originates with its subject. When their subject is not visible, the real-time representation stands in for the performance, substituting the mediated version. The implementation expands the presence of the original performance, and characterizes it with new affective discourse to reposition it within the narrative.
Fear of Flight uses two kinds of mediation to create real-time representations of its characters. The first relies on a live video feed of the stage performance. The feed is processed through image-manipulation software before being composited into the video projections. The second relies on shadows, created through interrupting stage lights or video projections. Movements 1, 2, and 5 incorporate a live video feed, while shadows are used all over the production, being a natural result of the video projection set-up on stage.
In Movement 1, the dancer is portrayed on stage by a live performer, and depicted in the projected video by a fixed representation. A real-time feed of the performer is composited within the projected video, offering a third depiction of the dancer to coexist with the others. The real-time representation of the dancer is captured from a viewpoint similar to, if higher than, the audience's. Over the course of its use, the feed's image is manipulated through spatial and temporal processes that slow down and blur the dancer's representation. Her fast movements disappear while her held positions are accentuated. The depiction is abstracted by the visual process, and made to resemble the fixed representations over which the real-time feed is composited.
On one hand, the real-time mediation of the dancer functions as a performance prosthetic to the dancer on stage. Her presence has been extended into the aesthetic realm of the projected video, where she is subject to its boundaries of representation and any ontological discourse it follows. The real-time representation functions without its referent as a fixed character would in the projected video. On the other hand, its context informs the dancer on stage so that her actions convey more than their individual meaning. The dancer's presence is expanded between these two representations, so that the discourse of one colors the other and vice versa.

Constructed Agency
The characters that are depicted in Fear of Flight's video, work precisely because they are signified to have agency within the theatrical discourse. Their actions matter, and form the basis for the production's narrative, as well as influence the actions of the performers on stage. It doesn't matter whether the projected characters can make decisions in the moment, or whether their referents can 'corpse'. It matters that they have been signified as being able to do so inside the conventions of the performance.
Constructed agency advances the complex intermedia discourse that informs Fear of F light's characters. At the end of Movement 4, the characters depicted within the projected video interact with the dancer on stage. A funereal procession depicted within the video turns and waits for the dancer on stage to approach them. The leader of the procession, a depicted version of the dancer, reveals her heart to the character on stage. It releases birds when the dancer on-stage extends her hand to touch it. The narrative interaction relies on the signified agency of the dancer's depiction in the video, in order to function. The depiction does something within the theatrical discourse. The depiction is performative , (Austin 5-6) whether or not it is performing.
An astute observer may remark that all agency attributed to characters within theatrical discourse is constructed, regardless of medium. Agency portrayed by performers on stage is no less constructed than that portrayed otherwise. While the process is transparent in the physical world of the stage, it is less so in the case of projected video. It is usually, thinking, feeling performers, who possess their own agency outside of the theatrical discourse, who portray the characters onstage. Projected representations are not (as of yet)

Fear of Flight: Expanded presence and Inter-sensory Gesture in Multimedia Performance
Dr. Freida Abtan 241 self-aware. Constructed agency is what turns narrative elements into characters within theatrical discourse. It imbues its subject with fictional presence, because agency implies an acting subject.

Audiovisual Composition and Narrative
Along with its role in establishing dramatic context and events, Fear of Flight's scenography lends affect to the narrative through its audiovisual discourse. The pacing, edits, and motion of the projected video, in dialog with the music and lighting, direct the production's composition. They temporally structure the work, preparing and conditioning the audience's senses to experience the narrative. While this dialog manifests at the level of affective perceptual influence, it also establishes the emotional tone of the production and contributes to the interpretation of dramatic events, as does the physical experience of the projected media immersion.
In Movement 1, visual motion created within the digitally manipulated image of the dancer is linked with the sound of processed bells. It is emphasized and temporalized to the slow pace of the music through synchresis. The audiovisual dialog of the scene informs the on-stage performance that follows. Though the music that accompanies the dancer is much faster paced, and the video that plays offers several scales of motion, the slow, dreamy, feeling of created through the audiovisual composition haunts the live performance. The audience has already developed a sensory experience related to the narrative that does not dissipate with the change in representation.

Presence and Gesture
Fear of F light's audiovisual media composition relies on the phenomenological, affective discourse that surrounds the perception of relative motion between the ocular and aural senses. In performance, the theatrical framework of the production heavily influences the perception of audiovisual dialog. To fully evaluate Fear of Flight's composition, the production's media elements must be re-examined within the context of their theatrical engagement. The theatrical discourse surrounding the media shapes their reception, just as does their inter-sensory discourse.
The audiovisual motion that occurs in Fear of Flight not all of equal importance to the spectator. The depicted dancer is signified as a character within the production's theatrical discourse. Her motion within Movement 3 holds much more importance than does the sweeping expanse of sky that unfolds around the projection space, even though that motion is much larger in scale. The deformations of her body emphasize the visual motion around her when they synchronize together. When they do not, however, the motion of her body remains visible while the competing motion is largely ignored. Confluences between the visual motion of the dancer's body and the music are significantly better perceived than confluences between the music and the other visual objects in the scene.
To illustrate how presence and attention shape the spectators perception of motion, consider a real world example in which a spectator listens to music a friend has performed. While the spectator can't see their friend, the knowledge of their friend's involvement adds the presence of a subject to the experience of their listening. If they listen with awareness of the subject, their experience of the music is shaped around interactions with the subject and any motion that is associated with the subject is emphasized. The affect of selective attention shapes the perception of motion based on whether external motion is considered to be correlated with or pertinent to the source (Dishon Berkovitz and Algom 1437).
In Movement 3, an image of the dancer is animated through digital manipulations. Its importance within the spectator's gaze rests in the fact that the dancer is a strong participant in the production's narrative, and her depiction maintains a complicated mix of presence in play, as well as its literal presence in the stage area of motion and light. A further distinction can be made regarding presence and the perception of motion according to attribute. Motion that occurs from a depicted body's gestures is different than if it is generated from distortions of the image plane. Such motion is perceptually reinforced through kinesthetic empathy.
The theatrical dialog of presence helps guide the spectator's attention through the production's media elements. It allows audiovisual representations to participate in the human perceptual dialog of empathy and subjectivity, as well as the phenomenological dialog of the senses. The conventions of theatricalization, and in the case of specific human representation, kinesthetic empathy, allows the projected media to function as more than anthropomorphized sound and color, but as performative characters who voice (constructed) agency within the context of the performance.

Porous Affect
The compositional dialog of Fear of F light inherits from the structural and phenomenological audiovisual discourse of its media construction, as well as the theatrical discourse of its presentation and of its live performance. Spectators are forced to engage with its multiple elements Expanded presence blends modes of sense perception across time. The inter-animation between mediums lingers with their contact. This is true both in cases where the contact is obvious, and in cases where the contact occurs more subtly, such as when multiple mediums participate in the production of affect but don't share an easily discernible interaction.
The inter-medium affective discourse that takes place within Fear of F light creates an unsettling aura around its characters. The audience utilizes a perceptual frame that has been widened by crossmodal sensory interplay to perceive them and to decode the subtleties of their performance. It is as if, along with each character on stage, ghost versions are present, manifested from within the different spheres of representation they have been represented and have had contact, informing every act. These ghosts, though not actually on stage, haunt each character's presence, so that their subtle communications are embodied within the performance, even while they themselves are not visible. In Fear of Flight, this haunting is an area of compositional construction.

CONCLUSION
Fear of F light explores how different media inform each other through affective leakage and contamination, and through intermedial compositional structure. The piece participates in a lineage of performance that questions the perceptual characteristics of media and the draw that they exert upon each other. Compositionally, Fear of Flight demonstrates multiple ways in which the production of affect between media can evolve successively within a compositional structure to achieve strong aesthetic results.