Sonic City

Sound is a key factor when dealing with any degree of immersion. We define our perception of our worlds by our relationship to space. The way our minds spatially locate sounds within our environments tell our brains what sort of place we are in. The minimalist aesthetic of Sonic City empowers our brain to mentally fill in the gaps not visually seen in the environment. This action of focusing on the sound sources enables the audience to contemplate their relationship and perception of space.


INTRODUCTION
As humans, we predominately use our sense of sight to navigate our environments.This visual sense of sight constructs the world around us in our minds.We receive cues from the visual information received that tells our brains in which direction to move, what we should avoid, and the distance between these visual cues.This constructs an instantaneous map with our brains, but the sounds we perceive along with the visual information subconsciously contribute an equal amount to the development of this map.We receive a ton of sound information daily that construct our environmental realities -such as, feet walking, wind blowing, or vehicles in the distance.Unlike the visual counterpart, the sense of hearing is primarily always active.Unless a person actively blocks their ears with a tool, the brain will pick-up external audio wavelengths.Therefore, the nature of our sense of hearing is developed throughout time from birth to become a subconscious trait.Sonic City aims to bring this subconscious trait to the forefront by navigating the virtually perceived space in a sound dominated environment.The contrast of a heavily audio driven environment against a minimalist aesthetic design brings the audience's attention to the sound.This juxtaposition guides the audience to contemplate their perception and relationship with space.

BACKGROUND
Our relationship with the space around us is formed from birth as our eyes and ears monitor the world.

Sonic City
Bhakti Duran,Jinsil Hwaryoung Seo,Brian Smith & Echo Zhao 97 We use our hands to feel around our immediate space as infants to guide us in directions that we'd like to go.Our eyes fixate on visual information to construct three-dimensional maps within our minds.As these maps are being developed almost instantaneously in our brains, the sounds emitting from each visual source is archived along with each reference.Our senses swim in seas of rich information that help establish what you (implicitly and explicitly) know about the world and how you can behave toward it (Rosenblum 2010).

Senses
The ways we navigate the world are predicated on our five senses -each sense offering a supplemental method of building upon our brain's library of references.Our sense of taste tells our brain which foods we prefer and those we do not.Our sense of touch tells it what is safe to hold and what should be left alone.Our sense of smell can similarly detect which pheromones we prefer and what odours to avoid.We might think these types of sensorial information is only available to other species, but new research in perceptual psychology and brain science is revealing otherwise (Rosenblum 2010).We are constantly accomplishing several feats -you can hear things that do not make noise, smelling things that have no discernible odour, or see things that have no form.These exotic skills are all based on the physical information available to your senses along with the physiology that allows your senses to take in this information (Rosenblum 2010).
These are by far overwhelmed by our most emphasised sense -our sense of sight.The majority of the population that is not visibly disabled predominately navigates their worlds reacting to what is seen.Once they get to their homes they visibly see where the door is and move in that direction.Though this is our main method of navigation, our sense of hearing is equally as important in our perception of this world.

Sound
Our sense of hearing is an equal half of the two senses that predominately help us navigate the environment.Though our hearing is a passive trait, which we cannot actively turn on and off without a tool, the natural sound cues we have developed as humans uniquely map our worlds.The loudness of a sound can detect the objects size or ferocity, and the time delay difference from each ear, along with the sound wave generated by an external source is diffracted by its interaction with the head and external ears.The resulting changes provide cues about the locus of a sound relative to the head (Middlebrooks & Green 1991)."It is the business of sound to reveal for us our acoustic environment, the acoustic landscape in which we live, the speech of things, and the intimate whisperings of nature; all that has speech beyond human speech, and speaks to us with vast conversational powers of life and incessantly influences and directs our thoughts and emotions, from the muttering of the sea to the din of a great city" (Balázs 1953).

IMMERSION
How can we achieve an embodied experience that feels immersive?When listening to a story, reading a book, or watching a movie, what keeps you entranced is the quality of immersion.How well the character is defined is only a part of it -the extra information, the sounds of the environment, and the world the story builds in your mind combine to create that immersive experience.According to a paper on the emotional and cognitive effect on immersion in film, higher immersion leads to more intense emotions when connecting with a story (Visch et al. 2010).Which that link between immersion and emotions induce higher satisfaction in any context (Fornerino et al. 2008).
Immersion has been approached from multiple disciplines of expertise for a long time.The term is commonly associated with virtual reality, installation art, and video games.The common through line between the multitudes of approaches have been the connotation of being absorbed or embodied within the context.The word comes from comes from the Latin word, in (into) + mergere (plunge, dip).The origin of "immersion" suggests the literal meaning of "absorption in some interest or situation" and it is more about the sensuous experience that expands to being plunged into mental activity.Historically, many immersive works have focused on generating visual illusions exclusively -though some scientific research suggests that adding other sensory modalities (such as spatially localised sound) enhance or provide unique qualities of immersive experience (Thalmann 1997).
Char Davies, an immersive virtual reality artist, thought of immersion as "a quality of being spatially encompassed by being sensuously enveloped, as if by a lover or sea water or even the air itself pressing upon the skin" (Davies 2005).Her infamous immersive virtual reality installation Osmose integrated 3D visual elements and spatially localised sound with interaction based on breath and balance.Davies wanted to "reaffirm the priority of being in the world compared to doing things in it or to it" (Gatti 2009).
Mel Slater, one of the most prominent virtual reality researchers of immersion, understands immersion to be "a technical condition for presence in a virtual Sonic City Bhakti Duran, Jinsil Hwaryoung Seo, Brian Smith & Echo Zhao 98 environment" (Slater & Usoh 1993).Emphasising the requirement for immersion is mapping between the audience's proprioceptive feedback about the body's movements and the visual information displayed in the virtual system.

Virtual Reality
The first references to the contemporary concept of virtual reality comes from Stanley Weinbaum's short story Pygmalion's Spectacles.Mentioning a goggle-based virtual reality system with holographic recording of fictional experiences (Weinbaum 1935).Other prior hardware versions have been developed since the 1950s.The Sensorama by Morton Heilig, Ivan Sutherland's augmented reality and virtual reality head-mounted display system (precursors to today's HTC Vive and Oculus Rift), and even stereoscopic simulator's like the View-Master.Now virtual reality sits in a new paradigm of possibilities.The ability to not only trick our perception of sight into believing we are navigating a new world, but the spatial sound component accompanied by it amplifies the immersive experience.We can create aesthetically unique worlds that our physical bodies can navigate through.One less barrier of fiction our minds have to overlook.With a VR headset our sight, although not perfectly replicated, tracks our real-world head movement to mimic the reality of a threedimensional world.Along with a pair of headphone, the VR hardware uses binaural patterns to mimic our ears natural sound cues.These two senses combined meld to remove a layer of doubt in the immersive experience.Tailoring the correct balance of visual aesthetic and sound mixing amplifies the quality of immersion to a whole new degree.In an investigation of soundscape design to increase sense of place in virtual reality, the sounds alone resulted in giving the subjects a sense of place even though they are not precisely able to recognise a place by only listening to the soundscape (Serafin & Serafin 2004).

SONIC CITY
Using new technological advances in Virtual Reality, artists can now construct contemporary art experiences once thought to be science fiction.Sonic City is an immersive virtual reality installation that integrates spatial sounds recorded from various streets of Hong Kong with a visually minimalist aesthetic model of the city.A participant can navigate the virtual city with a virtual reality helmet and controller.Taking inspiration from many sources, Janet Cardiff's and George Bures Miller's Storm Room relates in the type of experience being sought by the artist.Their work emphasises aural and visual experiences that transport the viewer to other realms of consciousness -the audience either witnesses a phenomenon or becomes immersed in a scenario (Morgan-Feir 2013).

Implementation
The design started with finding visual references for not only the Hong Kong city structure, but also color schemes that resemble the city's ambiance and abstract digital drawings of cities that would give the structure of a city without being too specific of resembling recognisable objects.
Using CAD Mapper we can export an area travelled to from the Hong Kong city model in the software, which has three-dimensional information of the mapped area chosen.CAD Mapper allows a user to log into their database online and find a city CAD model they have constructed from public sources such as OpenStreeMap, NASA, and USGS.The CAD model is selected in an area km measurement that has depth, height, and street intersection information.This model is then imported into Autodesk Maya for refinement -to clean up edges and delete unneeded polygons.This model is then exported into the game engine, Unity3D, where the rest of the virtual world is built.Within Unity the city is scaled to realistic proportions for virtual reality, given a sky, distance fog, light, and shadows to begin breaking down the barriers of immersion.Then using a Houdini Engine plug-in for Unity we populate each polygon face of the city with a group of randomised squares.The squares bunched together on each polygonal plane form the abstract form of the city.Afterward, we optimise the virtual reality settings and add the SteamVR plug-in controls for Unity into the scene.This allows the virtual reality hardware to function within the created environment.From there we contact our partner in Hong Kong to record authentic sound bites of a plethora of different sources that someone would normally come across if they were to walk around the city.Along with those original recordings, we add additional sounds from digital libraries that support the believability of the city environment.Lastly, we add in animated lights that move throughout the virtual city mimicking the people walking throughout Hong Kong.These lights are not only used to represent people, but to also help the audience navigate the space by having something tangible to follow if they so choose.The virtual city is made to be exploratory.We give the audience small hints to draw them in directions if they so choose, but at all times they have the flexibility to manoeuvre around the virtual set as they like.This development process is then a matter of testing the spatial audio levels within the SteamVR platform to balance them for a quality immersive experience.

Experience
Sonic City has been shown during a monthly exhibition at Viz North in downtown Bryan, College Station and was partially presented at iDMAa (International Digital Media Arts Association).It will be presented at conferences such as ISEA (International Symposium on Electronic Arts) and EVA (Electronic Visualisation and the Arts) this summer.There are a range of positive experiences from the participating audience members of Sonic City.Most audiences really enjoyed the feeling of being immersed in a city that feels realistic in scale.Also enjoyed the unique aesthetic as they discovered the environment on their own.There are a few that truly dug down into the experience and pieced together in their minds the missing Sonic City Bhakti Duran,Jinsil Hwaryoung Seo,Brian Smith & Echo Zhao 101 information not visually seen but heard that once combined became a full picture.The audience felt the expansive environment as believable to the degree they explored based on their own curiosities.There were even a few cases of vertigo from the planar streets with no visible ground underneath them.Their ears heard specific sound sources as they moved around that, just as in any normal environment, drew them in unique paths of navigation to the participant.

DISCUSSION
We are currently working on other methods of installation that can amplify the immersive qualities within an environment sonically.Using multiple mediums to address the same concept allows the audience to interact in uniquely different ways to experience spatial sound environments -bringing their sense of hearing to the forefront for at the very least the amount time spent experiencing the installation.This body of work revolves around the perception of space through the focus of spatial sound using multiple mediums.Past examples we have created had similar outcomes with the exploration of spatial sound.Sound Painting was created using Adobe After Effects to create an impressionistic animated painting using motifs found in the works of Monet.The use of spatial sound amplified the slow movements within the animation drawing the audience in to contemplate the perception of motion within their own minds.A series of Aspect-to-Aspect Photography was also created using spatial sound to not only draw the audience toward the still frames, but contributed to the audience's contemplation on the perception of space as they saw still images of a specific area.This area was only partially show so with the assistance of the soundscape the audience filled in the blanks to feel the photographs moving in their minds.Our future iteration, Upside Down, is another case in which we use an interactive design so the audience themselves can physically revolve the spatial world around them.This action creates a tethered connection to another dimension almost.The audience finds themselves within a spatial sound environment that they also have control to manipulate.

CONCLUSION
Through works such as these we have found there is a unique approach to the way we are building experiences for the audience.The speeds that technological advances are coming at in our current times is tremendous, and the next huge wave of consumer tech will be in spatial audio which accompanies virtual reality hardware.This work can be not only a precursor to 3D sound for the audience, but a learning opportunity that will begin to open their minds to the possibilities of the world soon to come.By creating spatial sound environment installations, we are exploring the possible methods an audience can sonically engage with a work of art.These dormant qualities we overlook normally are being thrust upon the audience so they are seen.We are guiding the uninitiated audience through a passive trait.

Figure 3 :
Figure 3: Sword of Damocles by Ivan Sutherland

Figure 4 :
Figure 4: Sonic City aesthetic4.1 ConceptThe lead artist's, Bhakti Duran, photographs from a recent trip to China became the inspiration to create the installation.When the artist visited Hong Kong the city left a certain indescribable impression.Sonic City is an attempted representation of that impression -an after image of a cultural experience and a visual metaphor about what it feels like in an unfamiliar place.It is the artist's attempt to share a feeling more so than a literal representation of a real place.Through the