Costume Embedded Haptics with Virtual Reality for Immersive Storytelling

Haptic interaction within the virtual world has been applied in novel ways for over 20 years (Srinivasan & Basdogan 1997). Over the decades there have been many variations of haptic peripherals to enhance virtual multisensory stories, gaming and experiences. For example, the recent ‘wearable’ Wolverine glove, which grasps in VR (Choi et al. 2018) or the commercially available augmented bass wearable (Drempetic & Potter 2018).


INTRODUCTION
Haptic interaction within the virtual world has been applied in novel ways for over 20 years (Srinivasan & Basdogan 1997).Over the decades there have been many variations of haptic peripherals to enhance virtual multisensory stories, gaming and experiences.For example, the recent 'wearable' Wolverine glove, which grasps in VR (Choi et al. 2018) or the commercially available augmented bass wearable (Drempetic & Potter 2018).
Since 2015, Burge's own enquiry has focused on haptic and costume as part of a multi-sensory story.The work has been demonstrated at numerous public events where she gained insights of how the components, the aesthetics of the wearable, content and overall wearability of electronics contributed to the perception of the experience.
Her initial approach was to focus on the simulated sensations from various electronic hapticsbuzzers, electro muscle stimulation, pneumatic pressure and, in the latest work, servomotors (using off the shelf hardware such as Arduinos).
After demonstrating the wearable immersive stories on the public, she realised that the sensation of touch was not just about electronic actuators simulating reality.In fact, the experience was about an overall effect in relation to the story, enhanced by the combined sensation of the fabric diffusing, and buffering the sensation of the electronic device (even over other layers of clothes).The nuanced external touch felt more authentic and dissociated from the electronic trigger.
Other factors which significantly impacted the immersive story was the garmentspecifically the styling, drape and cut of the garment including weight distribution, tightness, personal space distortion, temperature effect and crucially as a reference narrative.Strong garment aesthetics created enclothed cognition and a strong mental imagery altering self-perception (Adam, H. & Galinsky, A. 2012).In particular, the aesthetics encouraged inclusion of a wider demographic to try the new experience.
These factors led to the development of the current wearable immersive story "The Great Wave".This multi-sensory story has been awarded an EU grant, and led to industry enquiries about touch-focused design frameworks, and consumer products for the sensory impaired.

UNIT9 (innovation production studio) Electro Muscle Stimulation (EMS) Gamingwear
After developing wearables on self-initiated artistic enquiries and commissioned costume work, in 2015 Burge was employed as the Product Owner to further develop a working prototype and concept by Yifei Chai funded by InnovateUK, called the The Pretender Project (Robertson 2014); and led by UNIT9 Technical Director Yates Buckley.The haptic wearable and the body were used as a technology interface in VR (Tanenbaum & Bizzocch 2009).
In a team of technical and creatives, Burge attempted to move away from the traditional retrofuturistic exoskeleton aesthetics of gaming wear and peripherals, and instead used smart fabrics to distribute the electronics (though the Arduinocontroller and power were encased in a plastic box), with an eventual aim to scale the components from the arm across the body with minimal redesign 125 requirements and commercial constraints.Use cases explored included gaming and healthcare.

Developments
were regularly tested and demonstrated within the London gaming community; and also exhibited at tradeshows Tech Crunch 2015 London and at Heiße Scheiße Hamburg.Insightful feedback regarding the wearability, materials, overall user experience, relevance and immersiveness was gathered and improvements made for future iterations.

Bushra Burge Studio (creative innovation studio).
Costume embedded pneumatic and buzzer in for an immersive science story about dark matter.Finalist Award at the New European Media Awards (Porto 2016).Building on learnings and challenges from the previous project, Burge collaborated with engineers and a filmmaker to create an immersive haptic story, which was more comfortable and provided better cohesion across the senses.Burge diverged away from the unnatural zapping of EMS and dehumanising nature of utilitarian aesthetics of hardware peripherals.
The haptic elements were made from a hacked blood pressure machine and separate buzzing neck piece.The 360 animation used free software called Space Engine.The themed garment which held the electronics had striking aesthetics and matched the headset.The garment was designed to be multi-fit to cater for unexpected user sizes.There was an overall theme but no literal cohesion of garment and story.The slightly random, less controlled feeling of the hacked blood pressure machine created a disorientation which was more immersive than an attempt at simulation.The project gained industry attention and was shown nationally and internationally.
The VR and haptics were not integratedthis lack of synchronisation did not seem to be an issue, especially as there was no bodily sense of reference about dark matter.In fact, people formed their own sensory patterns.Feedback requested, the garment to be more present in the VR visuals, and contribute more to the narrative.Sometimes waiting in the queue to try on the experience is longer than the VR experience itself and the spectacle-aesthetic of the wearable served a purpose for a wider audience, rather than just the wearer.This waiting and seeing someone else wearing the garment created a very strong mental imagery in anticipation of the experience.
As an additional note, the fashion-aesthetics attracted mostly women to try this piece of abstract science and empowered people to give feedback on the multisensory story.Though this is not strictly a game, in the UK there are 19% of females in the gaming industry (>4% for BAME) yet 40% of games are played by women (Densham 2018).
The diverse feedback and insights gained from these previous projects was the basis for the current work -The Great Wave.

Bushra Burge Studio (creative innovation studio). Costume embedded kinetic dress with buzzers, integrated with VR.
This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 732098.Evaluation from the previous projects showed that the costume wearable was a reference for 'transformation and exploration of the pathways to narrative and ludic (playful) (Isbister & Abe 2015).and people wanted: Costume Embedded Haptics with Virtual Reality for Immersive Storytelling Bushra Burge

126
 the garment to be literally represented and abstracted in the VR story;  to see touch both virtually and in real life through the garment.
The theme of the story was inspired by Burge's artist residency and exhibition, entitled 'Rural Decay'painting and photo-collaging fallen 'sakura' (Ikisan, Japan, 2016).This led to her hand painting flowers for an interactive graphic media projection map dress installation for a King's Road fashion boutique during the Chelsea Flower Show.She then developed these flowers into fabric print and worked with her team to create kimonoesque origami folding garments making kinetics appropriate as a haptic choice.The garments were 3D rendered using Marvellous Designer, imported and abstracted further.The fabric prints have been converted into particles within the VR experience using Unity.
The haptics were produced by servomotors with a pulley system gliding the garment around and brushing the wearernot dissimilar to a Japanese paintbrush, and reinforcing the overall multi-sensory creative vision.This resulted in subtle, unnerving and nuanced touch as a sensory storytelling tool, further distanced from its electronic triggers.For the audience watching the person in the garment as a spectaclethey can see touch.
Burge's studio is developing this concept to create a more integrated story, aiming to connect smart fabrics, biometric sensors (to control the VR) and also the VR story to guide the body to move and create mechanical resistance (using stretch).
As an additional note, the piece is made with sustainable fabrics and ethical production with an east London women's charity.
Currently Burge is abstracting the brain and body from a complex entanglement of sense of self, agency and ownership.Burge hopes to explore the neuroscience perspectives with expert partners.

CONCLUSION
The Great Wave project is in progress.It has been awarded H2020 funding and led to meetings with engineering and consumer good firms.
The use cases discussed included:  multimedia consumer engagement with a wider demographic  multisensory design framework for nonvisual explanations such as turbulence  touch focused consumables to cater for either an alternative to vision impaired or enhanced touch impaired for aging populations.
Burge's bigger vision is to create a deployable creative and technical framework for inclusive immersive multisensory wearable storytelling, with a focus on haptics, VR and appropriate non-neutral fashion styled aesthetics using ethical and sustainable production.