“Twhirleds”: Spun and whirled affordances controlling multimodal mobile-ambient environments with reality distortion and synchronized lighting to preserve intuitive alignment

Bekker, T, Sturm J, Barakova E. PUC: Personal and Ubiquitious Computing. 2010; 14(5):381–3. doi: 10.1007/s00779-009-0269-9 . Design for Social Interaction through Physical Play
Brull, W, Lindt I, Herbst I, Ohlenburg J, Braun AK, Wetzel R. Towards Next-Gen Mobile AR Games. Comput Graphics Animation. 2008; 28(4):40–8. doi: 10.1109/MCG.2008.85 .
Billinghurst, M, Piumsomboon T, Bai H. Hands in space: Gesture interaction with augmented-reality interfaces. IEEE Comp. Graphics & App. 2014; 34(1):77–80. doi: 10.1109/MCG.2014.8 .
Chikashi, M. Pure Data Tutorial and Reference (in Japanese). Tokyo: Works Corporation; 2013.
Cohen, M. Integration of laptop sudden motion sensor as accelerometric control for virtual environments In: Rahardja, S, Wu E, Thalmann D, Huang Z, editors. VRCAI: Proc. ACM Int. Conf. on Virtual-Reality Continuum and Its Applications in Industry. Singapore: 2008. doi: 10.1145/1477862.1477911 .
Cohen, M, Ranaweera R, Nishimura K, Sasamoto Y, Endo S, Oyama T, Ohashi T, Nishikawa Y, Kanno R, Nakada A, Villegas J, Chen YP, Holesch S, Yamadera J, Ito H, Saito Y, Sasaki A. “Tworlds”: Twirled Worlds for Multimodal ‘Padiddle’ Spinning & Tethered ‘Poi’ Whirling. In: SIGGRAPH. Anaheim: 2013. doi: 10.1145/2503385.2503459 . http://www.youtube.com/watch?v=sKruOxXJBNU .
Cohen, M, Villegas J. Applications of audio augmented reality: Wearware, everyware, anyware, & awareware In: Barfield, W, editor. Fundamentals of Wearable Computers and Augmented Reality. 2nd ed. Mahwah: CRC Press: Lawrence Erlbaum Associates: 2016. p. 309–30. doi: 10.1201/b18703-17 . ISBN 978-750-4822-4350-5, 978-1-4822-4351-2
Cohen, M. Quantity of presence: Beyond person, number, and pronouns In: Kunii, TL, Luciani A, editors. Cyberworlds. Chap. 19. Tokyo: Springer: 1998. p. 289–308. doi: 10.1007/978-4-431-67941-7_19 .
Cohen, M, Jayasingha I, Villegas J. Spin-around: Phase-locked synchronized rotation and revolution in a multistandpoint panoramic browser In: Miyazaki, T, Paik I, Wei D, editors. Proc. CIT: 7th Int. Conf. on Computer and Information Technology. Aizu-Wakamatsu: 2007. p. 511–6. doi: 10.1109/CIT.2007.141 .
Cohen, M, Sasa K. An interface for a soundscape-stabilized spiral-spring swivel-seat In: Kuwano, S, Kato T, editors. Proc. WESTPRAC VII: 7Th Western Pacific Regional Acoustics Conf. Kumamoto: 2000. p. 321–4.
Cohen, M. Poi Poi: Point-of-Interest Poi for Multimodal Tethered Whirling. In: MobileHCI: Proc. 14th Int. Conf. on Human-Computer Interaction with Mobile Devices and Services. San Francisco: 2012. p. 199–202. doi: 10.1145/2371664.2371709 .
Cohen, M. The Internet Chair. IJHCI: Int J Human-Comput Interact. 2003; 15(2):297–311. doi: 10.1207/S15327590IJHC1502_7 .
Cohen, M. Demo: Smartphone Rigging with GUI Control Emulation for Freeware Rapid Prototyping of Mixed Virtuality Scenes. In: SIGGRAPH Asia Symp. on Mobile Graphics and Interactive Applications. Macao: 2016. doi: 10.1145/2999508.2999511 .
Cohen, M, Ranaweera R, Ryskeldiev B, Oyama T, Hashimoto A, Tsukida N, Toshimune M. Multimodal mobile-ambient transmedial twirling with environmental lighting to complement fluid perspective with phase-perturbed affordance projection. In: SIGGRAPH Asia Symp. on Mobile Graphics and Interactive Applications. Shenzhen: 2014. doi: 10.1145/2669062.2669080 .
Cohen, M, Oyama T. Exocentric Rendering of “Reality Distortion” User Interface to Illustrate Egocentric Reprojection. In: Proc. SUI: ACM Symp. on Spatial User Interaction. Los Angeles: 2015. p. 130. doi: 10.1145/2788940.2794357. Poster demonstration . Poster demonstration
Cooperstock, JR, Fels SS, Buxton W, Smith KC. Reactive environments: Throwing away your keyboard and mouse. Commun Acm. 1997; 40(9):65–73. doi: 10.1145/260750.260774 .
Cohen, M. Dimensions of spatial sound and interface styles of audio augmented reality: Whereware, wearware, & everyware In: Barfield, W, editor. Fundamentals of Wearable Computers and Augmented Reality. Chap. 12. 2nd ed. Mahwah: CRC Press: Lawrence Erlbaum Associates: 2016. p. 277–308. doi: 10.1201/b18703-16 . ISBN 978-750-4822-4350-5, 978-1-4822-4351-2
Cohen, M, Győrbiró N. Personal and portable, plus practically panoramic: Mobile and ambient display and control of virtual worlds. Innov Mag Singap Mag Res Technol Educ. 2008; 8(3):33–5.
Dann, WP, Cooper SP, Ericson B. Exploring Wonderland: Java Programming Using Alice and Media Computation. Upper Saddle River: Pearson; 2010.
Fernando, ONN, Adachi K, Duminduwardena U, Kawaguchi M, Cohen M. Audio Narrowcasting and Privacy for Multipresent Avatars on Workstations and Mobile Phones. Ieice Trans Inf Syst. 2006; E89-D(1):73–87. doi: 10.1093/ietisy/e89-d.1.73 . http://search.ieice.org/bin/summary.php?id=e89-d_1_73&category=D&year=2006&lang=E . http://i-scover.ieice.org/iscover/page/ARTICLE_TRAN_E89-D_1_73 .
Ghosh, A, Trentacoste M, Seetzen H, Heidrich W. Real illumination from virtual environments. In: Proc. Eurographics Symp. on Rendering. Konstanz: 2005. p. 243–52. doi: 10.1145/1187112.1187161 .
Groh, B, Fleckenstein M, Eskofier B. Wearable Trick Classification in Freestyle Snowboarding. In: IEEE EMBS 13th Annual Int. Body Sensor Networks Conf. San Francisco: 2016. p. 89–93. doi: 10.1109/BSN.2016.7516238 . https://www5.informatik.uni-erlangen.de/Forschung/Publikationen/2016/Groh16-WTC.pdf .
He, Z. ERA- Intersection of Time; The Journey of Chinese Acrobats. Shanghai: Shanghai People’s Publishing House; 2009. Translated by Haiming Liu and Hui Ma.
Ishii, H. Tangible bits. IPSJ Magazine. 1998; 39(8):745–51. (In Japanese).
Ishii, H, Ullmer B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In: Proc. CHI: Conf. on Computer-Human Interaction. Atlanta: 1997. p. 234–41. doi: 10.1145/258549.258715 .
Kanno, T, Cohen M, Nagashima Y, Hoshino T. Mobile control of multimodal groupware in a distributed virtual environment In: Tachi, S, Hirose M, Nakatsu R, Takemura H, editors. Proc. ICAT: Int. Conf. on Artificial Reality and Telexistence. University of Tokyo: Tokyo: 2001. p. 147–54.
Kaplan, J, Yankelovich N. Open wonderland: an extensible virtual world architecture. IEEE Internet Comput. 2011; 15(5):38–45. doi: 10.1109/MIC.2011.76 .
Kostakos, V, Ojala T. Public displays invade urban spaces. IEEE Pervasive Comput. 2013; 12(1):8–13. doi: 10.1109/MPRV.2013.15 .
Matsumura, S. Pd Recipe Book— Introduction to Sound Programming with Pure Data (in Japanese). Tokyo: BNN; 2012.
Memarovic, N. Understanding future challenges for networked public display systems in community settings. In: Proc. 7th Int. Conf. on Communities and Technologies. Limerick: ACM: 2015. p. 39–48. doi: 10.1145/2768545.2768559 . ISBN 978-1-4503-3460-0
Milgram, P, Coquhoun Jr H. A Taxonomy of Real and Virtual World Display Integration In: Ohta, Y, Tamura H, editors. Mixed Reality: Merging Real and Virtual Worlds. Chap. 1. Secaucus, Berlin, Heidelberg: Springer-Verlag: 1999. p. 5–30.
Milgram, P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. 1994; E77-D(12):1321–9.
Olsen, VS. Alice 3 Cookbook. Birmingham: Packt Publishing Ltd; 2011.
Pedersen, ER, Sokoler T, Nelson L. Paperbuttons: expanding a tangible user interface. In: DIS: Proc. 3rd Conf. on Designing Interactive Systems. New York: ACM: 2000. p. 216–23. doi: 10.1145/347642.347723 .
Pulkki, V. Virtual source positioning using vector base amplitude panning. J Aud Eng Soc. 1997; 45(6):456–66.
Pulkki, V, Lokki T, Rocchesso D. Spatial effects In: Zölzer, U, editor. DAFX: Digital Audio Effects. 2nd ed. Wiley: 2011. p. 139–84. Chap. 5.
Ranaweera, R, Cohen M. Gestural Interface for Conducting Virtual Concerts. IEEJ Trans. Electron Inf Syst (C). 2016; 136(11):1567–73. doi: 10.1541/ieejeiss.136.1567 .
Rekimoto, J. Organic interaction technologies: From stone to skin. Commun Acm. 2008; 51(6):38–44. doi: 10.1145/1349026.1349035 .
Silva, JM, El-Saddik A. Exertion interfaces for computer videogames using smartphones as input controllers. Multimedia Sys. 2013; 19(3):289–302. doi: 10.1007/s00530-012-0268-y .
Sra, M, Schmandt C. Expanding social mobile games beyond the device screen. Pers Ubiquit Comput. 2015; 19(3-4):495–508. doi: 10.1007/s00779-015-0845-0 .
Steenblik, RA. Chromastereoscopy In: McAllister, DF, editor. Stereo Computer Graphics and Other True 3D Technologies. Princeton: Princeton University Press: 1993. p. 183–95.
Streitz, N, Kameas A, Mavrommati I. The Disappearing Computer: Interaction Design, System Infrastructures and Applications for Smart Environments. Berlin: State-of-the-Art Survey, Springer LNCS 4500; 2007. doi: 10.1007/978-3-540-72727-9 .
van Dam, A. Visualization research problems in next-generation educational software. IEEE Comput Graphics Appl. 2005; 25(5):88–92. doi: 10.1109/MCG.2005.118 .
van den Hoven, E, van de Garde-Perik E, Offermans S, van Boerdonk K, Lenssen K-MH. Moving tangible interaction systems to the next level. Computer. 2013; 46(8):70–6. doi: 10.1109/MC.2012.360 .

Comments

Comment on this article

[1] Bekker, T, Sturm J, Barakova E. PUC: Personal and Ubiquitious Computing. 2010; 14(5):381–3. doi: 10.1007/s00779-009-0269-9 . Design for Social Interaction through Physical Play

[2] Brull, W, Lindt I, Herbst I, Ohlenburg J, Braun AK, Wetzel R. Towards Next-Gen Mobile AR Games. Comput Graphics Animation. 2008; 28(4):40–8. doi: 10.1109/MCG.2008.85 .

[3] Billinghurst, M, Piumsomboon T, Bai H. Hands in space: Gesture interaction with augmented-reality interfaces. IEEE Comp. Graphics & App. 2014; 34(1):77–80. doi: 10.1109/MCG.2014.8 .

[4] Chikashi, M. Pure Data Tutorial and Reference (in Japanese). Tokyo: Works Corporation; 2013.

[5] Cohen, M. Integration of laptop sudden motion sensor as accelerometric control for virtual environments In: Rahardja, S, Wu E, Thalmann D, Huang Z, editors. VRCAI: Proc. ACM Int. Conf. on Virtual-Reality Continuum and Its Applications in Industry. Singapore: 2008. doi: 10.1145/1477862.1477911 .

[6] Cohen, M, Ranaweera R, Nishimura K, Sasamoto Y, Endo S, Oyama T, Ohashi T, Nishikawa Y, Kanno R, Nakada A, Villegas J, Chen YP, Holesch S, Yamadera J, Ito H, Saito Y, Sasaki A. “Tworlds”: Twirled Worlds for Multimodal ‘Padiddle’ Spinning & Tethered ‘Poi’ Whirling. In: SIGGRAPH. Anaheim: 2013. doi: 10.1145/2503385.2503459 . http://www.youtube.com/watch?v=sKruOxXJBNU .

[7] Cohen, M, Villegas J. Applications of audio augmented reality: Wearware, everyware, anyware, & awareware In: Barfield, W, editor. Fundamentals of Wearable Computers and Augmented Reality. 2nd ed. Mahwah: CRC Press: Lawrence Erlbaum Associates: 2016. p. 309–30. doi: 10.1201/b18703-17 . ISBN 978-750-4822-4350-5, 978-1-4822-4351-2

[8] Cohen, M. Quantity of presence: Beyond person, number, and pronouns In: Kunii, TL, Luciani A, editors. Cyberworlds. Chap. 19. Tokyo: Springer: 1998. p. 289–308. doi: 10.1007/978-4-431-67941-7_19 .

[9] Cohen, M, Jayasingha I, Villegas J. Spin-around: Phase-locked synchronized rotation and revolution in a multistandpoint panoramic browser In: Miyazaki, T, Paik I, Wei D, editors. Proc. CIT: 7th Int. Conf. on Computer and Information Technology. Aizu-Wakamatsu: 2007. p. 511–6. doi: 10.1109/CIT.2007.141 .

[10] Cohen, M, Sasa K. An interface for a soundscape-stabilized spiral-spring swivel-seat In: Kuwano, S, Kato T, editors. Proc. WESTPRAC VII: 7Th Western Pacific Regional Acoustics Conf. Kumamoto: 2000. p. 321–4.

[11] Cohen, M. Poi Poi: Point-of-Interest Poi for Multimodal Tethered Whirling. In: MobileHCI: Proc. 14th Int. Conf. on Human-Computer Interaction with Mobile Devices and Services. San Francisco: 2012. p. 199–202. doi: 10.1145/2371664.2371709 .

[12] Cohen, M. The Internet Chair. IJHCI: Int J Human-Comput Interact. 2003; 15(2):297–311. doi: 10.1207/S15327590IJHC1502_7 .

[13] Cohen, M. Demo: Smartphone Rigging with GUI Control Emulation for Freeware Rapid Prototyping of Mixed Virtuality Scenes. In: SIGGRAPH Asia Symp. on Mobile Graphics and Interactive Applications. Macao: 2016. doi: 10.1145/2999508.2999511 .

[14] Cohen, M, Ranaweera R, Ryskeldiev B, Oyama T, Hashimoto A, Tsukida N, Toshimune M. Multimodal mobile-ambient transmedial twirling with environmental lighting to complement fluid perspective with phase-perturbed affordance projection. In: SIGGRAPH Asia Symp. on Mobile Graphics and Interactive Applications. Shenzhen: 2014. doi: 10.1145/2669062.2669080 .

[15] Cohen, M, Oyama T. Exocentric Rendering of “Reality Distortion” User Interface to Illustrate Egocentric Reprojection. In: Proc. SUI: ACM Symp. on Spatial User Interaction. Los Angeles: 2015. p. 130. doi: 10.1145/2788940.2794357. Poster demonstration . Poster demonstration

[16] Cooperstock, JR, Fels SS, Buxton W, Smith KC. Reactive environments: Throwing away your keyboard and mouse. Commun Acm. 1997; 40(9):65–73. doi: 10.1145/260750.260774 .

[17] Cohen, M. Dimensions of spatial sound and interface styles of audio augmented reality: Whereware, wearware, & everyware In: Barfield, W, editor. Fundamentals of Wearable Computers and Augmented Reality. Chap. 12. 2nd ed. Mahwah: CRC Press: Lawrence Erlbaum Associates: 2016. p. 277–308. doi: 10.1201/b18703-16 . ISBN 978-750-4822-4350-5, 978-1-4822-4351-2

[18] Cohen, M, Győrbiró N. Personal and portable, plus practically panoramic: Mobile and ambient display and control of virtual worlds. Innov Mag Singap Mag Res Technol Educ. 2008; 8(3):33–5.

[19] Dann, WP, Cooper SP, Ericson B. Exploring Wonderland: Java Programming Using Alice and Media Computation. Upper Saddle River: Pearson; 2010.

[20] Fernando, ONN, Adachi K, Duminduwardena U, Kawaguchi M, Cohen M. Audio Narrowcasting and Privacy for Multipresent Avatars on Workstations and Mobile Phones. Ieice Trans Inf Syst. 2006; E89-D(1):73–87. doi: 10.1093/ietisy/e89-d.1.73 . http://search.ieice.org/bin/summary.php?id=e89-d_1_73&category=D&year=2006&lang=E . http://i-scover.ieice.org/iscover/page/ARTICLE_TRAN_E89-D_1_73 .

[21] Ghosh, A, Trentacoste M, Seetzen H, Heidrich W. Real illumination from virtual environments. In: Proc. Eurographics Symp. on Rendering. Konstanz: 2005. p. 243–52. doi: 10.1145/1187112.1187161 .

[22] Groh, B, Fleckenstein M, Eskofier B. Wearable Trick Classification in Freestyle Snowboarding. In: IEEE EMBS 13th Annual Int. Body Sensor Networks Conf. San Francisco: 2016. p. 89–93. doi: 10.1109/BSN.2016.7516238 . https://www5.informatik.uni-erlangen.de/Forschung/Publikationen/2016/Groh16-WTC.pdf .

[23] He, Z. ERA- Intersection of Time; The Journey of Chinese Acrobats. Shanghai: Shanghai People’s Publishing House; 2009. Translated by Haiming Liu and Hui Ma.

[24] Ishii, H. Tangible bits. IPSJ Magazine. 1998; 39(8):745–51. (In Japanese).

[25] Ishii, H, Ullmer B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In: Proc. CHI: Conf. on Computer-Human Interaction. Atlanta: 1997. p. 234–41. doi: 10.1145/258549.258715 .

[26] Kanno, T, Cohen M, Nagashima Y, Hoshino T. Mobile control of multimodal groupware in a distributed virtual environment In: Tachi, S, Hirose M, Nakatsu R, Takemura H, editors. Proc. ICAT: Int. Conf. on Artificial Reality and Telexistence. University of Tokyo: Tokyo: 2001. p. 147–54.

[27] Kaplan, J, Yankelovich N. Open wonderland: an extensible virtual world architecture. IEEE Internet Comput. 2011; 15(5):38–45. doi: 10.1109/MIC.2011.76 .

[28] Kostakos, V, Ojala T. Public displays invade urban spaces. IEEE Pervasive Comput. 2013; 12(1):8–13. doi: 10.1109/MPRV.2013.15 .

[29] Matsumura, S. Pd Recipe Book— Introduction to Sound Programming with Pure Data (in Japanese). Tokyo: BNN; 2012.

[30] Memarovic, N. Understanding future challenges for networked public display systems in community settings. In: Proc. 7th Int. Conf. on Communities and Technologies. Limerick: ACM: 2015. p. 39–48. doi: 10.1145/2768545.2768559 . ISBN 978-1-4503-3460-0

[31] Milgram, P, Coquhoun Jr H. A Taxonomy of Real and Virtual World Display Integration In: Ohta, Y, Tamura H, editors. Mixed Reality: Merging Real and Virtual Worlds. Chap. 1. Secaucus, Berlin, Heidelberg: Springer-Verlag: 1999. p. 5–30.

[32] Milgram, P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst. 1994; E77-D(12):1321–9.

[33] Olsen, VS. Alice 3 Cookbook. Birmingham: Packt Publishing Ltd; 2011.

[34] Pedersen, ER, Sokoler T, Nelson L. Paperbuttons: expanding a tangible user interface. In: DIS: Proc. 3rd Conf. on Designing Interactive Systems. New York: ACM: 2000. p. 216–23. doi: 10.1145/347642.347723 .

[35] Pulkki, V. Virtual source positioning using vector base amplitude panning. J Aud Eng Soc. 1997; 45(6):456–66.

[36] Pulkki, V, Lokki T, Rocchesso D. Spatial effects In: Zölzer, U, editor. DAFX: Digital Audio Effects. 2nd ed. Wiley: 2011. p. 139–84. Chap. 5.

[37] Ranaweera, R, Cohen M. Gestural Interface for Conducting Virtual Concerts. IEEJ Trans. Electron Inf Syst (C). 2016; 136(11):1567–73. doi: 10.1541/ieejeiss.136.1567 .

[38] Rekimoto, J. Organic interaction technologies: From stone to skin. Commun Acm. 2008; 51(6):38–44. doi: 10.1145/1349026.1349035 .

[39] Silva, JM, El-Saddik A. Exertion interfaces for computer videogames using smartphones as input controllers. Multimedia Sys. 2013; 19(3):289–302. doi: 10.1007/s00530-012-0268-y .

[40] Sra, M, Schmandt C. Expanding social mobile games beyond the device screen. Pers Ubiquit Comput. 2015; 19(3-4):495–508. doi: 10.1007/s00779-015-0845-0 .

[41] Steenblik, RA. Chromastereoscopy In: McAllister, DF, editor. Stereo Computer Graphics and Other True 3D Technologies. Princeton: Princeton University Press: 1993. p. 183–95.

[42] Streitz, N, Kameas A, Mavrommati I. The Disappearing Computer: Interaction Design, System Infrastructures and Applications for Smart Environments. Berlin: State-of-the-Art Survey, Springer LNCS 4500; 2007. doi: 10.1007/978-3-540-72727-9 .

[43] van Dam, A. Visualization research problems in next-generation educational software. IEEE Comput Graphics Appl. 2005; 25(5):88–92. doi: 10.1109/MCG.2005.118 .

[44] van den Hoven, E, van de Garde-Perik E, Offermans S, van Boerdonk K, Lenssen K-MH. Moving tangible interaction systems to the next level. Computer. 2013; 46(8):70–6. doi: 10.1109/MC.2012.360 .