+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review


      , , , , ,

      Proceedings of the 30th International BCS Human Computer Interaction Conference (HCI)


      11 - 15 July 2016

      3D Gestures, HCI, Interaction Gestures, Mid-Air Hand Gestures, Systematic Literature Review



            3D gesture based systems are becoming ubiquitous and there are many mid-air hand gestures that exist for interacting with digital surfaces and displays. There is no well defined gesture set for 3D mid-air hand gestures which makes it difficult to develop applications that have consistent gestures. To understand what gestures exist we conducted the first comprehensive systematic literature review on mid-air hand gestures following existing researchmethods. The results of the review identified 65 paperswhere themid-air hand gestures supported tasks for selection, navigation, and manipulation. We also classified the gestures according to a gesture classification scheme and identified how these gestures have been empirically evaluated. The results of the review provide a richer understanding of what mid-air hand gestures have been designed, implemented, and evaluated in the literature which can help developers design better user experiences for digital interactive surfaces and displays.


            Author and article information

            July 2016
            July 2016
            : 1-13
            [0001]Department of Computer Science

            Middlesex University

            London, NW4 4BT, UK
            © Groenewald et al. Published by BCS Learning and Development Ltd. Proceedings of British HCI 2016 Conference Fusion, Bournemouth, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Proceedings of the 30th International BCS Human Computer Interaction Conference
            Bournemouth University, Poole, UK
            11 - 15 July 2016
            Electronic Workshops in Computing (eWiC)
            Product Information: 1477-9358BCS Learning & Development
            Self URI (journal page): https://ewic.bcs.org/
            Electronic Workshops in Computing


            1. [P1.] Cuenesics: Using mid-air gestures to select items on interactive public displays Proc. of MobileHCI 299 308 ACM 2014

            2. [P2.] Mid-air pan-and-zoom on wall-sized displays Proc. of CHI 177 186 ACM 2011

            3. [P3.] A handle bar metaphor for virtual object manipulation with mid-air interaction Proc. of CHI 1297 1306 ACM 2012

            4. [P4.] Touchfree medical interfaces Proceedings of the International Conference on Engineering in Medicine and Biology Society(EMBC) 6597 6600 IEEE 2014

            5. [P5.] Using marking menus to develop command sets for computer vision based hand gesture interfaces Proc. of NordiCHI 239 242 ACM 2002

            6. [P6.] Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus Proc. of Interact 248 262 Springer-Verlag 2011

            7. [P7.] Interactive public ambient displays: Transitioning from implicit to explicit, public to personal, interaction with multiple users Proc. of UIST, 137 146 ACM 2004

            8. [P8.] Wavewindow: Public, performative gestural interaction Proc. of ITS 109 112 ACM 2010

            9. [P9.] Mockup builder: Direct 3D modeling on and above the surface in a continuous interaction space Proc. of GI 173 180 Canadian Information Processing Society 2012

            10. [P10.] User-defined gestures for free-hand tv control Proc. of European Conference on Interactive Tv and Video (EuroiTV) 45 48 ACM 2012

            11. [P11.] Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures Int J. Hum.-Comput. Stud. 69 9): 551 562 August 2011

            12. [P12.] Exploring the usefulness of finger-based 3D gesture menu selection Proc. of CHI, 1093 1102 ACM 2014

            13. [P13.] Educational tourism through a virtual reality platform Procedia Computer Science 25 0 382 388 2013 Proc. of Conference on Virtual and Augmented Realty in Education.

            14. [P14.] How to click in mid-air editors Distributed, Ambient, and Pervasive Interactions volume 8028 of Lecture Notes in Computer Science 78 86 Springer 2013

            15. [P15.] Natural interactive 3D medical image viewer based on finger and arm gestures Proc. of Biomedical Engineering International Conference (BMEiCON) 1 5 2013

            16. [P16.] Homeproxy: Exploring a physical proxy for video communication in the home Proc. of CHI 1339–1342 ACM 2013

            17. [P17.] Informatics in radiology: Developing a touchless user interface for intraoperative image control during interventional radiology procedures RadioGraphics 33 2 E61 E70 2013

            18. [P18.] Multi-viewer gesture-based interaction for omni-directional video Proc. of CHI 4077 4086 ACM 2014

            19. [P19.] A natural user interface for navigating in organized 3D virtual contents Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments volume 8525 of Lecture Notesin Computer Science 93 104 Springer 2014

            20. [P20.] Kinect, as interaction device with a tiled display Proc. of HCI International 301 311 Springer 2013

            21. [P21.] Error behaviours in an unreliable in-air gesture recognizer Proc. of CHI EA pp. 1603–1608 ACM 2014

            22. [P22.] Airauth: Towards attack-resilient biometric authentication using in-air gestures Proc. of CHIEA pp. 1585 1590 ACM 2014

            23. [P23.] The performance of un-instrumented in-air pointing Proc. of GI pp. 59 66 Canadian Information Processing Society 2014

            24. [P24.] Touchless circular menus: Toward an intuitive ui for touchless interactions with large displays Proc. of AVI pp. 33 40 ACM 2014

            25. [P25.] Intention prediction approach to interact naturally with the microworld Proc. of Advanced Intelligent Mechatronics (AIM) pp. 396 401 IEEE July 2014

            26. [P26.] Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays Proc. of PerDis pp. 6:1 6:6 ACM 2012

            27. [P27.] Favoured attributes of in-air gestures in the home environment Proc. of OzCHI pp. 171 174 ACM 2013

            28. [P28.] A study of touchless versus touch-based interactions with bacterial biofilm images Proc. of Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI) pp. 285 290 ACM 2013

            29. [P29.] Touch-less interaction with medical images using hand & foot gestures Proc. of UbiComp pp. 1265 1274 ACM 2013

            30. [P30.] Hierarchical menu selection with a body-centered remote interface Interacting with Computers 2013

            31. [P31.] Intuitive NUI for controlling virtual objects based on hand movements Future Information Technology, volume 309 of Lecture Notes in Electrical Engineering pp. 457 461 Springer 2014

            32. [P32.] Magic: Developing a multimedia gallery supporting mid-air gesture-based interaction and control Proc. of HCI International EA pp. 303 307 Springer 2013

            33. [P33.] Prototyping a gestural interface for selecting and buying goods in a public environment Proc. of AVI pp. 784 785 ACM 2012

            34. [P34.] Wave to me: User identification using body lengths and natural gestures Proc. of CHI pp. 3453 3462 ACM 2014

            35. [P35.] Touchless gestural interaction with small displays: A case study Proc. of CHItaly, pp. 26:1 26:10 ACM 2013

            36. [P36.] Airtouch panel: A re-anchorable virtual touch panel Proc. of Multimedia pp. 625 628 ACM 2013

            37. [P37.] Selection-based mid-air text entry on large displays Proc. of Interact pp. 401 418 Springer 2013

            38. [P38.] Cognitive effects of interactive public display applications Proc.of PerDis pp. 13 18 ACM 2013

            39. [P39.] Multi-point interactions with immersive omnidirectional visualizations in a dome Proc. of ITS pp. 19 28 ACM 2010

            40. [P40] Depthtouch: Using depth-sensing camera to enable freehand interactions on and above the interactive surface. TechReport MSR-TR-2009-23, Microsoft Research 2009

            41. [P41] A study on the use of semaphoric gestures to support secondary task interactions Proc. of CHI EA pp. 1961 1964 ACM 2005

            42. [P42] A user-developed 3-D hand gesture set for human-computer interaction Human Factors: The Journal of the Human Factors and Ergonomics Society 57 4 607 621 2015

            43. [P43] Personalized interactive public screens Proc. of Workshop Interacting with Smart Objects at IUI. ACM 2015

            44. [P44] Towards usable and acceptable above-device interactions Proc. of MobileHCI pp. 459 464 ACM 2014

            45. [P45] An elicitation study on gesture preferences and memorability toward a practical hand-gesture vocabulary for smart televisions Access 3:543 555 2015

            46. [P46] GPU-based realtime hand gesture interaction and rendering for volume datasets using leap motion Proc. of Conference on Cyberworlds (CW) pp. 85 92 IEEE Oct 2014

            47. [P47] Investigating freehand pan and zoom Mensch & Computer pp. 303 312 Oldenbourg Verlag 2012

            48. [P48] Freehand interaction with large-scale 3D map data Proc. of 3DTV-Conference (3DTV-CON) pp. 1 4 IEEE 2014

            49. [P49] Freehand pose-based gestural interaction: Studies and implications for interface design Proc. of Conference on Intelligent Human Computer Interaction (IHCI) pp. 1 6 IEEE 2012

            50. [P50] Towards more natural digital content manipulation via user freehand gestural interaction in a living room Proc. of UbiComp, pp. 617 626 ACM 2013

            51. [P51] User-defined body gestures for TV-based applications Proc. of International Conference on Digital Home (ICDH) pp. 415 420 IEEE 2012

            52. [P52] Utilizing Gesture Based Interaction for Supporting Collaborative Explorations of Visualizations in TEL Proc. of Wireless, Mobile and Ubiquitous Technology in Education (WMUTE), pp. 177 181 IEEE 2012

            53. [P53] User Elicitation on Single-hand Microgestures Proc. of CHI pp. 3403 3414 ACM 2016

            54. [P54] Gunslinger: Subtle Arms-down Mid-air Interaction Proc. of UIST pp. 63 71 ACM 2015

            55. [P55] BackHand: Sensing Hand Gestures via Back of the Hand Proc. of UIST pp. 557 564 ACM 2015

            56. [P56] Interactive Hand Gesture-based Assembly for Augmented Reality Applications Proc. of Advances in Computer-Human Interactions (ACHI) pp.303 308 IARIA 2012

            57. [P57] Gesture Interaction System for Social Web Applications on Smart TVs Proc. of Open Research Areas in Information Retrieval (OAIR) pp. 225 226 Le Centre de Hautes Etudes Internationales D’Informatique Documentarie 2013

            58. [P58] Leap Gestures for TV: Insights from an Elicitation Study Proc. of Interactive Experiences for TV and Online Video (TVX) pp. 131 138 ACM 2014

            59. [P59] Implementation of a Preliminary Natural User Interface for Video on Demand Systems Future Information Technology: FutureTech pp. 475 480 Springer 2014

            60. [P60] A Gesture Interface Game for Energy Consumption Awareness Proc. of Advances in Computer Entertainment (ACE) pp. 352 367 Springer 2012

            61. [P61] User-centered approach for the definition of free-hand gestures applied to controlling music playback Multimedia Syst. 18(1): pp. 15 31 Springer 2012

            62. [P62] On free-hand TV control: experimental results on user-elicited gestures with Leap Motion Personal and Ubiquitous Computing 19 5): pp. 821 838 August 2015

            63. [P63] DETI-Interact: Interaction with Large Displays in Public Spaces Using Kinect Proc. of Distributed, Ambient, and Pervasive Interactions (DAPI) pp. 196 206 Springer 2014

            64. [P64] Instructing people for training gestural interactive systems Proc. of CHI pp. 1737 1746 ACM 2012

            65. [P65] ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data Proc. of UIST pp. 539 548 ACM 2015


            Comment on this article