Self-Previewing Gestures and the Gesture-and-Effect Model: Experimentation with Responsive Visual Feedback for New and Unlearned Interactions

Multi-touch gestures embedded in touch-based interfaces and devices present challenges. It can be difficult for users to discover different gestures and understand their effects. The study reported in this paper hypothesises that presenting automatic visual prompts, termed self-previewing gestures (SPGs) in this research, that depict touch and preview gesture execution will mitigate the problems that users encounter with unfamiliar gestural interfaces. A within-subjects experiment (n=45) is reported in which an iPad application with two alternative gestural designs, and five alternative user interface versions (one industry, two research baselines, and two SPGs) was created with the purpose of making the available gestures evident. A rating system that adapts Norman’s Theory of Action to touch-based interactions by making use of known principles within interaction design (perceptible affordances, feedforward and feedback) is proposed. The system was used to assess participants’ perceptions of and interactions with the SPGs, and the results revealed positive and negative aspects of designs and UI versions.

Content

Author and article information

Contributors

Jacques Chueke

George Buchanan

Stephanie Wilson

Luis Anunciação

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 1-14

Affiliations

[0001]Centre for HCI Design

[0002]PUC-Rio

Article

DOI: 10.14236/ewic/HCI2017.34

SO-VID: 0900b093-7941-4269-92a4-9a7869d0cb37

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 31st International BCS Human Computer Interaction Conference (HCI 2017)

Conference acronym: HCI

Conference number: 31

Conference location: Sunderland, UK

Conference date: 3 - 6 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: digital make-believe, with delegates considering our expansive

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2017.34

Self URI (journal page): https://ewic.bcs.org/

References

K. B. BærentsenJ. Trettvik 2002 An activity theory approach to affordance Proceedings of NordiCHI 51 60 ACM New York, NY USA
O. BauW. Mackay 2008 OctoPocus: A dynamic guide for learning gesture-based command sets Proceedings of UIST2008 37 46 ACM New York, NY USA
M. Beaudouin-Lafon 2000 Instrumental interaction: an interaction model for designing post-wimp user interfaces Proceedings of SIGCHI conference on human factors in computing systems 446 453 ACM New York, NY USA
M. BennettK. McCarthyS. O’ModhrainB. Smyth 2011 Simpleflow: Enhancing gestural interaction with gesture prediction, abbreviation and autocompletion Proceedings of INTERACT 2011 591 608 ACM New York, NY USA
A. BragdonR. ZeleznikB. WilliamsonT. MillerJr., J. J. LaViola 2009 Gesturebar: Improving the approachability of gesture-based interfaces Proceedings of CHI ’09 2269 2278 ACM New York, NY USA
J. Chueke 2016 Perceptible affordances and feedforward for gestural interfaces: Assessing effectiveness of gesture acquisition with unfamiliar interactions Unpublished doctoral dissertation City, University of London, London UK Retrieved from http://ethos.bl.uk/OrderDetails.do?did=3&uin=uk.bl.ethos.697556
N. Cross 1999 Design research: A disciplined conversation Design Issues 15 2 5 10
A. Dam 1997 Post-WIMP user interfaces Communications of the ACM 40 2 63 67
A. DixR. ManciniS. Levialdi 1996 Alas I am undone—Reducing the risk of interaction? HCI96 adjunct proceedings 51 56 Imperial College, London ACM New York, NY USA
J. P. DjajadiningratC. J. Overbeeke 2004 Interaction Frogger: A design framework to couple action and function through feedback and feedforward Proceedings of DIS '04: the 5th conference on designing interactive systems: Processes, practices, methods, and techniques 177 184 ACM New York, NY USA
T. DjajadiningratK. OverbeekeS. Wensveen 2002 But how, Donald, tell us how? On the creation of meaning in interaction design through Feedforward and inherent Feedback Proceedings of the 4th conference on designing interactive systems: Processes, practices, methods, and techniques 285 291 ACM New York, NY USA
C. Frayling 1993 Research in art and design Royal College of Art Research Papers 1 1 1 5
D. FreemanH. BenkoM. RingelD. Wigdor 2009 ShadowGuides: Visualizations for in-situ learning of multi-touch and whole-hand gestures Proceedings of ACM Intl.Conf.on interactive tabletops and surfaces 165 172
G. FreitagM. TränknerM. Wacker 2012 Enhanced feedforward for a user aware multitouch device Proceedings of NordiCHI '12: Making sense through design 578 586 ACM New York, NY USA
W. Gaver 1991 Technology affordances Proceedings of Chi 79 84 ACM New York, NY US
J. GeiwitzJ. KornellB. McCloskey 1990 An expert system for the selection of knowledge acquisition techniques Technical Report 785-2 California Anacapa Sciences
I. GolodF. HeidrichC. MölleringM. Ziefle 2013 Design principles of hand gesture interfaces for microinteractions Proceedings. of the 6th International Conference on Designing Pleasurable Products and Interfaces—DPPI ’13 11 11 20
C. GutwinA. CockburnJ. ScarrS. MalacriaS. C. Olson 2014 Faster command selection on tablets with FastTap Proceedings of CHI '14: 32nd annual ACM conference on human factors in computing systems 2617 2626 ACM New York, NY US
H. Hartson 2003 Cognitive, physical, sensory, and functional affordances in interaction design Behaviour & Information Technology 22 5 315 338
K. HofmeesterJ. Wolfe 2012 Self-revealing gestures: Teaching new touch interactions in Windows 8 Proceedings of CHI12 extended abstracts on human factors in computing systems 815 828 ACM New York, NY USA
E. Hudlicka 1997 Summary of knowledge elicitation techniques for requirements analysis Course material for human computer interaction Worcester Polytechnic Institute
V. KaptelininB. Nardi 2012 Affordances in HCI: Toward a mediated action perspective Proceedings of CHI '12 967 976
I. KoskinenJ. ZimmermanT. BinderJ. RedstromS. Wensveen 2011 Design research through practice: From the lab, field, and showroom San Francisco, CA 1st Morgan Kaufmann Publishers Inc
J. McGrenereW. Ho 2000 Affordances: Clarifying and evolving a concept Proceedings of Graphic Interfaces 2000 179 186 Toronto, Canada Canadian Human-Computer Communications Society
D. Norman 1988 The design of everyday things Basic Books. New York, NY USA
D. Norman 1999 Affordance, conventions and design Interactions 6 3 38 43
D. Norman 2010 Natural user interfaces are not natural Interactions 17 3 6 10
D. Norman March 2012 Microsoft has a real opportunity with Windows 8 Retrieved from http://tabtimes.com/feature/ittech-os-windows/2012/03/23/don-norman-microsoft-has-real-opportunity-windows-8
D. NormanJ. Nielsen 2010 Gestural interfaces: A step backward in usability Interactions 17 5 46 49
OECD 2015 Frascati manual 2015: Guidelines for collecting and reporting data on research and experimental development Paris OECD Publishing
B. ShneidermanC. Plaisant 2010 Designing the user interface: Strategies for effective humancomputer interaction Pearson Higher Education. ACM New York, NY USA Retrieved from https://www.cs.umd.edu/users/ben/goldenrules.html
M. Sorensen 2010 Usability and affordances: Examinations of object-naming and object-task performance in haptic interfaces World Academy of Science, Engineering and Technology 63 228 233
M. SpindlerS. StellmachR. Dachselt 2009 PaperLens: Advanced magic lens interaction above the tabletop Proceedings of the ACM international conference on interactive tabletops and surfaces (ITS09) 69 76 ACM New York, NY USA
R. St. Amant 1999 User interface affordances in a planning representation Interact 14 3 317 354
P. Turner 2005 Affordance as context Interacting with Computers 17 6 787 800
J. VermeulenK. LuytenE. Van den HovenK. Coninx 2013 Proceedings of CHI '13 conference on human factors in computing systems 1931 1940 ACM New York, NY USA
D. Wigdor 2010 Architecting next-generation user interfaces Proceedings of AVI '10: The international conference on advanced visual interfaces 16 22 ACM New York, NY USA
D. WigdorD. Wixon 2011 Brave NUI world: Designing natural user interfaces for touch and gesture Morgan Kauffman Publishers Burlington, MA, USA
D. WigdorS. WilliamsM. CroninR. LevyK. WhiteM. MazeevH. Benko 2009 Ripples: Utilizing per-contact visualizations to improve user interaction with touch displays Proceedings of the 22nd Annual ACM symposium on user interface software and technology (UIST‘09) 3 12 ACM New York, NY USA
M. WuC. ShenK. RyallC. ForlinesR. Balakrishnan 2005 Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces Proceedings of TABLETOP '06: The first IEEE international workshop on horizontal interactive human-computer systems 185 192 ACM New York, NY USA
J. ZimmermanJ. ForlizziS. Evenson 2007 Research through design as a method for interaction design research in HCI Proceedings of the SIGCHI conference on human factors in computing systems 493 502

Comments

Comment on this article

[1] K. B. BærentsenJ. Trettvik 2002 An activity theory approach to affordance Proceedings of NordiCHI 51 60 ACM New York, NY USA

[2] O. BauW. Mackay 2008 OctoPocus: A dynamic guide for learning gesture-based command sets Proceedings of UIST2008 37 46 ACM New York, NY USA

[3] M. Beaudouin-Lafon 2000 Instrumental interaction: an interaction model for designing post-wimp user interfaces Proceedings of SIGCHI conference on human factors in computing systems 446 453 ACM New York, NY USA

[4] M. BennettK. McCarthyS. O’ModhrainB. Smyth 2011 Simpleflow: Enhancing gestural interaction with gesture prediction, abbreviation and autocompletion Proceedings of INTERACT 2011 591 608 ACM New York, NY USA

[5] A. BragdonR. ZeleznikB. WilliamsonT. MillerJr., J. J. LaViola 2009 Gesturebar: Improving the approachability of gesture-based interfaces Proceedings of CHI ’09 2269 2278 ACM New York, NY USA

[6] J. Chueke 2016 Perceptible affordances and feedforward for gestural interfaces: Assessing effectiveness of gesture acquisition with unfamiliar interactions Unpublished doctoral dissertation City, University of London, London UK Retrieved from http://ethos.bl.uk/OrderDetails.do?did=3&uin=uk.bl.ethos.697556

[7] N. Cross 1999 Design research: A disciplined conversation Design Issues 15 2 5 10

[8] A. Dam 1997 Post-WIMP user interfaces Communications of the ACM 40 2 63 67

[9] A. DixR. ManciniS. Levialdi 1996 Alas I am undone—Reducing the risk of interaction? HCI96 adjunct proceedings 51 56 Imperial College, London ACM New York, NY USA

[10] J. P. DjajadiningratC. J. Overbeeke 2004 Interaction Frogger: A design framework to couple action and function through feedback and feedforward Proceedings of DIS '04: the 5th conference on designing interactive systems: Processes, practices, methods, and techniques 177 184 ACM New York, NY USA

[11] T. DjajadiningratK. OverbeekeS. Wensveen 2002 But how, Donald, tell us how? On the creation of meaning in interaction design through Feedforward and inherent Feedback Proceedings of the 4th conference on designing interactive systems: Processes, practices, methods, and techniques 285 291 ACM New York, NY USA

[12] C. Frayling 1993 Research in art and design Royal College of Art Research Papers 1 1 1 5

[13] D. FreemanH. BenkoM. RingelD. Wigdor 2009 ShadowGuides: Visualizations for in-situ learning of multi-touch and whole-hand gestures Proceedings of ACM Intl.Conf.on interactive tabletops and surfaces 165 172

[14] G. FreitagM. TränknerM. Wacker 2012 Enhanced feedforward for a user aware multitouch device Proceedings of NordiCHI '12: Making sense through design 578 586 ACM New York, NY USA

[15] W. Gaver 1991 Technology affordances Proceedings of Chi 79 84 ACM New York, NY US

[16] J. GeiwitzJ. KornellB. McCloskey 1990 An expert system for the selection of knowledge acquisition techniques Technical Report 785-2 California Anacapa Sciences

[17] I. GolodF. HeidrichC. MölleringM. Ziefle 2013 Design principles of hand gesture interfaces for microinteractions Proceedings. of the 6th International Conference on Designing Pleasurable Products and Interfaces—DPPI ’13 11 11 20

[18] C. GutwinA. CockburnJ. ScarrS. MalacriaS. C. Olson 2014 Faster command selection on tablets with FastTap Proceedings of CHI '14: 32nd annual ACM conference on human factors in computing systems 2617 2626 ACM New York, NY US

[19] H. Hartson 2003 Cognitive, physical, sensory, and functional affordances in interaction design Behaviour & Information Technology 22 5 315 338

[20] K. HofmeesterJ. Wolfe 2012 Self-revealing gestures: Teaching new touch interactions in Windows 8 Proceedings of CHI12 extended abstracts on human factors in computing systems 815 828 ACM New York, NY USA

[21] E. Hudlicka 1997 Summary of knowledge elicitation techniques for requirements analysis Course material for human computer interaction Worcester Polytechnic Institute

[22] V. KaptelininB. Nardi 2012 Affordances in HCI: Toward a mediated action perspective Proceedings of CHI '12 967 976

[23] I. KoskinenJ. ZimmermanT. BinderJ. RedstromS. Wensveen 2011 Design research through practice: From the lab, field, and showroom San Francisco, CA 1st Morgan Kaufmann Publishers Inc

[24] J. McGrenereW. Ho 2000 Affordances: Clarifying and evolving a concept Proceedings of Graphic Interfaces 2000 179 186 Toronto, Canada Canadian Human-Computer Communications Society

[25] D. Norman 1988 The design of everyday things Basic Books. New York, NY USA

[26] D. Norman 1999 Affordance, conventions and design Interactions 6 3 38 43

[27] D. Norman 2010 Natural user interfaces are not natural Interactions 17 3 6 10

[28] D. Norman March 2012 Microsoft has a real opportunity with Windows 8 Retrieved from http://tabtimes.com/feature/ittech-os-windows/2012/03/23/don-norman-microsoft-has-real-opportunity-windows-8

[29] D. NormanJ. Nielsen 2010 Gestural interfaces: A step backward in usability Interactions 17 5 46 49

[30] OECD 2015 Frascati manual 2015: Guidelines for collecting and reporting data on research and experimental development Paris OECD Publishing

[31] B. ShneidermanC. Plaisant 2010 Designing the user interface: Strategies for effective humancomputer interaction Pearson Higher Education. ACM New York, NY USA Retrieved from https://www.cs.umd.edu/users/ben/goldenrules.html

[32] M. Sorensen 2010 Usability and affordances: Examinations of object-naming and object-task performance in haptic interfaces World Academy of Science, Engineering and Technology 63 228 233

[33] M. SpindlerS. StellmachR. Dachselt 2009 PaperLens: Advanced magic lens interaction above the tabletop Proceedings of the ACM international conference on interactive tabletops and surfaces (ITS09) 69 76 ACM New York, NY USA

[34] R. St. Amant 1999 User interface affordances in a planning representation Interact 14 3 317 354

[35] P. Turner 2005 Affordance as context Interacting with Computers 17 6 787 800

[36] J. VermeulenK. LuytenE. Van den HovenK. Coninx 2013 Proceedings of CHI '13 conference on human factors in computing systems 1931 1940 ACM New York, NY USA

[37] D. Wigdor 2010 Architecting next-generation user interfaces Proceedings of AVI '10: The international conference on advanced visual interfaces 16 22 ACM New York, NY USA

[38] D. WigdorD. Wixon 2011 Brave NUI world: Designing natural user interfaces for touch and gesture Morgan Kauffman Publishers Burlington, MA, USA

[39] D. WigdorS. WilliamsM. CroninR. LevyK. WhiteM. MazeevH. Benko 2009 Ripples: Utilizing per-contact visualizations to improve user interaction with touch displays Proceedings of the 22nd Annual ACM symposium on user interface software and technology (UIST‘09) 3 12 ACM New York, NY USA

[40] M. WuC. ShenK. RyallC. ForlinesR. Balakrishnan 2005 Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces Proceedings of TABLETOP '06: The first IEEE international workshop on horizontal interactive human-computer systems 185 192 ACM New York, NY USA

[41] J. ZimmermanJ. ForlizziS. Evenson 2007 Research through design as a method for interaction design research in HCI Proceedings of the SIGCHI conference on human factors in computing systems 493 502

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65