Augmenting User-Maintained Interaction Through Mode Locking and Reversing

User-maintained mode is often adopted in facilitating a bimanual interaction using multitouch devices, in which a thumb of a non-dominant hand holding the device can press a part of the screen and keep pressed in order to activate and maintain a temporary mode that is required only for a short period of time, until the user releases the thumb. In this paper, we characterise this particular interactivity and propose novel locking mechanism that supports easy swapping between primary and secondary modes enabling further useful bimanual interaction mechanism that we call reversed mode relationship. We design three prototypes that employ this interactivity and conduct a usability testing with 27 participants. Findings suggest that this interaction works better in applications where there are frequent but short and non-intensive sub-tasks that interrupt the main workflow. We use this insight to further explore more advanced ways to support such interaction.

Content

Author and article information

Contributors

Katherine Fennedy

Hyowon Lee

Insuk Ko

Yi Ren Tan

Dongliang Zhang

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 1-12

Affiliations

[0001]Singapore University of Technology and Design 8 Somapah Road, Singapore 487372

[0002]International Design Institute Zhejiang University Hangzhou, China 300058

Article

DOI: 10.14236/ewic/HCI2018.9

SO-VID: abfa0327-abef-4e30-91cb-90ab3b7697e6

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 32

Conference location: Belfast, UK

Conference date: 4 - 6 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Human Computer Interaction Conference

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.9

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

P. BaudischG. Chu 2009 Back-of-device interaction allows creating very small touch devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Boston, MA, USA. ACM
A. Degani 1999 Modes in Human-Machine Systems: Constructs, Representation, and Classification. The International journal of aviation psychology. 9 2 125
d. FoucaultM. MicauxD. BonnetM. Beaudouin-Lafon 2014 SPad: a bimanual interaction technique for productivity applications on multi-touch tablets. CHI ’14 Extended Abstracts on Human Factors in Computing Systems Toronto, Ontario, Canada ACM
Y. Guiard 1987 Asymmetric Division of Labor in Human Skilled Bimanual Action. Journal of Motor Behavior 19 4 486 517
C. GutwinA. CockburnJ. ScarrS. MalacriaS. C. Olson 2014 Faster command selection on tablets with FastTap. Proceedings of the 32nd annual ACM conference on Human factors in computing systems Toronto, Ontario, Canada. ACM
W. HamiltonA. KerneT. Robbins 2012 High-performance pen + touch modality interactions: a real-time strategy game eSports context. Proceedings of the 25th annual ACM symposium on User interface software and technology Cambridge, Massachusetts, USA. ACM
K. HinckleyF. GuimbretiereP. BaudischR. SarinM. AgrawalaE. Cutrell 2006 The Springboard: Multiple Modes in One Spring-Loaded Control. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Montreal, Quebec, Canada, 181-190. ACM 1124801
C. HolzP. Baudisch 2010 The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta, Georgia, USA. ACM
InstagramPress 2016 Introducing Instagram Stories http://instagram-press.com/blog/2016/08/02/introducing-instagram-stories/ (14th March 2018)
J. JohnsonG. Engelbeck 1989 Modes Survey Results. SIGCHI Bull. 20 4 38 50
J. Johnson 1990 Modes in non-computer devices. International Journal of Man-Machine Studies 32 4 423 438
J. KleimolaM. LaineE. LitvinovaP. Vuorimaa 2013 TouchModifier: enriched multi-touch gestures for tablet browsers. Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces St. Andrews, Scotland, United Kingdom. ACM
E. LankJ. RuizW. Cowan 2006 Concurrent bimanual stylus interaction: a study of non-preferred hand mode manipulation. Proceedings of Graphics Interface 2006 Quebec, Canada. Canadian Information Processing Society
H. V. LeS. MayerP. BaderF. BastianN. Henze 2017 Interaction Methods and Use Cases for a Full-Touch Sensing Smartphone. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems Denver, Colorado, USA. ACM
Y. LiK. HinckleyZ. GuanJ. A. Landay 2005 Experimental Analysis of Mode Switching Techniques in Pen-Based User Interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Portland, Oregon, USA. ACM
F. MatulicM. C. Norrie 2012 Supporting active reading on pen and touch-operated tabletops. Proceedings of the International Working Conference on Advanced Visual Interfaces Capri Island, Italy. ACM
F. MatulicM. C. Norrie 2013 Pen and Touch Gestural Environment for Document Editing on Interactive Tabletops. Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces St. Andrews, Scotland, United Kingdom, 41-50. ACM 2512802
D. OdellV. Chandrasekaran 2012 Enabling comfortable thumb interaction in tablet computers: a Windows 8 case study. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 56 1 1907 1911
A. OulasvirtaA. ReichelW. LiY. ZhangM. BachynskyiK. VertanenP. O. Kristensson 2013 Improving two-thumb text entry on touchscreen devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Paris, France. ACM
K. PfeufferK. HinckleyM. PahudB. Buxton 2017 Thumb + Pen Interaction on Tablets. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems Denver, Colorado, USA. ACM
M. F. PollerS. K. Garter 1983 A Comparative Study of Moded and Modeless Text Editing by Experienced Editor Users. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Boston, Massachusetts, USA, 166-170. ACM 801603
J. Raskin 2000 The Humane Interface: New Directions for Designing Interactive Systems ACM Press/Addison-Wesley Publishing Co
J. RodC. LiD. ZhangH. Lee 2016 Designing a 3D modelling tool for novice users. Proceedings of the 28th Australian Conference on Computer-Human Interaction Launceston, Tasmania, Australia. ACM
D. Saffer 2013 Microinteractions: Designing with Details O’Reilly Media, Inc
E. SaundE. Lank 2003 Stylus Input and Editing Without Prior Selection of Mode. Proceedings of the 16th annual ACM symposium on User interface software and technology Vancouver, Canada, 213-216. ACM 964720
A. J. SellenG. P. KurtenbachW. A. S. Buxton 1992 The Prevention of Mode Errors Through Sensory Feedback. Hum.-Comput. Interact. 7 2 141 164
K. A. SiekY. RogersK. H. Connelly 2005 Fat finger worries: how older and younger users physically interact with PDAs. Proceedings of the 2005 IFIP TC13 international conference on Human-Computer Interaction Rome, Italy. Springer-Verlag
H. B. SuraleF. MatulicD. Vogel 2017 Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces 3267 3280
M. S. UddinC. GutwinB. Lafreniere 2016 HandMark Menus: Rapid Command Selection and Large Command Sets on Multi-Touch Displays. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems Santa Clara, California, USA. ACM
D. VogelP. Baudisch 2007 Shift: a technique for operating pen-based interfaces using touch. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems San Jose, California, USA. ACM
D. VogelR. Balakrishnan 2010 Occlusion-aware interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta, Georgia, USA. ACM
WABetaInfo 2017 WhatsApp to support locked recordings http://wabetainfo.com/whatsapp-to-support-locked-recordings/ (14th March 2018)
J. Wagnerp. St, HuotW. Mackay 2012 BiTouch and BiPad: designing bimanual interaction for hand-held tablets. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Austin, Texas, USA. ACM
D. WigdorC. ForlinesP. BaudischJ. BarnwellC. Shen 2007 Lucid touch: a see-through mobile device. Proceedings of the 20th annual ACM symposium on User interface software and technology Newport, Rhode Island, USA. ACM
D. D. WoodsL. J. JohannesenR. I. CookN. B. Sarter 1994 Behind human error: Cognitive systems, computers and hindsight. DAYTON UNIV RESEARCH INST (URDI) OH
H. XiaT. GrossmanG. Fitzmaurice 2015 NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology Charlotte, NC, USA. ACM

Comments

Comment on this article

[1] P. BaudischG. Chu 2009 Back-of-device interaction allows creating very small touch devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Boston, MA, USA. ACM

[2] A. Degani 1999 Modes in Human-Machine Systems: Constructs, Representation, and Classification. The International journal of aviation psychology. 9 2 125

[3] d. FoucaultM. MicauxD. BonnetM. Beaudouin-Lafon 2014 SPad: a bimanual interaction technique for productivity applications on multi-touch tablets. CHI ’14 Extended Abstracts on Human Factors in Computing Systems Toronto, Ontario, Canada ACM

[4] Y. Guiard 1987 Asymmetric Division of Labor in Human Skilled Bimanual Action. Journal of Motor Behavior 19 4 486 517

[5] C. GutwinA. CockburnJ. ScarrS. MalacriaS. C. Olson 2014 Faster command selection on tablets with FastTap. Proceedings of the 32nd annual ACM conference on Human factors in computing systems Toronto, Ontario, Canada. ACM

[6] W. HamiltonA. KerneT. Robbins 2012 High-performance pen + touch modality interactions: a real-time strategy game eSports context. Proceedings of the 25th annual ACM symposium on User interface software and technology Cambridge, Massachusetts, USA. ACM

[7] K. HinckleyF. GuimbretiereP. BaudischR. SarinM. AgrawalaE. Cutrell 2006 The Springboard: Multiple Modes in One Spring-Loaded Control. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Montreal, Quebec, Canada, 181-190. ACM 1124801

[8] C. HolzP. Baudisch 2010 The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta, Georgia, USA. ACM

[9] InstagramPress 2016 Introducing Instagram Stories http://instagram-press.com/blog/2016/08/02/introducing-instagram-stories/ (14th March 2018)

[10] J. JohnsonG. Engelbeck 1989 Modes Survey Results. SIGCHI Bull. 20 4 38 50

[11] J. Johnson 1990 Modes in non-computer devices. International Journal of Man-Machine Studies 32 4 423 438

[12] J. KleimolaM. LaineE. LitvinovaP. Vuorimaa 2013 TouchModifier: enriched multi-touch gestures for tablet browsers. Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces St. Andrews, Scotland, United Kingdom. ACM

[13] E. LankJ. RuizW. Cowan 2006 Concurrent bimanual stylus interaction: a study of non-preferred hand mode manipulation. Proceedings of Graphics Interface 2006 Quebec, Canada. Canadian Information Processing Society

[14] H. V. LeS. MayerP. BaderF. BastianN. Henze 2017 Interaction Methods and Use Cases for a Full-Touch Sensing Smartphone. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems Denver, Colorado, USA. ACM

[15] Y. LiK. HinckleyZ. GuanJ. A. Landay 2005 Experimental Analysis of Mode Switching Techniques in Pen-Based User Interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Portland, Oregon, USA. ACM

[16] F. MatulicM. C. Norrie 2012 Supporting active reading on pen and touch-operated tabletops. Proceedings of the International Working Conference on Advanced Visual Interfaces Capri Island, Italy. ACM

[17] F. MatulicM. C. Norrie 2013 Pen and Touch Gestural Environment for Document Editing on Interactive Tabletops. Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces St. Andrews, Scotland, United Kingdom, 41-50. ACM 2512802

[18] D. OdellV. Chandrasekaran 2012 Enabling comfortable thumb interaction in tablet computers: a Windows 8 case study. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 56 1 1907 1911

[19] A. OulasvirtaA. ReichelW. LiY. ZhangM. BachynskyiK. VertanenP. O. Kristensson 2013 Improving two-thumb text entry on touchscreen devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Paris, France. ACM

[20] K. PfeufferK. HinckleyM. PahudB. Buxton 2017 Thumb + Pen Interaction on Tablets. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems Denver, Colorado, USA. ACM

[21] M. F. PollerS. K. Garter 1983 A Comparative Study of Moded and Modeless Text Editing by Experienced Editor Users. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Boston, Massachusetts, USA, 166-170. ACM 801603

[22] J. Raskin 2000 The Humane Interface: New Directions for Designing Interactive Systems ACM Press/Addison-Wesley Publishing Co

[23] J. RodC. LiD. ZhangH. Lee 2016 Designing a 3D modelling tool for novice users. Proceedings of the 28th Australian Conference on Computer-Human Interaction Launceston, Tasmania, Australia. ACM

[24] D. Saffer 2013 Microinteractions: Designing with Details O’Reilly Media, Inc

[25] E. SaundE. Lank 2003 Stylus Input and Editing Without Prior Selection of Mode. Proceedings of the 16th annual ACM symposium on User interface software and technology Vancouver, Canada, 213-216. ACM 964720

[26] A. J. SellenG. P. KurtenbachW. A. S. Buxton 1992 The Prevention of Mode Errors Through Sensory Feedback. Hum.-Comput. Interact. 7 2 141 164

[27] K. A. SiekY. RogersK. H. Connelly 2005 Fat finger worries: how older and younger users physically interact with PDAs. Proceedings of the 2005 IFIP TC13 international conference on Human-Computer Interaction Rome, Italy. Springer-Verlag

[28] H. B. SuraleF. MatulicD. Vogel 2017 Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces 3267 3280

[29] M. S. UddinC. GutwinB. Lafreniere 2016 HandMark Menus: Rapid Command Selection and Large Command Sets on Multi-Touch Displays. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems Santa Clara, California, USA. ACM

[30] D. VogelP. Baudisch 2007 Shift: a technique for operating pen-based interfaces using touch. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems San Jose, California, USA. ACM

[31] D. VogelR. Balakrishnan 2010 Occlusion-aware interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Atlanta, Georgia, USA. ACM

[32] WABetaInfo 2017 WhatsApp to support locked recordings http://wabetainfo.com/whatsapp-to-support-locked-recordings/ (14th March 2018)

[33] J. Wagnerp. St, HuotW. Mackay 2012 BiTouch and BiPad: designing bimanual interaction for hand-held tablets. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Austin, Texas, USA. ACM

[34] D. WigdorC. ForlinesP. BaudischJ. BarnwellC. Shen 2007 Lucid touch: a see-through mobile device. Proceedings of the 20th annual ACM symposium on User interface software and technology Newport, Rhode Island, USA. ACM

[35] D. D. WoodsL. J. JohannesenR. I. CookN. B. Sarter 1994 Behind human error: Cognitive systems, computers and hindsight. DAYTON UNIV RESEARCH INST (URDI) OH

[36] H. XiaT. GrossmanG. Fitzmaurice 2015 NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology Charlotte, NC, USA. ACM

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65