A Hindi Virtual Keyboard Interface with Multimodal Feedback: A Case Study with a Dyslexic Child

Up to 15% of the Indian school-going children suffer from dyslexia. This paper aims to determine the extent to which existing knowledge about the eye-tracking based human-computer interface can be used to assist these children in their reading and writing activities. A virtual keyboard system with multimodal feedback is proposed and designed for a lexically and structurally complex language and optimized for multimodal feedback involving several portable, non-invasive, and low-cost input devices: a touch screen, an eye-tracker, and a soft-switch. The performance was evaluated in terms of text-entry rate, information transfer rate, and type of errors with three different experimental conditions: 1) touch-screen condition with auditory feedback 2) eye-tracking condition with auditory and visual feedback, and 3) eye-tracking and soft-switch condition with auditory and visual feedback. The proposed multimodal feedback has shown a significant improvement in text-entry rate with less error. This work represents the first virtual keyboard with multimodal feedback for dyslexic children in the Hindi language, which can be extended to other languages.

Content

Author and article information

Contributors

Yogesh Kumar Meena

Anirban Chowdhury

Ujjwal Sharma

Hubert Cecotti

Braj Bhushan

Ashish Dutta

Girijesh Prasad

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 1-5

Affiliations

[0001]Intelligent Systems Research

Centre, Ulster University

Derry~Londonderry, UK

[0002]Centre of Mechatronics

Indian Institute of Technology

Kanpur, India

[0003]Department of Humanities and Social

Sciences, Indian Institute of Technology

Kanpur, India

[0004]Department of Computer Science

College of Science and Mathematics

Fresno State University, USA

Article

DOI: 10.14236/ewic/HCI2018.148

SO-VID: 965ab3d5-f34f-47f7-a4d1-41ce59e6c808

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Conference acronym: HCI

Conference number: 32

Conference location: Belfast, UK

Conference date: 4 - 6 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Human Computer Interaction Conference

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.148

Self URI (journal page): https://ewic.bcs.org/

References

S. BellocchiM. MuneauxM. Bastien-ToniazzoS. Ducrot 2013 I can read it in your eyes: What eye movements tell us about visuo-attentional processes in developmental dyslexia Research in Developmental Disabilities 34 452 460
Z. Breznitz 2002 Asynchrony of visual-orthographic and auditory-phonological word recognition processes: An underlying factor in dyslexia Reading and Writing 15 15 42
M. P. BucciD. Bre´mond-GignacZ. Kapoula 2008 Latency of saccades and vergence eye movements in dyslexic children Experimental Brain Research 188 1 12
H. Cecotti 2016 A multimodal gaze-controlled virtual keyboard IEEE Trans. Human-Machine Systems 46 4 601 606
De LucaM., M. BorrelliA. JudicaD. SpinelliP. Zoccolotti 2002 Reading words and pseudowords: An eye movement study of developmental dyslexia Brain and Language 80 617 626
S. HawelkaB. GaglH. Wimmer 2010 A dual-route perspective on eye movements of dyslexic readers Cognition 115 367 379
B. Joly-PottuzM. MercierA. LeynaudM. Habib 2008 Combined auditory and articulatory training improves phonological deficit in children with dyslexia Neuropsychological Rehabilitation 18 402 429
M KapurA JohnJ RozarioA OommenH. Uma 2002 Assessment of specific learning disabilities U HirisaveA OommenM. Kapur (eds) Psychological Assessment of Children in the clinical Setting Bangalore Vinayaka Offset Printers
Z. KubovaM. KubaJ. PeregrinV. Novakova 1996 Visually evoked potential evidence for magnocellular system deficit in dyslexia Physiological Research 45 87 89
X. LiP. LiuK. Rayner 2011 Eye movements guidance in Chinese reading: Is there a preferred viewing location? Vision Research 51 1146 1156
S. L. MacknikM. S. Livingstone 1998 Neuronal correlates of visibility and invisibility in the primate visual system Nature Neuroscience 1 144 149
Y. K. MeenaH. CecottiK. Wong-LinA. DuttaG. Prasad 2018 Toward Optimization of Gaze-Controlled Human–Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients IEEE Trans. Neural Syst Rehabil Eng 26 4 911 922
Y. K. MeenaH. CecottiK. Wong-LinG. Prasad 2017 A multimodal interface to resolve the Midas-Touch problem in gaze controlled wheelchair IEEE Conf. on Engineering in Medicine and Biology Society (EMBS) 905 908
Y. K. MeenaH. CecottiK. Wong-LinG. Prasad 2016 A novel multimodal gaze-controlled Hindi virtual keyboard for disabled users IEEE Conf. on Systems, Man, and Cybernetics 3688 3693
K. Rayner 1998 Eye movements in reading and information processing: 20 years of research Psychological Bulletin 124 372 422
L. RelloG. KanvindeR. Baeza-Yates 2012 Layout guidelines for web text and a web service to improve accessibility for dyslexics In Proc. W4A ’12 Lyon, France 2012 ACM Press
M. M. ShovmanM. Ahissar 2006 Isolating the impact of visual perception on dyslexics’ reading ability Vision Research 46 3514 3525
J. V. SinghG. Prasad 2015 Enhancing an eye-tracker based human computer interface with multi-modal accessibility applied for text entry Int. J. of Computer Applications 130 16 16 22
W. L. SlaghuisJ. F. Ryan 2006 Directional motion contrast sensitivity in developmental dyslexia Vision Research 46 3291 3303
J. Stein 2001 The sensory basis of reading problems Developmental Neuropsychology 20 2 509 534
D. Wechsler 1949 Wechsler intelligence scale for children New York The Psychological Corporation
C. M. WrightE. G. Conlon 2009 Auditory and Visual Processing in Children With Dyslexia Developmental Neuropsychology 34 330 355

Comments

Comment on this article

[1] S. BellocchiM. MuneauxM. Bastien-ToniazzoS. Ducrot 2013 I can read it in your eyes: What eye movements tell us about visuo-attentional processes in developmental dyslexia Research in Developmental Disabilities 34 452 460

[2] Z. Breznitz 2002 Asynchrony of visual-orthographic and auditory-phonological word recognition processes: An underlying factor in dyslexia Reading and Writing 15 15 42

[3] M. P. BucciD. Bre´mond-GignacZ. Kapoula 2008 Latency of saccades and vergence eye movements in dyslexic children Experimental Brain Research 188 1 12

[4] H. Cecotti 2016 A multimodal gaze-controlled virtual keyboard IEEE Trans. Human-Machine Systems 46 4 601 606

[5] De LucaM., M. BorrelliA. JudicaD. SpinelliP. Zoccolotti 2002 Reading words and pseudowords: An eye movement study of developmental dyslexia Brain and Language 80 617 626

[6] S. HawelkaB. GaglH. Wimmer 2010 A dual-route perspective on eye movements of dyslexic readers Cognition 115 367 379

[7] B. Joly-PottuzM. MercierA. LeynaudM. Habib 2008 Combined auditory and articulatory training improves phonological deficit in children with dyslexia Neuropsychological Rehabilitation 18 402 429

[8] M KapurA JohnJ RozarioA OommenH. Uma 2002 Assessment of specific learning disabilities U HirisaveA OommenM. Kapur (eds) Psychological Assessment of Children in the clinical Setting Bangalore Vinayaka Offset Printers

[9] Z. KubovaM. KubaJ. PeregrinV. Novakova 1996 Visually evoked potential evidence for magnocellular system deficit in dyslexia Physiological Research 45 87 89

[10] X. LiP. LiuK. Rayner 2011 Eye movements guidance in Chinese reading: Is there a preferred viewing location? Vision Research 51 1146 1156

[11] S. L. MacknikM. S. Livingstone 1998 Neuronal correlates of visibility and invisibility in the primate visual system Nature Neuroscience 1 144 149

[12] Y. K. MeenaH. CecottiK. Wong-LinA. DuttaG. Prasad 2018 Toward Optimization of Gaze-Controlled Human–Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients IEEE Trans. Neural Syst Rehabil Eng 26 4 911 922

[13] Y. K. MeenaH. CecottiK. Wong-LinG. Prasad 2017 A multimodal interface to resolve the Midas-Touch problem in gaze controlled wheelchair IEEE Conf. on Engineering in Medicine and Biology Society (EMBS) 905 908

[14] Y. K. MeenaH. CecottiK. Wong-LinG. Prasad 2016 A novel multimodal gaze-controlled Hindi virtual keyboard for disabled users IEEE Conf. on Systems, Man, and Cybernetics 3688 3693

[15] K. Rayner 1998 Eye movements in reading and information processing: 20 years of research Psychological Bulletin 124 372 422

[16] L. RelloG. KanvindeR. Baeza-Yates 2012 Layout guidelines for web text and a web service to improve accessibility for dyslexics In Proc. W4A ’12 Lyon, France 2012 ACM Press

[17] M. M. ShovmanM. Ahissar 2006 Isolating the impact of visual perception on dyslexics’ reading ability Vision Research 46 3514 3525

[18] J. V. SinghG. Prasad 2015 Enhancing an eye-tracker based human computer interface with multi-modal accessibility applied for text entry Int. J. of Computer Applications 130 16 16 22

[19] W. L. SlaghuisJ. F. Ryan 2006 Directional motion contrast sensitivity in developmental dyslexia Vision Research 46 3291 3303

[20] J. Stein 2001 The sensory basis of reading problems Developmental Neuropsychology 20 2 509 534

[21] D. Wechsler 1949 Wechsler intelligence scale for children New York The Psychological Corporation

[22] C. M. WrightE. G. Conlon 2009 Auditory and Visual Processing in Children With Dyslexia Developmental Neuropsychology 34 330 355

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65