+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      A Hindi Virtual Keyboard Interface with Multimodal Feedback: A Case Study with a Dyslexic Child

      , , , , , ,
      Proceedings of the 32nd International BCS Human Computer Interaction Conference (HCI)
      Human Computer Interaction Conference
      4 - 6 July 2018
      Virtual keyboard, graphical user interface, eye-tracking; multimodal feedback, dyslexia


            Up to 15% of the Indian school-going children suffer from dyslexia. This paper aims to determine the extent to which existing knowledge about the eye-tracking based human-computer interface can be used to assist these children in their reading and writing activities. A virtual keyboard system with multimodal feedback is proposed and designed for a lexically and structurally complex language and optimized for multimodal feedback involving several portable, non-invasive, and low-cost input devices: a touch screen, an eye-tracker, and a soft-switch. The performance was evaluated in terms of text-entry rate, information transfer rate, and type of errors with three different experimental conditions: 1) touch-screen condition with auditory feedback 2) eye-tracking condition with auditory and visual feedback, and 3) eye-tracking and soft-switch condition with auditory and visual feedback. The proposed multimodal feedback has shown a significant improvement in text-entry rate with less error. This work represents the first virtual keyboard with multimodal feedback for dyslexic children in the Hindi language, which can be extended to other languages.


            Author and article information

            July 2018
            July 2018
            : 1-5
            [0001]Intelligent Systems Research

            Centre, Ulster University

            Derry~Londonderry, UK
            [0002]Centre of Mechatronics

            Indian Institute of Technology

            Kanpur, India
            [0003]Department of Humanities and Social

            Sciences, Indian Institute of Technology

            Kanpur, India
            [0004]Department of Computer Science

            College of Science and Mathematics

            Fresno State University, USA
            © Meena et al. Published by BCS Learning and Development Ltd. Proceedings of British HCI 2018. Belfast, UK.

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Proceedings of the 32nd International BCS Human Computer Interaction Conference
            Belfast, UK
            4 - 6 July 2018
            Electronic Workshops in Computing (eWiC)
            Human Computer Interaction Conference
            Product Information: 1477-9358BCS Learning & Development
            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.148
            Self URI (journal page): https://ewic.bcs.org/
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Virtual keyboard,graphical user interface,eye-tracking; multimodal feedback,dyslexia


            1. 2013 I can read it in your eyes: What eye movements tell us about visuo-attentional processes in developmental dyslexia Research in Developmental Disabilities 34 452 460

            2. 2002 Asynchrony of visual-orthographic and auditory-phonological word recognition processes: An underlying factor in dyslexia Reading and Writing 15 15 42

            3. 2008 Latency of saccades and vergence eye movements in dyslexic children Experimental Brain Research 188 1 12

            4. 2016 A multimodal gaze-controlled virtual keyboard IEEE Trans. Human-Machine Systems 46 4 601 606

            5. M., 2002 Reading words and pseudowords: An eye movement study of developmental dyslexia Brain and Language 80 617 626

            6. 2010 A dual-route perspective on eye movements of dyslexic readers Cognition 115 367 379

            7. 2008 Combined auditory and articulatory training improves phonological deficit in children with dyslexia Neuropsychological Rehabilitation 18 402 429

            8. 2002 Assessment of specific learning disabilities (eds) Psychological Assessment of Children in the clinical Setting Bangalore Vinayaka Offset Printers

            9. 1996 Visually evoked potential evidence for magnocellular system deficit in dyslexia Physiological Research 45 87 89

            10. 2011 Eye movements guidance in Chinese reading: Is there a preferred viewing location? Vision Research 51 1146 1156

            11. 1998 Neuronal correlates of visibility and invisibility in the primate visual system Nature Neuroscience 1 144 149

            12. 2018 Toward Optimization of Gaze-Controlled Human–Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients IEEE Trans. Neural Syst Rehabil Eng 26 4 911 922

            13. 2017 A multimodal interface to resolve the Midas-Touch problem in gaze controlled wheelchair IEEE Conf. on Engineering in Medicine and Biology Society (EMBS) 905 908

            14. 2016 A novel multimodal gaze-controlled Hindi virtual keyboard for disabled users IEEE Conf. on Systems, Man, and Cybernetics 3688 3693

            15. 1998 Eye movements in reading and information processing: 20 years of research Psychological Bulletin 124 372 422

            16. 2012 Layout guidelines for web text and a web service to improve accessibility for dyslexics In Proc. W4A ’12 Lyon, France 2012 ACM Press

            17. 2006 Isolating the impact of visual perception on dyslexics’ reading ability Vision Research 46 3514 3525

            18. 2015 Enhancing an eye-tracker based human computer interface with multi-modal accessibility applied for text entry Int. J. of Computer Applications 130 16 16 22

            19. 2006 Directional motion contrast sensitivity in developmental dyslexia Vision Research 46 3291 3303

            20. 2001 The sensory basis of reading problems Developmental Neuropsychology 20 2 509 534

            21. 1949 Wechsler intelligence scale for children New York The Psychological Corporation

            22. 2009 Auditory and Visual Processing in Children With Dyslexia Developmental Neuropsychology 34 330 355


            Comment on this article