+1 Recommend
1 collections

      To submit your manuscript, please click here

      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.



          Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad.


          The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing.


          This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic–virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features.


          Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings.


          The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers.

          International Registered Report Identifier (IRRID)


          Related collections

          Most cited references58

          • Record: found
          • Abstract: not found
          • Article: not found

          The Measurement of Observer Agreement for Categorical Data

            • Record: found
            • Abstract: not found
            • Article: not found

            How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability

              • Record: found
              • Abstract: found
              • Article: not found

              Achieving integration in mixed methods designs-principles and practices.

              Mixed methods research offers powerful tools for investigating complex processes and systems in health and health care. This article describes integration principles and practices at three levels in mixed methods research and provides illustrative examples. Integration at the study design level occurs through three basic mixed method designs-exploratory sequential, explanatory sequential, and convergent-and through four advanced frameworks-multistage, intervention, case study, and participatory. Integration at the methods level occurs through four approaches. In connecting, one database links to the other through sampling. With building, one database informs the data collection approach of the other. When merging, the two databases are brought together for analysis. With embedding, data collection and analysis link at multiple points. Integration at the interpretation and reporting level occurs through narrative, data transformation, and joint display. The fit of integration describes the extent the qualitative and quantitative findings cohere. Understanding these principles and practices of integration can help health services researchers leverage the strengths of mixed methods. © Health Research and Educational Trust.

                Author and article information

                JMIR Res Protoc
                JMIR Res Protoc
                JMIR Research Protocols
                JMIR Publications (Toronto, Canada )
                6 June 2023
                : 12
                : e46601
                [1 ] Department of Educational Psychology University of Nebraska-Lincoln Lincoln, NE United States
                [2 ] Department of Family Medicine University of Michigan Ann Arbor, MI United States
                [3 ] Department of Psychology Old Dominion University Norfolk, VA United States
                [4 ] Section of General Internal Medicine Department of Internal Medicine Yale School of Medicine New Haven, CT United States
                [5 ] Department of Psychology University of Michigan Ann Arbor, MI United States
                [6 ] Department of Internal Medicine University of Michigan Ann Arbor, MI United States
                [7 ] Department of Family and Community Medicine University of Illinois College of Medicine Chicago, IL United States
                [8 ] School of Information University of Michigan Ann Arbor, MI United States
                Author notes
                Corresponding Author: Timothy C Guetterman tguetter@ 123456umich.edu
                Author information
                ©Analay Perez, Michael D Fetters, John W Creswell, Mark Scerbo, Frederick W Kron, Richard Gonzalez, Lawrence An, Masahito Jimbo, Predrag Klasnja, Timothy C Guetterman. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 06.06.2023.

                This is an open-access article distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.

                : 19 February 2023
                : 24 April 2023
                : 27 April 2023
                : 27 April 2023
                Custom metadata
                The proposal for this study was peer-reviewed by: National Library of Medicine Special Emphasis Panel - Conflict of Interest and Career Awards (National Institutes of Health, USA). See the Multimedia Appendix for the peer-review report;

                human technology,mpathic-vr,nonverbal communication behavior,patient-provider communication,virtual human


                Comment on this article