A Natural Language Processing System That Links Medical Terms in Electronic Health Record Notes to Lay Definitions: System Development Using Physician Reviews

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Many health care systems now allow patients to access their electronic health record (EHR) notes online through patient portals. Medical jargon in EHR notes can confuse patients, which may interfere with potential benefits of patient access to EHR notes.

Objective

The aim of this study was to develop and evaluate the usability and content quality of NoteAid, a Web-based natural language processing system that links medical terms in EHR notes to lay definitions, that is, definitions easily understood by lay people.

Methods

NoteAid incorporates two core components: CoDeMed, a lexical resource of lay definitions for medical terms, and MedLink, a computational unit that links medical terms to lay definitions. We developed innovative computational methods, including an adapted distant supervision algorithm to prioritize medical terms important for EHR comprehension to facilitate the effort of building CoDeMed. Ten physician domain experts evaluated the user interface and content quality of NoteAid. The evaluation protocol included a cognitive walkthrough session and a postsession questionnaire. Physician feedback sessions were audio-recorded. We used standard content analysis methods to analyze qualitative data from these sessions.

Results

Physician feedback was mixed. Positive feedback on NoteAid included (1) Easy to use, (2) Good visual display, (3) Satisfactory system speed, and (4) Adequate lay definitions. Opportunities for improvement arising from evaluation sessions and feedback included (1) improving the display of definitions for partially matched terms, (2) including more medical terms in CoDeMed, (3) improving the handling of terms whose definitions vary depending on different contexts, and (4) standardizing the scope of definitions for medicines. On the basis of these results, we have improved NoteAid’s user interface and a number of definitions, and added 4502 more definitions in CoDeMed.

Conclusions

Physician evaluation yielded useful feedback for content validation and refinement of this innovative tool that has the potential to improve patient EHR comprehension and experience using patient portals. Future ongoing work will develop algorithms to handle ambiguous medical terms and test and evaluate NoteAid with patients.

Related collections

Most cited references 46

Record: found
Abstract: found
Article: not found

Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead.

Tom Delbanco, Jan Walker, Sigall K Bell … (2012)

Little information exists about what primary care physicians (PCPs) and patients experience if patients are invited to read their doctors' office notes. To evaluate the effect on doctors and patients of facilitating patient access to visit notes over secure Internet portals. Quasi-experimental trial of PCPs and patient volunteers in a year-long program that provided patients with electronic links to their doctors' notes. Primary care practices at Beth Israel Deaconess Medical Center (BIDMC) in Massachusetts, Geisinger Health System (GHS) in Pennsylvania, and Harborview Medical Center (HMC) in Washington. 105 PCPs and 13 564 of their patients who had at least 1 completed note available during the intervention period. Portal use and electronic messaging by patients and surveys focusing on participants' perceptions of behaviors, benefits, and negative consequences. 11 797 of 13 564 patients with visit notes available opened at least 1 note (84% at BIDMC, 92% at GHS, and 47% at HMC). Of 5391 patients who opened at least 1 note and completed a postintervention survey, 77% to 87% across the 3 sites reported that open notes helped them feel more in control of their care; 60% to 78% of those taking medications reported increased medication adherence; 26% to 36% had privacy concerns; 1% to 8% reported that the notes caused confusion, worry, or offense; and 20% to 42% reported sharing notes with others. The volume of electronic messages from patients did not change. After the intervention, few doctors reported longer visits (0% to 5%) or more time addressing patients' questions outside of visits (0% to 8%), with practice size having little effect; 3% to 36% of doctors reported changing documentation content; and 0% to 21% reported taking more time writing notes. Looking ahead, 59% to 62% of patients believed that they should be able to add comments to a doctor's note. One out of 3 patients believed that they should be able to approve the notes' contents, but 85% to 96% of doctors did not agree. At the end of the experimental period, 99% of patients wanted open notes to continue and no doctor elected to stop. Only 3 geographic areas were represented, and most participants were experienced in using portals. Doctors volunteering to participate and patients using portals and completing surveys may tend to offer favorable feedback, and the response rate of the patient surveys (41%) may further limit generalizability. Patients accessed visit notes frequently, a large majority reported clinically relevant benefits and minimal concerns, and virtually all patients wanted the practice to continue. With doctors experiencing no more than a modest effect on their work lives, open notes seem worthy of widespread adoption. The Robert Wood Johnson Foundation, the Drane Family Fund, the Richard and Florence Koplow Charitable Foundation, and the National Cancer Institute.

0 comments Cited 206 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: not found
Conference Proceedings: not found

A mathematical model of the finding of usability problems

Jakob Nielsen, Thomas Landauer (1993)

0 comments Cited 162 times – based on 0 reviews

Bookmark

Record: found
Abstract: not found
Article: not found

Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?

Robert Virzi (2016)

0 comments Cited 137 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Jinying Chen:

Department of Quantitative Health SciencesUniversity of Massachusetts Medical School368 Plantation StreetWorcester, MA, 01605United States1 774 455 35271 508 856 8993jinying.chen@umassmed.edu

ORCID: http://orcid.org/0000-0001-7259-4301

Journal

Journal ID (nlm-ta): J Med Internet Res

Journal ID (iso-abbrev): J. Med. Internet Res

Journal ID (publisher-id): JMIR

Title: Journal of Medical Internet Research

Publisher: JMIR Publications (Toronto, Canada )

ISSN (Print): 1439-4456

ISSN (Electronic): 1438-8871

Publication date Collection: January 2018

Publication date (Electronic): 22 January 2018

Volume: 20

Issue: 1

Electronic Location Identifier: e26

Affiliations

[1] ¹ Department of Quantitative Health Sciences University of Massachusetts Medical School Worcester, MA United States

[2] ² Bedford Veterans Affairs Medical Center Center for Healthcare Organization and Implementation Research Bedford, MA United States

[3] ³ Optum Boston, MA United States

[4] ⁴ Veterans Affairs Connecticut Health Care System West Haven, CT United States

[5] ⁵ Center for Medical Informatics Yale University New Haven, CT United States

[6] ⁶ Division of Primary Care and Population Health Stanford University School of Medicine Stanford, CA United States

[7] ⁷ Veterans Affairs Palo Alto Health Care System Menlo Park, CA United States

[8] ⁸ School of Medicine Boston University Boston, MA United States

[9] ⁹ Diabetes Center of Excellence University of Massachusetts Medical School Worcester, MA United States

Author notes

Corresponding Author: Jinying Chen jinying.chen@ 123456umassmed.edu

Author information

Jinying Chen http://orcid.org/0000-0001-7259-4301

Emily Druhl http://orcid.org/0000-0003-3547-5359

Balaji Polepalli Ramesh http://orcid.org/0000-0003-2885-9194

Thomas K Houston http://orcid.org/0000-0002-2909-4018

Cynthia A Brandt http://orcid.org/0000-0001-8179-1796

Donna M Zulman http://orcid.org/0000-0001-8904-2810

Varsha G Vimalananda http://orcid.org/0000-0003-0738-672X

Samir Malkani http://orcid.org/0000-0001-9076-4327

Hong Yu http://orcid.org/0000-0001-9263-5035

Article

Publisher ID: v20i1e26

DOI: 10.2196/jmir.8669

PMC ID: 5799720

PubMed ID: 29358159

SO-VID: 517e799d-d0cc-433b-81e2-319b6a132189

Copyright © ©Jinying Chen, Emily Druhl, Balaji Polepalli Ramesh, Thomas K Houston, Cynthia A Brandt, Donna M Zulman, Varsha G Vimalananda, Samir Malkani, Hong Yu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 22.01.2018.

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

History

Date received : 8 August 2017

Date revision requested : 28 September 2017

Date revision received : 21 November 2017

Date accepted : 6 December 2017

Comments

Comment on this article

scite_

Cited by 16

See all cited by

Most referenced authors 530

See all reference authors

Submit your digital health research with an established publisher
- celebrating 25 years of open access