Improving the Identification of Diabetic Retinopathy and Related Conditions in the Electronic Health Record Using Natural Language Processing Methods

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Purpose

To compare the performance of 3 phenotyping methods in identifying diabetic retinopathy (DR) and related clinical conditions.

Design

Three phenotyping methods were used to identify clinical conditions including unspecified DR, nonproliferative DR (NPDR) (mild, moderate, severe), consolidated NPDR (unspecified DR or any NPDR), proliferative DR, diabetic macular edema (DME), vitreous hemorrhage, retinal detachment (RD) (tractional RD or combined tractional and rhegmatogenous RD), and neovascular glaucoma (NVG). The first method used only International Classification of Diseases, 10th Revision (ICD-10) diagnosis codes ( ICD-10 Lookup System). The next 2 methods used a Bidirectional Encoder Representations from Transformers with a dense Multilayer Perceptron output layer natural language processing (NLP) framework. The NLP framework was applied either to free-text of provider notes ( Text-Only NLP System) or both free-text and ICD-10 diagnosis codes ( Text-and-International Classification of Diseases [ ICD] NLP System).

Subjects

Adults ≥18 years with diabetes mellitus seen at the Wilmer Eye Institute.

Methods

We compared the performance of the 3 phenotyping methods in identifying the DR related conditions with gold standard chart review. We also compared the estimated disease prevalence using each method.

Main Outcome Measures

Performance of each method was reported as the macro F1 score. The agreement between the methods was calculated using the kappa statistic. Prevalence estimates were also calculated for each method.

Results

A total of 91 097 patients and 692 486 office visits were included in the study. Compared with the gold standard, the Text-and-ICD NLP System had the highest F1 score for most clinical conditions (range 0.39–0.64). The agreement between the ICD-10 Lookup System and Text-Only NLP System varied (kappa of 0.21–0.81). The prevalence of DR and related conditions ranged from 1.1% for NVG to 17.9% for DME (using the Text-and-ICD NLP System).

Conclusions

The prevalence of DR and related conditions varied significantly depending on the methodology of identifying cases. The best performing phenotyping method was the Text-and-ICD NLP System that used information in both diagnosis codes as well as free-text notes.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Related collections

Most cited references 33

Record: found
Abstract: found
Article: found

Is Open Access

The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement

Eric Benchimol, Liam Smeeth, Astrid Guttmann … (2015)

Routinely collected health data, obtained for administrative and clinical purposes without specific a priori research goals, are increasingly used for research. The rapid evolution and availability of these data have revealed issues not addressed by existing reporting guidelines, such as Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). The REporting of studies Conducted using Observational Routinely collected health Data (RECORD) statement was created to fill these gaps. RECORD was created as an extension to the STROBE statement to address reporting items specific to observational studies using routinely collected health data. RECORD consists of a checklist of 13 items related to the title, abstract, introduction, methods, results, and discussion section of articles, and other information required for inclusion in such research reports. This document contains the checklist and explanatory and elaboration information to enhance the use of the checklist. Examples of good reporting for each RECORD checklist item are also included herein. This document, as well as the accompanying website and message board (http://www.record-statement.org), will enhance the implementation and understanding of RECORD. Through implementation of RECORD, authors, journals editors, and peer reviewers can encourage transparency of research reporting.

0 comments Cited 1630 times – based on 0 reviews      Review now

Record: found
Abstract: found
Article: not found

The prevalence of diabetic retinopathy among adults in the United States.

Benita O’Colmain, Richard Hamman, S Moss … (2004)

To determine the prevalence of diabetic retinopathy among adults 40 years and older in the United States. Pooled analysis of data from 8 population-based eye surveys was used to estimate the prevalence, among persons with diabetes mellitus (DM), of retinopathy and of vision-threatening retinopathy-defined as proliferative or severe nonproliferative retinopathy and/or macular edema. Within strata of age, race/ethnicity, and gender, US prevalence rates were estimated by multiplying these values by the prevalence of DM reported in the 1999 National Health Interview Survey and the 2000 US Census population. Among an estimated 10.2 million US adults 40 years and older known to have DM, the estimated crude prevalence rates for retinopathy and vision-threatening retinopathy were 40.3% and 8.2%, respectively. The estimated US general population prevalence rates for retinopathy and vision-threatening retinopathy were 3.4% (4.1 million persons) and 0.75% (899 000 persons). Future projections suggest that diabetic retinopathy will increase as a public health problem, both with aging of the US population and increasing age-specific prevalence of DM over time. Approximately 4.1 million US adults 40 years and older have diabetic retinopathy; 1 of every 12 persons with DM in this age group has advanced, vision-threatening retinopathy.

0 comments Cited 128 times – based on 0 reviews      Review now

Record: found
Abstract: found
Article: not found

Projection of diabetic retinopathy and other major eye diseases among people with diabetes mellitus: United States, 2005-2050.

Joseph Boyle, K. M. Venkat Narayan, Xinzhi Zhang … (2008)

To estimate the number of people with diabetic retinopathy (DR), vision-threatening DR (VTDR), glaucoma, and cataracts among Americans 40 years or older with diagnosed diabetes mellitus for the years 2005-2050. Using published prevalence data of DR, VTDR, glaucoma, and cataracts and data from the National Health Interview Survey and the US Census Bureau, we projected the number of Americans with diabetes with these eye conditions. The number of Americans 40 years or older with DR and VTDR will triple in 2050, from 5.5 million in 2005 to 16.0 million for DR and from 1.2 million in 2005 to 3.4 million for VTDR. Increases among those 65 years or older will be more pronounced (2.5 million to 9.9 million for DR and 0.5 million to 1.9 million for VTDR). The number of cataract cases among whites and blacks 40 years or older with diabetes will likely increase 235% by 2050, and the number of glaucoma cases among Hispanics with diabetes 65 years or older will increase 12-fold. Future increases in the number of Americans with diabetes will likely lead to significant increases in the number with DR, glaucoma, and cataracts. Our projections may help policy makers anticipate future demands for health care resources and possibly guide the development of targeted interventions. Efforts to prevent diabetes and to optimally manage diabetes and its complications are needed.

0 comments Cited 114 times – based on 0 reviews      Review now

All references

Author and article information

Contributors

Cindy X. Cai

Journal

Journal ID (nlm-ta): Ophthalmol Sci

Journal ID (iso-abbrev): Ophthalmol Sci

Title: Ophthalmology Science

Publisher: Elsevier

ISSN (Electronic): 2666-9145

Publication date PMC-release: 18 July 2024

Publication date Collection: Nov-Dec 2024

Publication date (Electronic): 18 July 2024

Volume: 4

Issue: 6

Electronic Location Identifier: 100578

Affiliations

[1 ]Department of Computer Science, Johns Hopkins University, Baltimore, Maryland

[2 ]Wilmer Eye Institute, Johns Hopkins School of Medicine, Baltimore, Maryland

[3 ]Department of Biomedical Informatics and Data Science, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland

[4 ]Center for Population Health Information Technology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland

Author notes

[∗ ]Correspondence: Cindy X. Cai, MD, MS, Wilmer Eye Institute, 1800 Orleans Street, Room 711, Baltimore, MD 21287. ccai6@ 123456jhmi.edu

Article

Publisher Item ID: S2666-9145(24)00114-3 Publisher ID: 100578

DOI: 10.1016/j.xops.2024.100578

PMC ID: 11382176

PubMed ID: 39253550

SO-VID: 48839acc-aafb-44f3-8c4c-666ad892b151

License:

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

History

Date received : 19 April 2024

Date revision received : 4 July 2024

Date accepted : 12 July 2024