Using Natural Language Processing of Clinical Notes to Supplement Structured Electronic Health Record Data for Phenotyping Smoking and Obesity in a Healthcare System

Abstract

Purpose Studies based on electronic health records (EHR) often rely on structured data, which may incompletely capture important clinical phenotypes in EHR notes. The purpose of this study was to assess two natural language processing (NLP) tools to extract phenotypes from unstructured EHR notes, and to evaluate the added value of integrating NLP-derived phenotypes with structured EHR data at a health system scale. Methods This retrospective study is based on inpatient and outpatient EHR data from the Mass General Brigham healthcare system between January 1, 2019 and December 31, 2020. Two established rule-based NLP tools were applied to extract smoking and obesity information from 19 215 303 clinical notes of 503 025 patients. NLP performance was evaluated through a manual review of stratified samples. Phenotype prevalence was estimated using structured EHR data alone and compared with prevalence estimates obtained by supplementing structured data with NLP-derived features. Results Both NLP tools exhibited high performance, with both accuracy and the F1 score of 0.99 for smoking, and 0.92 and 0.91 for obesity, respectively. The combination of NLP and structured data identified 220 714 patients (43.88%) with smoking, compared with 170 396 patients (33.87%) identified using structured data alone, representing a 29.5% relative increase. For obesity, NLP identified 121 360 patients (24.13%) from EHR notes, and 169 905 patients (33.78%) were documented in structured data; the inclusion of NLP-derived features contributed an additional 32 823 patients, corresponding to a 19.3% relative increase. Conclusion NLP-derived phenotypes from unstructured EHR notes substantially improve patient identification for both smoking and obesity compared with structured EHR data alone at scale.

Contribution highlights

• The purpose of this study was to assess two natural language processing (NLP) tools to extract phenotypes from unstructured EHR notes, and to evaluate the added value of integrating NLP-derived phenotypes with structured EHR data at a health system scale.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1002/pds.70415

PMID

42301002

Volume

35

Issue

7

Pages / article

e70415

Version relationship

• Preprint: https://doi.org/10.64898/2026.01.18.26344356
• Version of record: https://doi.org/10.1002/pds.70415

BibTeX

@article{Yang2026Naturallanguage,
  title = {Using Natural Language Processing of Clinical Notes to Supplement Structured Electronic Health Record Data for Phenotyping Smoking and Obesity in a Healthcare System},
  author = {Jie Yang and Bowen Gu and Haritha Pillai and Joyce Lii and David Cronkite and Keith A Marsolo and Rishi J Desai},
  journal = {Pharmacoepidemiology and Drug Safety},
  year = {2026},
  url = {https://onlinelibrary.wiley.com/doi/full/10.1002/pds.70415},
  doi = {10.1002/pds.70415},
  pmid = {42301002},
  volume = {35},
  number = {7},
  pages = {e70415}
}