Comparative Ranking of Marginal Confounding Impact of Natural Language Processing-Derived versus Structured Features in Pharmacoepidemiology

Abstract

Objective To explore the ability of natural language processing (NLP) methods to identify confounder information beyond what can be identified using claims codes alone for pharmacoepidemiology. Methods We developed a retrospective cohort for high vs low dose proton pump inhibitors from linked Medicare claims (2008-2017) and electronic health record data for patients with a history of peptic ulcer disease or osteoarthritis. Clinical notes authored one year before first dispensing date were processed by off-the-shelf tools: bag-of-n-grams, latent Dirichlet allocation, a linguistics-focused tool, BERT sentence embeddings, BioBERT word embeddings, and GloVe word embeddings. Candidate features were ranked using Bross formula, a simple way to rank the marginal confounding impact of binary features on estimated causal effects. Results The marginal confounding impact in the Bross rankings of NLP-derived features trended from 39 % in the top 100 to 77 % in the top 500 to 93 % in the top 5000 among patients with peptic ulcer disease. More specifically, the top 25 confounders are largely from factors identified by domain experts and structured fields, and the marginal impact of these confounders is stronger than others. Features 25 to 50 include features identified by a linguistics-focused tool and embeddings, whereas features 50 to 100 include more embeddings and bag-of-ngrams. After 100, the curve flattens, meaning that the marginal impact of those potential confounders gets smaller. Similarly, among patients with osteoarthritis, NLP-derived features trended from 66 % in the top 100 to 84 % in the top 500 to 95 % in the top 5000 when the outcome was gastrointestinal bleed and from 47 % in the top 100 to 81 % in the top 500 to 94 % in the top 5000 when the outcome was acute kidney injury. Similar trends were observed in the information gain data, though NLP-derived features had higher baselines. Conclusions NLP contributed to finding large numbers of features that can supplement claims data and prespecified variables to help provide additional confounder information. We found that unsupervised off-the-shelf NLP tools can scale to generate large numbers of features appropriate for high-dimensional proxy adjustment and pharmacoepidemiology use cases.

Contribution highlights

• Objective To explore the ability of natural language processing (NLP) methods to identify confounder information beyond what can be identified using claims codes alone for pharmacoepidemiology.
• Methods We developed a retrospective cohort for high vs low dose proton pump inhibitors from linked Medicare claims (2008-2017) and electronic health record data for patients with a history of peptic ulcer disease or osteoarthritis.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1016/j.compbiomed.2025.109855

PMID

39965395

Volume

188

Pages / article

109855

Version relationship

No separate preprint-to-publication relationship is currently recorded.

BibTeX

@article{Plasek2025Comparativeranking,
  title = {Comparative Ranking of Marginal Confounding Impact of Natural Language Processing-Derived versus Structured Features in Pharmacoepidemiology},
  author = {Joseph M Plasek and Richard D Wyss and Janick G Weberpals and Jie Yang and Thomas Deramus and Theodore N Tsacogianis and Kerry Ngan and Lily G Bessette and Kueiyu Joshua Lin and Li Zhou},
  journal = {Computers in Biology and Medicine},
  year = {2025},
  doi = {10.1016/j.compbiomed.2025.109855},
  pmid = {39965395},
  volume = {188},
  pages = {109855}
}