An Expedited Chart Review Process for Large Database Studies Using Natural Language Processing and Multiwave Adaptive Sampling

Abstract

Background One of the ways to enhance analyses conducted with large claims databases is by validating the measurement characteristics of the code-based algorithms used to identify health outcomes or other key study parameters of interest. These metrics can be used in quantitative bias analyses to assess the robustness of results for an inferential study, given potential bias from outcome misclassification. However, performing this validation through manual chart review of free-text notes from linked electronic health records requires extensive time and resource allocation. Methods We describe an expedited process for validating code-based algorithms that introduces efficiency using two distinct mechanisms: (1) use of natural language processing to reduce the time spent by human reviewers to review each chart, and (2) a multiwave adaptive sampling approach with predefined criteria to stop the validation study once performance characteristics are identified with sufficient precision. We illustrate this process in a case study that validates the performance of a claims-based outcome algorithm for intentional self-harm in patients with obesity. Results We empirically demonstrate that the natural language processing-assisted annotation process reduced the time spent on review per chart by 40%, and the use of the predefined stopping rule with multiwave samples would have prevented review of 77% of patient charts with limited compromise to the precision of performance characteristics. Conclusion This approach could facilitate more routine validation of code-based algorithms used to define key study parameters, ultimately enhancing understanding of the reliability of findings derived from database studies.

Contribution highlights

• Background One of the ways to enhance analyses conducted with large claims databases is by validating the measurement characteristics of the code-based algorithms used to identify health outcomes or other key study parameters of interest.
• These metrics can be used in quantitative bias analyses to assess the robustness of results for an inferential study, given potential bias from outcome misclassification.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1097/ede.0000000000001978

PMID

41944789

Volume

37

Issue

4

Pages / article

504-514

Version relationship

No separate preprint-to-publication relationship is currently recorded.

BibTeX

@article{Wang2026Expeditedchart,
  title = {An Expedited Chart Review Process for Large Database Studies Using Natural Language Processing and Multiwave Adaptive Sampling},
  author = {Shirley V Wang and Georg Hahn and Sreedhara Sushama Kattinakere and Mufaddal Mahesri and Haritha S Pillai and Rajendra Aldis and Joyce Lii and Sarah K Dutcher and Rhoda Eniafe and Jamal T Jones and Keewan Kim and Jiwei He and Hana Lee and Sengwee Toh and Rishi J Desai and Jie Yang},
  journal = {Epidemiology},
  year = {2026},
  url = {https://pubmed.ncbi.nlm.nih.gov/41944789/},
  doi = {10.1097/ede.0000000000001978},
  pmid = {41944789},
  volume = {37},
  number = {4},
  pages = {504--514}
}