Probabilistic Medical Predictions of Large Language Models

Abstract

Large Language Models (LLMs) have shown promise in clinical applications through prompt engineering, allowing flexible clinical predictions. However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making. While explicit prompts can lead LLMs to generate probability estimates, their numerical reasoning limitations raise concerns about reliability. We compared explicit probabilities from text generation to implicit probabilities derived from the likelihood of predicting the correct label token. Across six advanced open-source LLMs and five medical datasets, explicit probabilities consistently underperformed implicit probabilities in discrimination, precision, and recall. This discrepancy is more pronounced with smaller LLMs and imbalanced datasets, highlighting the need for cautious interpretation, improved probability estimation methods, and further research for clinical use of LLMs.

Contribution highlights

• Large Language Models (LLMs) have shown promise in clinical applications through prompt engineering, allowing flexible clinical predictions.
• However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1038/s41746-024-01366-4

PMID

39702641

PMCID

PMC11659327

Volume

7

Issue

1

Pages / article

367

Version relationship

No separate preprint-to-publication relationship is currently recorded.

BibTeX

@article{Gu2024Probabilisticmedical,
  title = {Probabilistic Medical Predictions of Large Language Models},
  author = {Bowen Gu and Rishi J. Desai and Kueiyu Joshua Lin and Jie Yang},
  journal = {npj Digital Medicine},
  year = {2024},
  url = {https://www.nature.com/articles/s41746-024-01366-4},
  doi = {10.1038/s41746-024-01366-4},
  pmid = {39702641},
  volume = {7},
  number = {1},
  pages = {367}
}