Clinical Pathway-Aware Large Language Models for Reliable and Transparent Medical Dialogue

Abstract

Objective Large language models (LLMs) offer promising potential in answering real-time medical queries, but they often produce lengthy, generic, and even hallucinatory responses. We aim to develop a reliable and interpretable medical dialogue system that incorporates clinical reasoning and then mitigates the risk of hallucination. Methods Two large datasets of real-world online consultation, MedDG and KaMed, were used for evaluation. We proposed a Medical Dialogue System with Knowledge Enhancement and Clinical Pathway Encoding (MedKP), which integrates an external medical knowledge graph and encodes internal clinical pathways to model physician reasoning. Performance was compared with state-of-the-art baselines, including GPT-4o and LLaMA3.1-70B. A multi-dimensional evaluation framework assessed (1) clinical relevance (medical entity-based), (2) textual similarity (ROUGE, BLEU), (3) semantic alignment (BERTScore), and (4) hallucination and consistency via an external LLM-based judge, as well as parallel human evaluation. Results Across both datasets, MedKP (6B) achieved the best overall performance, outperforming other advanced baselines and producing responses that align more closely with those of human physicians. For clinical relevance, MedKP reached a macro F1-score of medical entity at 31.41 on MedDG (previous best DFMed: 24.76, improved 30.41%) and 26.62 on KaMed (previous best LLaM-A3.1-70B: 20.67, improved 25.62%). Consistent improvements were observed across other metrics. Ablation studies further validated the effectiveness of each model component. Conclusion Our results highlight the critical role of clinical reasoning in advancing trustworthy AI for digital healthcare. By enhancing the reliability, coherence, and transparency of AI-generated responses, this pathway-aware approach bridges the gap between LLMs and real-world clinical workflows, improving the accessibility of high-quality telemedicine services, particularly benefiting underserved populations.

Contribution highlights

• Objective Large language models (LLMs) offer promising potential in answering real-time medical queries, but they often produce lengthy, generic, and even hallucinatory responses.
• We aim to develop a reliable and interpretable medical dialogue system that incorporates clinical reasoning and then mitigates the risk of hallucination.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1016/j.jbi.2025.104942

PMID

41177245

Volume

172

Pages / article

104942

Version relationship

• Preprint: https://arxiv.org/abs/2403.06611
• Version of record: https://doi.org/10.1016/j.jbi.2025.104942

BibTeX

@article{Wu2025Clinicalpathway,
  title = {Clinical Pathway-Aware Large Language Models for Reliable and Transparent Medical Dialogue},
  author = {Jiageng Wu and Xian Wu and Yefeng Zheng and Jie Yang},
  journal = {Journal of Biomedical Informatics},
  year = {2025},
  url = {https://arxiv.org/abs/2403.06611},
  doi = {10.1016/j.jbi.2025.104942},
  pmid = {41177245},
  volume = {172},
  pages = {104942}
}