FinTruthQA: A Benchmark for AI-Driven Financial Disclosure Quality Assessment in Investor-Firm Interactions

Abstract

Accurate and transparent financial information disclosure is essential for market efficiency, investor decision-making, and corporate governance. Chinese stock exchanges’ investor interactive platforms provide a widely used channel through which listed firms respond to investor concerns, yet these responses are often limited or non-substantive, making disclosure quality difficult to assess at scale. To address this challenge, we introduce FinTruthQA, to our knowledge the first benchmark for AI-driven assessment of financial disclosure quality in investor–firm interactions. FinTruthQA comprises 6,000 real-world financial Q&A entries, each manually annotated based on four key evaluation criteria: question identification, question relevance, answer readability, and answer relevance. We benchmark statistical machine learning models, pre-trained language models and their fine-tuned variants, as well as large language models (LLMs), on FinTruthQA. Experiments show that existing models achieve strong performance on question identification and question relevance (F1>95%), but remain substantially weaker on answer readability (Micro F1∼88%) and especially answer relevance (Micro F1∼80%), highlighting the nontrivial difficulty of fine-grained disclosure quality assessment. Domainand task-adapted pre-trained language models consistently outperform general-purpose models and LLM-based prompting on the most challenging settings. These findings position FinTruthQA as a practical foundation for AI-driven disclosure monitoring in capital markets, with value for regulatory oversight, investor protection, and disclosure governance in real-world financial settings.

Contribution highlights

• To address this challenge, we introduce FinTruthQA, to our knowledge the first benchmark for AI-driven assessment of financial disclosure quality in investor–firm interactions.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.1609/aaaiss.v9i1.42926

Volume

9

Issue

1

Pages / article

204-212

Version relationship

No separate preprint-to-publication relationship is currently recorded.

BibTeX

@inproceedings{Zhou2026FinTruthQA,
  title = {FinTruthQA: A Benchmark for AI-Driven Financial Disclosure Quality Assessment in Investor-Firm Interactions},
  author = {Peilin Zhou and Ziyue Xu and Xinyu Shi and Jiageng Wu and Yikang Jiang and Dading Chong and Wang Dong and Jun Chen and Bin Ke and Jie Yang},
  booktitle = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  url = {https://ojs.aaai.org/index.php/AAAI-SS/article/view/42926},
  doi = {10.1609/aaaiss.v9i1.42926},
  volume = {9},
  number = {1},
  pages = {204--212}
}