METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweets

Abstract

The COVID-19 pandemic continues to bring up various topics discussed or debated on social media. In order to explore the impact of pandemics on people's lives, it is crucial to understand the public's concerns and attitudes towards pandemic-related entities (e.g., drugs, vaccines) on social media. However, models trained on existing named entity recognition (NER) or targeted sentiment analysis (TSA) datasets have limited ability to understand COVID-19-related social media texts because these datasets are not designed or annotated from a medical perspective. This paper releases METS-CoV, a dataset containing medical entities and targeted sentiments from COVID-19-related tweets. METS-CoV contains 10,000 tweets with 7 types of entities, including 4 medical entity types (Disease, Drug, Symptom, and Vaccine) and 3 general entity types (Person, Location, and Organization). To further investigate tweet users' attitudes toward specific entities, 4 types of entities (Person, Organization, Drug, and Vaccine) are selected and annotated with user sentiments, resulting in a targeted sentiment dataset with 9,101 entities (in 5,278 tweets). To the best of our knowledge, METS-CoV is the first dataset to collect medical entities and corresponding sentiments of COVID-19-related tweets. We benchmark the performance of classical machine learning models and state-of-the-art deep learning models on NER and TSA tasks with extensive experiments. Results show that the dataset has vast room for improvement for both NER and TSA tasks. METS-CoV is an important resource for developing better medical social media tools and facilitating computational social science research, especially in epidemiology. Our data, annotation guidelines, benchmark models, and source code are publicly available (https://github.com/YLab-Open/METS-CoV) to ensure reproducibility.

Contribution highlights

• Results show that the dataset has vast room for improvement for both NER and TSA tasks.

Extracted from the public abstract; no claims have been added.

Code and data

Publication metadata

DOI

10.52202/068431-1593

Pages / article

21916-21932

Version relationship

No separate preprint-to-publication relationship is currently recorded.

BibTeX

@inproceedings{Zhou2022Metscov,
  title = {METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweets},
  author = {Peilin Zhou and Zeqiang Wang and Dading Chong and Zhijiang Guo and Yining Hua and Zichang Su and Zhiyang Teng and Jiageng Wu and Jie Yang},
  booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
  year = {2022},
  url = {https://openreview.net/forum?id=GP1Ncd8nTgn},
  doi = {10.52202/068431-1593},
  pages = {21916--21932}
}