Hierarchical Multi-Label Classification of Online Vaccine Concerns
CoRR(2024)
摘要
Vaccine concerns are an ever-evolving target, and can shift quickly as seen
during the COVID-19 pandemic. Identifying longitudinal trends in vaccine
concerns and misinformation might inform the healthcare space by helping public
health efforts strategically allocate resources or information campaigns. We
explore the task of detecting vaccine concerns in online discourse using large
language models (LLMs) in a zero-shot setting without the need for expensive
training datasets. Since real-time monitoring of online sources requires
large-scale inference, we explore cost-accuracy trade-offs of different
prompting strategies and offer concrete takeaways that may inform choices in
system designs for current applications. An analysis of different prompting
strategies reveals that classifying the concerns over multiple passes through
the LLM, each consisting a boolean question whether the text mentions a vaccine
concern or not, works the best. Our results indicate that GPT-4 can strongly
outperform crowdworker accuracy when compared to ground truth annotations
provided by experts on the recently introduced VaxConcerns dataset, achieving
an overall F1 score of 78.7
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要