MultiMedEval: A Benchmark and a Toolkit for Evaluating Medical Vision-Language Models
CoRR(2024)
摘要
We introduce MultiMedEval, an open-source toolkit for fair and reproducible
evaluation of large, medical vision-language models (VLM). MultiMedEval
comprehensively assesses the models' performance on a broad array of six
multi-modal tasks, conducted over 23 datasets, and spanning over 11 medical
domains. The chosen tasks and performance metrics are based on their widespread
adoption in the community and their diversity, ensuring a thorough evaluation
of the model's overall generalizability. We open-source a Python toolkit
(github.com/corentin-ryr/MultiMedEval) with a simple interface and setup
process, enabling the evaluation of any VLM in just a few lines of code. Our
goal is to simplify the intricate landscape of VLM evaluation, thus promoting
fair and uniform benchmarking of future models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要