Measuring the Sensitivity of Image Captioning Metrics to Caption Perturbations
Lecture notes in networks and systems(2023)
Abstract
Image captioning systems based on deep neural networks have made significant progress in generating image descriptions in fluent language. Despite the achieved success, one of the main challenges remains automatic evaluation of the results. Recently, research efforts focused on the development of increasingly sophisticated automatic metrics have intensified significantly, so finding an appropriate way to compare metrics has also gained importance. A good automatic metric should match the ratings human evaluators would assign to captions, so a common way to evaluate them is to compute the correlation with human judgments. In this paper, we chose a different approach that could complement the results of correlation analysis in order to get a more complete insight into a particular metric. In this regard, we analyzed how various caption perturbations affect the scores of selected image captioning metrics. We constructed a dataset of various caption modifications and analyzed the changes in scores resulting from such perturbations.
MoreTranslated text
Key words
image captioning metrics,sensitivity
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined