Large-Scale Evaluation of Open-Set Image Classification Techniques
arxiv(2024)
Abstract
The goal for classification is to correctly assign labels to unseen samples.
However, most methods misclassify samples with unseen labels and assign them to
one of the known classes. Open-Set Classification (OSC) algorithms aim to
maximize both closed and open-set recognition capabilities. Recent studies
showed the utility of such algorithms on small-scale data sets, but limited
experimentation makes it difficult to assess their performances in real-world
problems. Here, we provide a comprehensive comparison of various OSC
algorithms, including training-based (SoftMax, Garbage, EOS) and
post-processing methods (Maximum SoftMax Scores, Maximum Logit Scores, OpenMax,
EVM, PROSER), the latter are applied on features from the former. We perform
our evaluation on three large-scale protocols that mimic real-world challenges,
where we train on known and negative open-set samples, and test on known and
unknown instances. Our results show that EOS helps to improve performance of
almost all post-processing algorithms. Particularly, OpenMax and PROSER are
able to exploit better-trained networks, demonstrating the utility of hybrid
models. However, while most algorithms work well on negative test samples –
samples of open-set classes seen during training – they tend to perform poorly
when tested on samples of previously unseen unknown classes, especially in
challenging conditions.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined