An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification
arxiv(2024)
摘要
Predictive models may generate biased predictions when classifying imbalanced
datasets. This happens when the model favors the majority class, leading to low
performance in accurately predicting the minority class. To address this issue,
balancing or resampling methods are critical pre-processing steps in the
modeling process. However, there have been debates and questioning of the
functionality of these methods in recent years. In particular, many candidate
models may exhibit very similar predictive performance, which is called the
Rashomon effect, in model selection. Selecting one of them without considering
predictive multiplicity which is the case of yielding conflicting models'
predictions for any sample may lead to a loss of using another model. In this
study, in addition to the existing debates, the impact of balancing methods on
predictive multiplicity is examined through the Rashomon effect. It is
important because the blind model selection is risky from a set of
approximately equally accurate models. This may lead to serious problems in
model selection, validation, and explanation. To tackle this matter, we
conducted real dataset experiments to observe the impact of balancing methods
on predictive multiplicity through the Rashomon effect. Our findings showed
that balancing methods inflate the predictive multiplicity, and they yield
varying results. To monitor the trade-off between performance and predictive
multiplicity for conducting the modeling process responsibly, we proposed using
the extended performance-gain plot for the Rashomon effect.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要