Rethinking Diversity in Deep Neural Network Testing
CoRR(2023)
摘要
Motivated by the success of traditional software testing, numerous diversity
measures have been proposed for testing deep neural networks (DNNs). In this
study, we propose a shift in perspective, advocating for the consideration of
DNN testing as directed testing problems rather than diversity-based testing
tasks. We note that the objective of testing DNNs is specific and well-defined:
identifying inputs that lead to misclassifications. Consequently, a more
precise testing approach is to prioritize inputs with a higher potential to
induce misclassifications, as opposed to emphasizing inputs that enhance
"diversity."
We derive six directed metrics for DNN testing. Furthermore, we conduct a
careful analysis of the appropriate scope for each metric, as applying metrics
beyond their intended scope could significantly diminish their effectiveness.
Our evaluation demonstrates that (1) diversity metrics are particularly weak
indicators for identifying buggy inputs resulting from small input
perturbations, and (2) our directed metrics consistently outperform diversity
metrics in revealing erroneous behaviors of DNNs across all scenarios.
更多查看译文
关键词
deep learning testing,diversity
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要