Measuring Feature Dependency of Neural Networks by Collapsing Feature Dimensions in the Data Manifold
arxiv(2024)
摘要
This paper introduces a new technique to measure the feature dependency of
neural network models. The motivation is to better understand a model by
querying whether it is using information from human-understandable features,
e.g., anatomical shape, volume, or image texture. Our method is based on the
principle that if a model is dependent on a feature, then removal of that
feature should significantly harm its performance. A targeted feature is
"removed" by collapsing the dimension in the data distribution that corresponds
to that feature. We perform this by moving data points along the feature
dimension to a baseline feature value while staying on the data manifold, as
estimated by a deep generative model. Then we observe how the model's
performance changes on the modified test data set, with the target feature
dimension removed. We test our method on deep neural network models trained on
synthetic image data with known ground truth, an Alzheimer's disease prediction
task using MRI and hippocampus segmentations from the OASIS-3 dataset, and a
cell nuclei classification task using the Lizard dataset.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要