OpenXAI: Towards a Transparent Evaluation of Model Explanations
arxiv(2022)
摘要
While several types of post hoc explanation methods have been proposed in
recent literature, there is very little work on systematically benchmarking
these methods. Here, we introduce OpenXAI, a comprehensive and extensible
open-source framework for evaluating and benchmarking post hoc explanation
methods. OpenXAI comprises of the following key components: (i) a flexible
synthetic data generator and a collection of diverse real-world datasets,
pre-trained models, and state-of-the-art feature attribution methods, and (ii)
open-source implementations of eleven quantitative metrics for evaluating
faithfulness, stability (robustness), and fairness of explanation methods, in
turn providing comparisons of several explanation methods across a wide variety
of metrics, models, and datasets. OpenXAI is easily extensible, as users can
readily evaluate custom explanation methods and incorporate them into our
leaderboards. Overall, OpenXAI provides an automated end-to-end pipeline that
not only simplifies and standardizes the evaluation of post hoc explanation
methods, but also promotes transparency and reproducibility in benchmarking
these methods. While the first release of OpenXAI supports only tabular
datasets, the explanation methods and metrics that we consider are general
enough to be applicable to other data modalities. OpenXAI datasets and models,
implementations of state-of-the-art explanation methods and evaluation metrics,
are publicly available at this GitHub link.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要