Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
CoRR(2024)
摘要
Audits are critical mechanisms for identifying the risks and limitations of
deployed artificial intelligence (AI) systems. However, the effective execution
of AI audits remains incredibly difficult. As a result, practitioners make use
of various tools to support their efforts. Drawing on interviews with 35 AI
audit practitioners and a landscape analysis of 390 tools, we map the current
ecosystem of available AI audit tools. While there are many tools designed to
assist practitioners with setting standards and evaluating AI systems, these
tools often fell short of supporting the accountability goals of AI auditing in
practice. We thus highlight areas for future tool development beyond evaluation
– from harms discovery to advocacy – and outline challenges practitioners
faced in their efforts to use AI audit tools. We conclude that resources are
lacking to adequately support the full scope of needs for many AI audit
practitioners and recommend that the field move beyond tools for just
evaluation, towards more comprehensive infrastructure for AI accountability.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要