What's my role? Modelling responsibility for AI-based safety-critical systems
CoRR(2023)
摘要
AI-Based Safety-Critical Systems (AI-SCS) are being increasingly deployed in
the real world. These can pose a risk of harm to people and the environment.
Reducing that risk is an overarching priority during development and operation.
As more AI-SCS become autonomous, a layer of risk management via human
intervention has been removed. Following an accident it will be important to
identify causal contributions and the different responsible actors behind those
to learn from mistakes and prevent similar future events. Many authors have
commented on the "responsibility gap" where it is difficult for developers and
manufacturers to be held responsible for harmful behaviour of an AI-SCS. This
is due to the complex development cycle for AI, uncertainty in AI performance,
and dynamic operating environment. A human operator can become a "liability
sink" absorbing blame for the consequences of AI-SCS outputs they weren't
responsible for creating, and may not have understanding of.
This cross-disciplinary paper considers different senses of responsibility
(role, moral, legal and causal), and how they apply in the context of AI-SCS
safety. We use a core concept (Actor(A) is responsible for Occurrence(O)) to
create role responsibility models, producing a practical method to capture
responsibility relationships and provide clarity on the previously identified
responsibility issues. Our paper demonstrates the approach with two examples: a
retrospective analysis of the Tempe Arizona fatal collision involving an
autonomous vehicle, and a safety focused predictive role-responsibility
analysis for an AI-based diabetes co-morbidity predictor. In both examples our
primary focus is on safety, aiming to reduce unfair or disproportionate blame
being placed on operators or developers. We present a discussion and avenues
for future research.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要