The Ethics of AI: (How) Does Accountability Require Understanding Machine Learning Models?

Concerns about artificial intelligence—particularly systems based on machine learning models whose internal operations are opaque to human understanding—are frequently framed in terms of accountability. But these concerns are often left underspecified, encompassing a heterogeneous set of issues that call for distinct responses.

This talk focuses on a specific subset of accountability concerns: the “who” questions. We distinguish between two questions: When an AI system causes harm, who is to blame, and who bears the obligation to provide compensation? We argue that the answers to these two questions require different information and don’t entail one another.

Then we investigate whether the opacity of contemporary machine learning models undermines our capacity to hold the relevant actors accountable in these two senses of accountability. What is required, we argue, is not understanding of the internal processes of the particular machine learning model deployed, but rather, a general understanding of how upstream decisions in the development process shape downstream model behavior and internal processes. We conclude with various practical implications of our discussion. Among others, we argue for a pro tanto moral obligation to document contextual and historical information in the process of developing machine learning models.