Events

Why did the model fail?: Attributing model performance changes to distribution shifts

Speaker: Shalmali Joshi

Location: 60 Fifth Avenue, Room 7th floor common area

Date: Wednesday, May 3, 2023

To enable safe learning-enabled systems, especially in healthcare, we need robust tools for diagnosing and monitoring Machine Learning (ML) behavior, including characterizing their failure modes. One such challenge is that performance of ML models may differ between training and deployment settings for many reasons, many of which manifest as distribution shifts in the underlying data-generating mechanism. How do we characterize which distribution shifts explain the model’s performance changes? In this talk, we will introduce the problem of attributing ML performance differences between environments to shifts in the underlying data-generating mechanisms. We formulate attribution as a cooperative game, where mechanisms/distributions are players that form coalitions to change model performance. This formulation allows us to attribute the total performance change to each shifted distribution using Shapley values. We demonstrate the utility of our explanations for evaluating the transportability of prediction models in healthcare and providing actionable steps to mitigate performance deterioration.