Datamodels: Predicting Predictions with Training Data

Machine learning models tend to rely on an abundance of training data. Yet, understanding the underlying structure of this data---and models' exact dependence on it---remains a challenge.

In this talk, we will present a new framework---called datamodeling---for directly modeling predictions as functions of training data. This datamodeling framework, given a dataset and a learning algorithm, pinpoints---at varying levels of granularity---the relationships between train and test point pairs through the lens of the corresponding model class. Even in its most basic version, datamodels enable many applications, including discovering subpopulations, quantifying model brittleness via counterfactuals, and identifying train-test leakage.

Please report video problems to itsupport@slmath.org.

See more of our Streaming videos on our main VMath Videos page.