Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization

Vikas K. Garg, Adam Tauman Kalai, Katrina Ligett, Zhiwei Steven Wu

Research output: Contribution to journalConference articlepeer-review

8 Scopus citations

Abstract

Domain generalization is the problem of machine learning when the training data and the test data come from different “domains” (data distributions). We propose an elementary theoretical model of the domain generalization problem, introducing the concept of a meta-distribution over domains. In our model, the training data available to a learning algorithm consist of multiple datasets, each from a single domain, drawn in turn from the meta-distribution. We show that our model can capture a rich range of learning phenomena specific to domain generalization for three different settings: learning with Massart noise, learning decision trees, and feature selection. We demonstrate approaches that leverage domain generalization to reduce computational or data requirements in each of these settings. Experiments demonstrate that our feature selection algorithm indeed ignores spurious correlations and improves generalization.

Original languageAmerican English
Pages (from-to)3574-3582
Number of pages9
JournalProceedings of Machine Learning Research
Volume130
StatePublished - 2021
Event24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021 - Virtual, Online, United States
Duration: 13 Apr 202115 Apr 2021

Bibliographical note

Funding Information:
KL’s work was carried out in part while at Microsoft Research New England. This work was supported in part by NSF FAI grant#1939606, J.P. Morgan Faculty Award, Simons Foundation Collaboration 733792, Israel Science Foundation (ISF) grants 1044/16 and 2861/20, the United States Air Force and DARPA under contract FA8750-19-2-0222, and the Federmann Cyber Security Center in conjunction with the Israel national cyber directorate. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force and DARPA.

Funding Information:
KL's work was carried out in part while at Microsoft Research New England. This work was supported in part by NSF FAI grant#1939606, J.P. Morgan Faculty Award, Simons Foundation Collaboration 733792, Israel Science Foundation (ISF) grants 1044/16 and 2861/20, the United States Air Force and DARPA under contract FA8750-19-2-0222, and the Federmann Cyber Security Center in conjunction with the Israel national cyber directorate. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force and DARPA.

Publisher Copyright:
Copyright © 2021 by the author(s)

Fingerprint

Dive into the research topics of 'Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization'. Together they form a unique fingerprint.

Cite this