Skip to main content
Article
Towards Principled Disentanglement for Domain Generalization
arXiv
  • Hanlin Zhang, Carnegie Mellon University, United States
  • Yi-Fan Zhang, Chinese Academy of Science, China
  • Weiyang Liu, University of Cambridge, United Kingdom & Max Planck Institute for Intelligent Systems, Tübingen, Germany
  • Adrian Weller, University of Cambridge, United Kingdom & Alan Turing Institute, United Kingdom
  • Bernhard Schölkopf, Max Planck Institute for Intelligent Systems, Tübingen, Germany
  • Eric P. Xing, Carnegie Mellon University, United States & Mohamed bin Zayed University of Artificial Intelligence
Document Type
Article
Abstract

A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data, in part due to spurious correlations. To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG). We relax this non-trivial constrained optimization problem to a tractable form with finite-dimensional parameterization and empirical approximation. Then a theoretical analysis of the extent to which the above transformations deviates from the original problem is provided. Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization. In contrast to traditional approaches based on domain adversarial training and domain labels, DDG jointly learns semantic and variation encoders for disentanglement, enabling flexible manipulation and augmentation on training data. DDG aims to learn intrinsic representations of semantic concepts that are invariant to nuisance factors and generalizable across domains. Comprehensive experiments on popular benchmarks show that DDG can achieve competitive OOD performance and uncover interpretable salient structures within data. Copyright © 2021, The Authors. All rights reserved.

DOI
10.48550/arXiv.2111.13839
Publication Date
11-27-2021
Keywords
  • Benchmarking,
  • Machine learning,
  • Semantics,
  • Constrained domain,
  • Constrained optimi-zation problems,
  • Dimensional parameterization,
  • Empirical approximations,
  • Finite dimensional,
  • Generalisation,
  • Learn+,
  • Machine learning models,
  • Non-trivial,
  • Primal-dual algorithms,
  • Constrained optimization,
  • Computer Vision and Pattern Recognition (cs.CV),
  • Machine Learning (cs.LG)
Comments

IR Deposit conditions: non-described

Preprint: arXiv

Citation Information
H. Zhang, Y.F. Zhang, W. Liu, A. Weller, B. Schölkopf, and E. Xing, "Towards Principled Disentanglement for Domain Generalization", arXiv, Nov 2021, doi: 10.48550/arXiv.2111.13839