Skip to main content
Article
On the Identifiability of Nonlinear ICA: Sparsity and Beyond
arXiv
  • Yujia Zheng, Carnegie Mellon University, United States
  • Ignavier Ng, Carnegie Mellon University, United States
  • Kun Zhang, Carnegie Mellon University & Mohamed bin Zayed University of Artificial Intelligence
Document Type
Article
Abstract

Nonlinear independent component analysis (ICA) aims to recover the underlying independent latent sources from their observable nonlinear mixtures. How to make the nonlinear ICA model identifiable up to certain trivial indeterminacies is a longstanding problem in unsupervised learning. Recent breakthroughs reformulate the standard independence assumption of sources as conditional independence given some auxiliary variables (e.g., class labels and/or domain/time indexes) as weak supervision or inductive bias. However, nonlinear ICA with unconditional priors cannot benefit from such developments. We explore an alternative path and consider only assumptions on the mixing process, such as Structural Sparsity or Independent Influences. We show that under specific instantiations of such constraints, the independent latent sources can be identified from their nonlinear mixtures up to a permutation and a component-wise transformation, thus achieving nontrivial identifiability of nonlinear ICA without auxiliary variables. We provide estimation methods and validate the theoretical results experimentally. The results on image data suggest that our conditions may hold in a number of practical data generating processes. Copyright © 2022, The Authors. All rights reserved.

DOI
10.48550/arXiv.2206.07751
Publication Date
6-15-2022
Keywords
  • Independent component analysis,
  • Machine learning,
  • Analysis models,
  • Analysis sparsities,
  • Auxiliary variables,
  • Class labels,
  • Conditional independences,
  • Identifiability,
  • Independence assumption,
  • Nonlinear independent component analysis,
  • Nonlinear mixtures,
  • Time index,
  • Mixtures,
  • Artificial Intelligence (cs.AI),
  • Machine Learning (cs.LG),
  • Machine Learning (stat.ML)
Comments

Preprints available on arXiv

Citation Information
Y. Zheng, I. Ng, and K. Zhang, "On the Identifiability of Nonlinear ICA: Sparsity and Beyond", 2022, arXiv:2206.07751