Skip to main content
Article
Domain Adversarial Training on Conditional Variational Auto-Encoder for Controllable Music Generation
arXiv
  • Jingwei Zhao, Institute of Data Science, NUS, Singapore & Integrative Sciences and Engineering Programme, NUS Graduate School, Singapore
  • Gus Xia, Music X Lab, NYU Shanghai, China & Mohamed bin Zayed University of Artificial Intelligence
  • Ye Wang, School of Computing, NUS, Singapore & Institute of Data Science, NUS, Singapore & Integrative Sciences and Engineering Programme, NUS Graduate School, Singapore
Document Type
Article
Abstract

The variational auto-encoder has become a leading framework for symbolic music generation, and a popular research direction is to study how to effectively control the generation process. A straightforward way is to control a model using different conditions during inference. However, in music practice, conditions are usually sequential (rather than simple categorical labels), involving rich information that overlaps with the learned representation. Consequently, the decoder gets confused about whether to “listen to” the latent representation or the condition, and sometimes just ignores the condition. To solve this problem, we leverage domain adversarial training to disentangle the representation from condition cues for better control. Specifically, we propose a condition corruption objective that uses the representation to denoise a corrupted condition. Minimized by a discriminator and maximized by the VAE encoder, this objective adversarially induces a condition-invariant representation. In this paper, we focus on the task of melody harmonization 1 to illustrate our idea, while our methodology can be generalized to other controllable generative tasks. Demos and experiments show that our methodology facilitates not only condition-invariant representation learning but also higher-quality controllability compared to baselines. © 2022, CC BY

DOI
10.48550/arXiv.2209.07144
Publication Date
9-15-2022
Keywords
  • Music,
  • Auto encoders,
  • Condition,
  • De-Noise,
  • Generation process,
  • Harmonisation,
  • High quality,
  • Invariant representation,
  • Simple++
Comments

Preprint: arXiv

Archived with thanks to arXiv

Preprint License: CC by 4.0

Uploaded 31 October 2022

Citation Information
J. Zhao, G. Xia, and Y. Wang, "Domain Adversarial Training on Conditional Variational Auto-Encoder for Controllable Music Generation", 2022, doi10.48550/arXiv.2209.07144