Skip to main content
profile image
Selected Works of Martin Takáč
Article
On Scaled Methods for Saddle Point Problems
arXiv
  • Aleksandr Beznosikov, oscow Institute of Physics and Technology (MIPT), Moscow, Russian Federation
  • Aibek Alanov, Artificial Intelligence Research Institute (AIRI), Higher School of Economics (HSE), Moscow, Russian Federation
  • Dmitry Kovalev, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
  • Martin Takac, Mohamed Bin Zayed University of Artificial Intelligence
  • Alexander Gasnikov, Moscow Institute of Physics and Technology (MIPT), Moscow, Russian Federation
Link
Document Type
Article
Abstract

Methods with adaptive scaling of different features play a key role in solving saddle point problems, primarily due to Adam’s popularity for solving adversarial machine learning problems, including GANS training. This paper carries out a theoretical analysis of the following scaling techniques for solving SPPs: the well-known Adam and RmsProp scaling and the newer AdaHessian and OASIS based on Hutchison approximation. We use the Extra Gradient and its improved version with negative momentum as the basic method. Experimental studies on GANs show good applicability not only for Adam, but also for other less popular methods. Copyright © 2022, The Authors. All rights reserved.

DOI
10.48550/arXiv.2206.08303
Publication Date
6-16-2022
Keywords
  • Adaptive scaling,
  • Hutchison,
  • Machine learning problem,
  • Saddle point problems,
  • Scaled methods,
  • Scalings,
  • Machine learning,
  • Machine Learning (cs.LG),
  • Optimization and Control (math.OC)
Disciplines
Comments

IR Deposit conditions: non-described

Preprint available on arXiv

Citation Information
A. Beznosikov, A. Alanov, D. Kovalev, M. Takac, and A. Gasnikov, "On Scaled Methods for Saddle Point Problems", 2022, arXiv:2206.08303