Skip to main content
Article
Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Interconnected Systems using Hybrid Q-Learning Approach
IET Control Theory and Applications
  • Vignesh Narayanan
  • Jagannathan Sarangapani, Missouri University of Science and Technology
Abstract

A novel hybrid Q-learning algorithm is introduced for the design of a linear adaptive optimal regulator for a large-scale interconnected system with event-sampled inputs and state vector. Here, the time-driven Q-learning along with proposed iterative parameter learning updates are utilised within the event-sampled instants to both improve efficiency of the optimal regulator and obtain a more generalised online Q-learning framework. The network-induced losses due to the presence of a communication network among the subsystems are considered along with the uncertain system dynamics. Stochastic model-free Q-learning and dynamic programming are utilised in the hybrid learning mode for the optimal regulator design. The asymptotic convergence of the system state vector and boundedness of the parameter vector is demonstrated using Lyapunov analysis. Further, when the regression vector of the Q-function estimator satisfies the persistency of excitation condition, the Q-function parameters converge to the expected target values. The analytical design is evaluated using numerical examples via simulation. The net result is the design of a data-driven event-sampled adaptive optimal regulator for an uncertain large-scale interconnected system.

Department(s)
Electrical and Computer Engineering
Research Center/Lab(s)
Intelligent Systems Center
Comments
This research supported in part by NSF ECCS #1128281 and #1406533 and Intelligent Systems Center, at the Missouri University of Science and Technology, Rolla.
Keywords and Phrases
  • Algorithms,
  • Design,
  • Dynamic programming,
  • Iterative methods,
  • Large scale systems,
  • Stochastic control systems,
  • Stochastic models,
  • Stochastic systems,
  • Telecommunication networks,
  • Uncertainty analysis,
  • Vectors,
  • Asymptotic convergence,
  • Large-scale interconnected systems,
  • Lyapunov analysis,
  • Optimal regulators,
  • Parameter learning,
  • Parameter vectors,
  • Persistency of excitation,
  • Regression vectors,
  • Learning algorithms
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2016 Institution of Engineering and Technology (IEE), All rights reserved.
Publication Date
8-1-2016
Publication Date
01 Aug 2016
Citation Information
Vignesh Narayanan and Jagannathan Sarangapani. "Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Interconnected Systems using Hybrid Q-Learning Approach" IET Control Theory and Applications Vol. 10 Iss. 12 (2016) p. 1448 - 1457 ISSN: 1751-8644; 1751-8652
Available at: http://works.bepress.com/jagannathan-sarangapani/176/