Skip to main content
Article
Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems
IEEE Transactions on Cybernetics
  • Vignesh Narayanan
  • Hamidreza Modares
  • Jagannathan Sarangapani, Missouri University of Science and Technology
  • Frank L. Lewis
Abstract

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

Department(s)
Electrical and Computer Engineering
Comments
This work was supported in part by NSF under Grant ECCS 1406533 and Grant CMMI 1547042.
Keywords and Phrases
  • Decentralized Control,
  • Differential Game,
  • Event-Triggered Learning,
  • Off-Policy Learning
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2022 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
3-1-2022
Publication Date
01 Mar 2022
PubMed ID
32639933
Citation Information
Vignesh Narayanan, Hamidreza Modares, Jagannathan Sarangapani and Frank L. Lewis. "Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems" IEEE Transactions on Cybernetics Vol. 52 Iss. 3 (2022) p. 1936 - 1946 ISSN: 2168-2275; 2168-2267
Available at: http://works.bepress.com/jagannathan-sarangapani/262/