![](https://d3ilqtpdwi981i.cloudfront.net/MML2q5FfV6Y-ZJxAuDfK0__SyFQ=/425x550/smart/https://bepress-attached-resources.s3.amazonaws.com/uploads/9d/06/d0/9d06d0cd-f569-43c3-bba1-f488bedfe068/thumbnail_663770e9-7ba4-425b-bb8d-f5ce7caeb17c.jpg)
This article presents a novel efficient experience-replay-based adaptive dynamic programming (ADP) for the optimal control problem of a class of nonlinear dynamical systems within the Hamiltonian-driven framework. The quasi-Hamiltonian is presented for the policy evaluation problem with an admissible policy. With the quasi-Hamiltonian, a novel composite critic learning mechanism is developed to combine the instantaneous data with the historical data. In addition, the pseudo-Hamiltonian is defined to deal with the performance optimization problem. Based on the pseudo-Hamiltonian, the conventional Hamilton–Jacobi–Bellman (HJB) equation can be represented in a filtered form, which can be implemented online. Theoretical analysis is investigated in terms of the convergence of the adaptive critic design and the stability of the closed-loop systems, where parameter convergence can be achieved under a weakened excitation condition. Simulation studies are investigated to verify the efficacy of the presented design scheme.
- Convergence,
- Dynamic programming,
- Hamiltonian-driven adaptive dynamic programming (ADP),
- Hamilton–Jacobi–Bellman (HJB) equation,
- Iterative algorithms,
- Learning systems,
- Mathematical models,
- Optimal control,
- Optimization,
- pseudo-Hamiltonian,
- quasi-Hamiltonian,
- relaxed excitation condition
Available at: http://works.bepress.com/donald-wunsch/459/