Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance

Jennie Si, Lei Yang, Chao Lu, Konstantinos Tsakalis, Armando Rodriguez

Research output: Chapter in Book/Report/Conference proceedingChapter

2 Scopus citations


This chapter discusses nonlinear control system design using approximate/adaptive dynamic programming (ADP). ADP algorithms based on learning and approximation have shown great promise to reduce the curses of dimensionality suffered by dynamic programming (DP). They benefited from the design techinques of artificial neural networks and other function approximators, which have developed principled ways for universal function approximation. Direct heuristic dynamic programming (HDP) was introduced as an on-line learning control scheme inspired by adaptive critique designs, a family of ADP algorithms. Applications of the direct HDP to large and complex problems have demonstrated the feasibility and scalability of the learning controller design. The results, such as Apache helicopter control and coordination of large power networks for damping low-frequency oscillation, are encouraging and promising as proof of concepts toward scalable ADP designs, however, real controllers demand performance assurances, not merely a statistical learning success rate indicating that most of the time the controller works. With this in mind, this chapter discusses some recent developments in this direction.

Original languageEnglish (US)
Title of host publicationReinforcement Learning and Approximate Dynamic Programming for Feedback Control
PublisherJohn Wiley and Sons
Number of pages21
ISBN (Print)9781118104200
StatePublished - Feb 7 2013


  • ADP, reducing DP curse of dimensionality
  • Adaptive critique, control output/value
  • Direct HDP, sensitivity maps in action/critic
  • Nonlinear ADP, performance assurance
  • Nonlinear control design using ADP

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance'. Together they form a unique fingerprint.

Cite this