A boundedness result for the direct heuristic dynamic programming

Feng Liu, Jian Sun, Jennie Si, Wentao Guo, Shengwei Mei

Research output: Contribution to journalArticlepeer-review

99 Scopus citations


Approximate/adaptive dynamic programming (ADP) has been studied extensively in recent years for its potential scalability to solve large state and control space problems, including those involving continuous states and continuous controls. The applicability of ADP algorithms, especially the adaptive critic designs has been demonstrated in several case studies. Direct heuristic dynamic programming (direct HDP) is one of the ADP algorithms inspired by the adaptive critic designs. It has been shown applicable to industrial scale, realistic and complex control problems. In this paper, we provide a uniformly ultimately boundedness (UUB) result for the direct HDP learning controller under mild and intuitive conditions. By using a Lyapunov approach we show that the estimation errors of the learning parameters or the weights in the action and critic networks remain UUB. This result provides a useful controller convergence guarantee for the first time for the direct HDP design.

Original languageEnglish (US)
Pages (from-to)229-235
Number of pages7
JournalNeural Networks
StatePublished - Aug 2012


  • Approximate dynamic programming (ADP)
  • Direct heuristic dynamic programming (direct HDP)
  • Lyapunov stability
  • Uniformly ultimately boundedness (UUB)

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence


Dive into the research topics of 'A boundedness result for the direct heuristic dynamic programming'. Together they form a unique fingerprint.

Cite this