Robotic Knee Parameter Tuning Using Approximate Policy Iteration

Xiang Gao, Yue Wen, Minhan Li, Jennie Si, He (Helen) Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations


This paper presents an online model-free reinforcement learning based controller realized by approximate dynamic programming for a robotic knee as part of a human-machine system. Traditionally, prosthesis wearers’ gait performance is improved by manually tuning the impedance parameters. In this paper, we show that the parameter tuning problem can be formulated as an optimal control problem and thus solved by dynamic programming. Toward this goal, we constructed an quadratic instantaneous cost, which resulted in a value function that could be approximated by a neural network. The control policy is then solved by the least-squared method iteratively, a framework of which we refer to as approximate policy iteration. We performed extensive simulations based on prosthetic kinetics and human performance data extracted from real human subjects. Our results show that the proposed parameter tuning algorithm can be readily used for adaptive optimal tuning of prosthetic knee control parameters and the tuning process is time and sample efficient.

Original languageEnglish (US)
Title of host publicationCognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers
EditorsDewen Hu, Huaping Liu, Fuchun Sun
PublisherSpringer Verlag
Number of pages10
ISBN (Print)9789811379826
StatePublished - 2019
Event4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018 - Beijing, China
Duration: Nov 29 2018Dec 1 2018

Publication series

NameCommunications in Computer and Information Science
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937


Conference4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018


  • Approximate dynamic programming (ADP)
  • Lower limb prosthesis
  • Policy iteration
  • Sample efficient learning

ASJC Scopus subject areas

  • General Computer Science
  • General Mathematics


Dive into the research topics of 'Robotic Knee Parameter Tuning Using Approximate Policy Iteration'. Together they form a unique fingerprint.

Cite this