Knowledge-Guided Reinforcement Learning Control for Robotic Lower Limb Prosthesis

Xiang Gao, Jennie Si, Yue Wen, Minhan Li, He Helen Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations


Robotic prostheses provide new opportunities to better restore lost functions than passive prostheses for trans-femoral amputees. But controlling a prosthesis device automatically for individual users in different task environments is an unsolved problem. Reinforcement learning (RL) is a naturally promising tool. For prosthesis control with a user in the loop, it is desirable that the controlled prosthesis can adapt to different task environments as quickly and smoothly as possible. However, most RL agents learn or relearn from scratch when the environment changes. To address this issue, we propose the knowledge-guided Q-learning (KG-QL) control method as a principled way for the problem. In this report, we collected and used data from two able-bodied (AB) subjects wearing a RL controlled robotic prosthetic limb walking on level ground. Our ultimate goal is to build an efficient RL controller with reduced time and data requirements and transfer knowledge from AB subjects to amputee subjects. Toward this goal, we demonstrate its feasibility by employing OpenSim, a well-established human locomotion simulator. Our results show the OpenSim simulated amputee subject improved control tuning performance over learning from scratch by utilizing knowledge transfer from AB subjects. Also in this paper, we will explore the possibility of information transfer from AB subjects to help tuning for the amputee subjects.

Original languageEnglish (US)
Title of host publication2020 IEEE International Conference on Robotics and Automation, ICRA 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781728173955
StatePublished - May 2020
Event2020 IEEE International Conference on Robotics and Automation, ICRA 2020 - Paris, France
Duration: May 31 2020Aug 31 2020

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729


Conference2020 IEEE International Conference on Robotics and Automation, ICRA 2020

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering


Dive into the research topics of 'Knowledge-Guided Reinforcement Learning Control for Robotic Lower Limb Prosthesis'. Together they form a unique fingerprint.

Cite this