TY - GEN
T1 - Random-sampling multipath hypothesis propagation for cost approximation in long-horizon optimal control
AU - Ragi, Shankarachary
AU - Mittelmann, Hans D.
N1 - Funding Information:
This work was supported in part by Air Force Office of Scientific Research under grant FA9550-19-1-0070.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/8
Y1 - 2020/8
N2 - In this paper, we develop a Monte-Carlo based heuristic approach to approximate the objective function in long horizon optimal control problems. In this approach, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the weighted average of the costs along all the trajectories. We call these methods random sampling - multipath hypothesis propagation or RS-MHP. These methods (or variants) exist in the literature; however, the literature lacks convergence results for a generic class of nonlinear systems. This paper fills this knowledge gap to a certain extent. We derive convergence results for the cost approximation error from the MHP methods and discuss their convergence (in probability) as the sample size increases. As a case study, we apply RS-MHP to approximate the cost function in a linear quadratic control problem and demonstrate the benefits of our approach against an existing and closely related approximation approach called nominal belief-state optimization.
AB - In this paper, we develop a Monte-Carlo based heuristic approach to approximate the objective function in long horizon optimal control problems. In this approach, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the weighted average of the costs along all the trajectories. We call these methods random sampling - multipath hypothesis propagation or RS-MHP. These methods (or variants) exist in the literature; however, the literature lacks convergence results for a generic class of nonlinear systems. This paper fills this knowledge gap to a certain extent. We derive convergence results for the cost approximation error from the MHP methods and discuss their convergence (in probability) as the sample size increases. As a case study, we apply RS-MHP to approximate the cost function in a linear quadratic control problem and demonstrate the benefits of our approach against an existing and closely related approximation approach called nominal belief-state optimization.
KW - Approximate dynamic programming
KW - Cost approximation
KW - Long horizon optimal control
KW - Multipath hypothesis propagation
UR - http://www.scopus.com/inward/record.url?scp=85094131633&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094131633&partnerID=8YFLogxK
U2 - 10.1109/CCTA41146.2020.9206334
DO - 10.1109/CCTA41146.2020.9206334
M3 - Conference contribution
AN - SCOPUS:85094131633
T3 - CCTA 2020 - 4th IEEE Conference on Control Technology and Applications
SP - 14
EP - 18
BT - CCTA 2020 - 4th IEEE Conference on Control Technology and Applications
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 4th IEEE Conference on Control Technology and Applications, CCTA 2020
Y2 - 24 August 2020 through 26 August 2020
ER -