Data-driven Rollout for Deterministic Optimal Control

Yuchao Li; Karl H. Johansson; Jonas Martensson; DImitri P. Bertsekas

doi:10.1109/CDC45484.2021.9683499

Data-driven Rollout for Deterministic Optimal Control

Yuchao Li, Karl H. Johansson, Jonas Martensson, DImitri P. Bertsekas

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.

Original language	English (US)
Title of host publication	60th IEEE Conference on Decision and Control, CDC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2169-2176
Number of pages	8
ISBN (Electronic)	9781665436595
DOIs	https://doi.org/10.1109/CDC45484.2021.9683499
State	Published - 2021
Event	60th IEEE Conference on Decision and Control, CDC 2021 - Austin, United States Duration: Dec 13 2021 → Dec 17 2021

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2021-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	60th IEEE Conference on Decision and Control, CDC 2021
Country/Territory	United States
City	Austin
Period	12/13/21 → 12/17/21

ASJC Scopus subject areas

Control and Systems Engineering
Modeling and Simulation
Control and Optimization

Access to Document

10.1109/CDC45484.2021.9683499

Cite this

Li, Y., Johansson, K. H., Martensson, J., & Bertsekas, DI. P. (2021). Data-driven Rollout for Deterministic Optimal Control. In 60th IEEE Conference on Decision and Control, CDC 2021 (pp. 2169-2176). (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC45484.2021.9683499

Data-driven Rollout for Deterministic Optimal Control. / Li, Yuchao; Johansson, Karl H.; Martensson, Jonas et al.
60th IEEE Conference on Decision and Control, CDC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 2169-2176 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Li, Y, Johansson, KH, Martensson, J & Bertsekas, DIP 2021, Data-driven Rollout for Deterministic Optimal Control. in 60th IEEE Conference on Decision and Control, CDC 2021. Proceedings of the IEEE Conference on Decision and Control, vol. 2021-December, Institute of Electrical and Electronics Engineers Inc., pp. 2169-2176, 60th IEEE Conference on Decision and Control, CDC 2021, Austin, United States, 12/13/21. https://doi.org/10.1109/CDC45484.2021.9683499

@inproceedings{e0cbe35c04554c8eaf2005f4cbacc0a0,

title = "Data-driven Rollout for Deterministic Optimal Control",

abstract = "We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.",

author = "Yuchao Li and Johansson, {Karl H.} and Jonas Martensson and Bertsekas, {DImitri P.}",

note = "Funding Information: This work was supported by the Swedish Foundation for Strategic Research, the Swedish Research Council, and the Knut and Alice Wallenberg Foundation. Publisher Copyright: {\textcopyright} 2021 IEEE.; 60th IEEE Conference on Decision and Control, CDC 2021 ; Conference date: 13-12-2021 Through 17-12-2021",

year = "2021",

doi = "10.1109/CDC45484.2021.9683499",

language = "English (US)",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2169--2176",

booktitle = "60th IEEE Conference on Decision and Control, CDC 2021",

}

TY - GEN

T1 - Data-driven Rollout for Deterministic Optimal Control

AU - Li, Yuchao

AU - Johansson, Karl H.

AU - Martensson, Jonas

AU - Bertsekas, DImitri P.

N1 - Funding Information: This work was supported by the Swedish Foundation for Strategic Research, the Swedish Research Council, and the Knut and Alice Wallenberg Foundation. Publisher Copyright: © 2021 IEEE.

PY - 2021

Y1 - 2021

N2 - We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.

AB - We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.

UR - http://www.scopus.com/inward/record.url?scp=85126000020&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85126000020&partnerID=8YFLogxK

U2 - 10.1109/CDC45484.2021.9683499

DO - 10.1109/CDC45484.2021.9683499

M3 - Conference contribution

AN - SCOPUS:85126000020

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 2169

EP - 2176

BT - 60th IEEE Conference on Decision and Control, CDC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 60th IEEE Conference on Decision and Control, CDC 2021

Y2 - 13 December 2021 through 17 December 2021

ER -

Data-driven Rollout for Deterministic Optimal Control

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this