TY - GEN
T1 - Intrinsic-Motivated Sensor Management
T2 - 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022
AU - Yuan, Jingyi
AU - Weng, Yang
AU - Blasch, Erik
N1 - Publisher Copyright:
© 2022 Owner/Author.
PY - 2022/8/14
Y1 - 2022/8/14
N2 - In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.
AB - In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.
KW - intrinsic motivation
KW - learning-based control
KW - neural networks
KW - power systems
KW - reinforcement learning
KW - sensor management
KW - unmanned vehicles
UR - http://www.scopus.com/inward/record.url?scp=85137141926&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85137141926&partnerID=8YFLogxK
U2 - 10.1145/3534678.3539230
DO - 10.1145/3534678.3539230
M3 - Conference contribution
AN - SCOPUS:85137141926
T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
SP - 2399
EP - 2407
BT - KDD 2022 - Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
PB - Association for Computing Machinery
Y2 - 14 August 2022 through 18 August 2022
ER -