Twin-delayed deep deterministic policy gradient for low-frequency oscillation damping control

Qiushi Cui, Gyoungjae Kim, Yang Weng

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Due to the large scale of power systems, latency uncertainty in communications can cause severe problems in wide-area measurement systems. To resolve this issue, a significant amount of past work focuses on using emerging technology, including machine learning methods such as Q-learning, for addressing latency issues in modern controls. Although the method can deal with the stochastic characteristics of communication latency, the Q-values can be overestimated in Q-learning methods, leading to high bias. To address the overestimation bias issue, we redesign the learning structure of the deep deterministic policy gradient (DDPG). Then we develop a damping control twin-delayed deep deterministic policy gradient method to handle the damping control issue under unknown latency in the power network. The purpose is to address the damping control issue under unknown latency in the power network. This paper will create a novel reward algorithm, taking into account the machine speed deviation, the episode termination prevention, and the feedback from action space. In this way, the system optimally damps down frequency oscillations while maintaining the system’s stability and reliable operation within defined limits. The simulation results verify the proposed algorithm in various perspectives, including the latency sensitivity analysis under high renewable energy penetration and the comparison with conventional and machine learning control algorithms. The proposed method shows a fast learning curve and good control performance under varying communication latency.

Original languageEnglish (US)
Article number6695
Issue number20
StatePublished - Oct 1 2021


  • Damping control
  • Latency
  • Low-frequency oscillations
  • Twin-delayed deep deterministic policy gradient
  • Wide-area measurement systems

ASJC Scopus subject areas

  • Renewable Energy, Sustainability and the Environment
  • Fuel Technology
  • Energy Engineering and Power Technology
  • Energy (miscellaneous)
  • Control and Optimization
  • Electrical and Electronic Engineering


Dive into the research topics of 'Twin-delayed deep deterministic policy gradient for low-frequency oscillation damping control'. Together they form a unique fingerprint.

Cite this