Mean-field controllability and decentralized stabilization of Markov chains

Karthik Elamvazhuthi, Matthias Kawski, Shiba Biswal, Vaibhav Deshmukh, Spring Berman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations


In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.

Original languageEnglish (US)
Title of host publication2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781509028733
StatePublished - Jan 18 2018
Event56th IEEE Annual Conference on Decision and Control, CDC 2017 - Melbourne, Australia
Duration: Dec 12 2017Dec 15 2017

Publication series

Name2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017


Other56th IEEE Annual Conference on Decision and Control, CDC 2017

ASJC Scopus subject areas

  • Decision Sciences (miscellaneous)
  • Industrial and Manufacturing Engineering
  • Control and Optimization


Dive into the research topics of 'Mean-field controllability and decentralized stabilization of Markov chains'. Together they form a unique fingerprint.

Cite this