3M-RL: Multi-Resolution, Multi-Agent, Mean-Field Reinforcement Learning for Autonomous UAV Routing

Weichang Wang, Yongming Liu, Rayadurgam Srikant, Lei Ying

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Collision-free path planning is a major challenge in managing unmanned aerial vehicles (UAVs) fleets, especially in uncertain environments. In this paper, we consider the design of UAV routing policies using multi-agent reinforcement learning, and propose a Multi-resolution, Multi-agent, Mean-field reinforcement learning algorithm, named 3M-RL, for flight planning, where multiple vehicles need to avoid collisions with each other while moving towards their destinations. In the system we consider, each UAV makes decisions based on local observations, and does not communicate with other UAVs. The algorithm trains a routing policy using an Actor-Critic neural network with multi-resolution observations, including detailed local information and aggregated global information based on mean-field. The algorithm tackles the curse-of-dimensionality problem in multi-agent reinforcement learning and provides a scalable solution. We test our algorithm in different complex scenarios in both 2D and 3D space and our simulation results show that 3M-RL result in good routing policies.

Original languageEnglish (US)
Pages (from-to)8985-8996
Number of pages12
JournalIEEE Transactions on Intelligent Transportation Systems
Issue number7
StatePublished - Jul 1 2022


  • Multiagent reinforcement learning
  • actor-critic
  • mean-field

ASJC Scopus subject areas

  • Mechanical Engineering
  • Automotive Engineering
  • Computer Science Applications


Dive into the research topics of '3M-RL: Multi-Resolution, Multi-Agent, Mean-Field Reinforcement Learning for Autonomous UAV Routing'. Together they form a unique fingerprint.

Cite this