Learning and Adaptation for Millimeter-Wave Beam Tracking and Training: A Dual Timescale Variational Framework

Muddassar Hussain, Nicolo Michelusi

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Millimeter-wave vehicular networks incur enormous beam-training overhead to enable narrow-beam communications. This paper proposes a learning and adaptation framework in which the dynamics of the communication beams are learned and then exploited to design adaptive beam-tracking and training with low overhead: on a long-timescale, a deep recurrent variational autoencoder (DR-VAE) uses noisy beam-training feedback to learn a probabilistic model of beam dynamics and enable predictive beam-tracking; on a short-timescale, an adaptive beam-training procedure is formulated as a partially observable (PO-) Markov decision process (MDP) and optimized via point-based value iteration (PBVI) by leveraging beam-training feedback and a probabilistic prediction of the strongest beam pair provided by the DR-VAE. In turn, beam-training feedback is used to refine the DR-VAE via stochastic gradient ascent in a continuous process of learning and adaptation. The proposed DR-VAE learning framework learns accurate beam dynamics: it reduces the Kullback-Leibler divergence between the ground truth and the learned model of beam dynamics by 95% over the Baum-Welch algorithm and a naive learning approach that neglects feedback errors. Numerical results on a line-of-sight scenario with multipath and 3D beamforming reveal that the proposed dual timescale approach yields near-optimal spectral efficiency, and improves it by 130% over a policy that scans exhaustively over the dominant beam pairs, and by 20% over a state-of-the-art POMDP policy. Finally, a low-complexity policy is proposed by reducing the POMDP to an error-robust MDP, and is shown to perform well in regimes with infrequent feedback errors.

Original languageEnglish (US)
Pages (from-to)37-53
Number of pages17
JournalIEEE Journal on Selected Areas in Communications
Issue number1
StatePublished - Jan 1 2022


  • Beam alignment
  • Beam tracking
  • Deep recurrent variational autoencoders (DR-VAEs)
  • Millimeter-wave (mm-wave)
  • Point-based value iteration (PBVI)

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Learning and Adaptation for Millimeter-Wave Beam Tracking and Training: A Dual Timescale Variational Framework'. Together they form a unique fingerprint.

Cite this