Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Prediction

Muhammad Alrabeiah, Andrew Hredzak, Ahmed Alkhateeb

Research output: Chapter in Book/Report/Conference proceedingConference contribution

92 Scopus citations


This paper investigates a novel research direction that leverages vision to help overcome the critical wireless communication challenges. In particular, this paper considers millimeter wave (mmWave) communication systems, which are principal components of 5G and beyond. These systems face two important challenges: (i) the large training overhead associated with selecting the optimal beam and (ii) the reliability challenge due to the high sensitivity to link blockages. Interestingly, most of the devices that employ mmWave arrays will likely also use cameras, such as 5G phones, self-driving vehicles, and virtual/augmented reality headsets. Therefore, we investigate the potential gains of employing cameras at the mmWave base stations and leveraging their visual data to help overcome the beam selection and blockage prediction challenges. To do that, this paper exploits computer vision and deep learning tools to predict mmWave beams and blockages directly from the camera RGB images and the sub-6GHz channels. The experimental results reveal interesting insights into the effectiveness of such solutions. For example, the deep learning model is capable of achieving over 90% beam prediction accuracy, which only requires snapping a shot of the scene and zero overhead.

Original languageEnglish (US)
Title of host publication2020 IEEE 91st Vehicular Technology Conference, VTC Spring 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728152073
StatePublished - May 2020
Event91st IEEE Vehicular Technology Conference, VTC Spring 2020 - Antwerp, Belgium
Duration: May 25 2020May 28 2020

Publication series

NameIEEE Vehicular Technology Conference
ISSN (Print)1550-2252


Conference91st IEEE Vehicular Technology Conference, VTC Spring 2020

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Prediction'. Together they form a unique fingerprint.

Cite this