Computer Vision Aided mmWave Beam Alignment in V2X Communications

Weihua Xu, Feifei Gao, Xiaoming Tao, Jianhua Zhang, Ahmed Alkhateeb

Research output: Contribution to journalArticlepeer-review

19 Scopus citations


Visual information, captured for example by cameras, can effectively reflect the sizes and locations of the environmental scattering objects, and thereby can be used to infer communications parameters like propagation directions, receiver powers, as well as the blockage status. In this paper, we propose a novel beam alignment framework that leverages images taken by cameras installed at the mobile user. Specifically, we utilize 3D object detection techniques to extract the size and location information of the dynamic vehicles around the mobile user, and design a deep neural network (DNN) to infer the optimal beam pair for transceivers without any pilot signal overhead. Moreover, to avoid performing beam alignment too frequently or too slowly, a beam coherence time (BCT) prediction method is developed based on the vision information. This can effectively improve the transmission rate compared with the beam alignment approach with the fixed BCT. Simulation results show that the proposed vision based beam alignment methods outperform the existing LIDAR and vision based solutions, and demand for much lower hardware cost and communication overhead.

Original languageEnglish (US)
Pages (from-to)2699-2714
Number of pages16
JournalIEEE Transactions on Wireless Communications
Issue number4
StatePublished - Apr 1 2023


  • Deep learning
  • V2X communication
  • beam alignment
  • beam coherence time
  • computer vision

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Computer Vision Aided mmWave Beam Alignment in V2X Communications'. Together they form a unique fingerprint.

Cite this