Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

Gouranga Charan, Muhammad Alrabeiah, Tawfik Osman, Ahmed Alkhateeb

Research output: Contribution to journalArticlepeer-review

Abstract

Leveraging sensory information to aid the millimeter-wave (mmWave) and sub-terahertz (sub-THz) beam selection process is attracting increasing interest. This sensory data, captured for example by cameras at the basestations, has the potential of significantly reducing the beam sweeping overhead and enabling highly-mobile applications. The solutions developed so far, however, have mainly considered single-candidate scenarios, i.e., scenarios with a single candidate user in the visual scene, and were evaluated using synthetic datasets. To address these limitations, this paper extensively investigates the sensing-aided beam prediction problem in a real-world multi-object vehicle-to-infrastructure (V2I) scenario and presents a comprehensive machine learning based framework. In particular, this paper proposes to utilize visual and positional data to predict the optimal beam indices as an alternative to the conventional beam sweeping approaches. For this, a novel user (transmitter) identification solution has been developed, a key step in realizing sensing-aided multi-candidate and multi-user beam prediction solutions. The proposed solutions are evaluated on the large-scale real-world DeepSense 6G dataset. Experimental results in realistic V2I communication scenarios indicate that the proposed solutions achieve between 67-84% top-1 and close to 100% top-5 beam prediction accuracy for the scenarios with single-user, and between 65-80% top-1 and close to 95% top-5 beam prediction accuracy for multi-candidate scenarios. Furthermore, the proposed approach can identify the probable transmitting candidate with more than 93% accuracy across the different scenarios. This highlights a promising approach for significantly reducing the beam training overhead in mmWave/THz communication systems.

Original languageEnglish (US)
JournalIEEE Transactions on Vehicular Technology
DOIs
StateAccepted/In press - 2024

Keywords

  • beam prediction
  • computer vision
  • Deep learning
  • mmWave communication
  • multi-user

ASJC Scopus subject areas

  • Automotive Engineering
  • Aerospace Engineering
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios'. Together they form a unique fingerprint.

Cite this