Binarized depthwise separable neural network for object tracking in FPGA

Li Yang, Zhezhi He, Deliang Fan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations


Object tracking has achieved great advances in the past few years and has been widely applied in vision-based application. Nowadays, deep convolutional neural network has taken an important role in object tracking tasks. However, its enormous model size and massive computation cost have became the main obstacle for deployment of such powerful algorithm in low power and resource limited embedded system, such as FPGA. Due to the popularization of the power-sensitive mobile platform, low power real-time tracking solution is strongly required. In order to address these challenges, we propose a low power and energy-efficient object tracking FPGA implementation based on a newly proposed binarized depthwise separable deep convolutional neural network. It can significantly reduce the model size and computation complexity simultaneously utilizing binarized (i.e., +1 and-1) depthwise separable convolution kernel and our proposed trainable threshold group binarization activation function. It can completely converts the dot product and accumulation based convolution operations into bit-wise XNOR and bit-count operations, while achieving state-of-the-art accuracy. Our proposed binarized depthwise separable model achieves ∼57% Intersection over Union (IOU) on DJI object tracking dataset with only ∼143.9Kb model parameter size. We then deploy our proposed model into the Xilinx PYNQ Z1 board with only 4.9Mb on-chip RAM. The experiment results show that our FPGA implementation achieves 11.1 frames per second for object tracking with only 2.61W.

Original languageEnglish (US)
Title of host publicationGLSVLSI 2019 - Proceedings of the 2019 Great Lakes Symposium on VLSI
PublisherAssociation for Computing Machinery
Number of pages4
ISBN (Electronic)9781450362528
StatePublished - May 13 2019
Externally publishedYes
Event29th Great Lakes Symposium on VLSI, GLSVLSI 2019 - Tysons Corner, United States
Duration: May 9 2019May 11 2019

Publication series

NameProceedings of the ACM Great Lakes Symposium on VLSI, GLSVLSI


Conference29th Great Lakes Symposium on VLSI, GLSVLSI 2019
Country/TerritoryUnited States
CityTysons Corner


  • Binarized convolutional neural network (bnn)
  • Field-programmable gate array (FPGA)
  • Object tracking

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'Binarized depthwise separable neural network for object tracking in FPGA'. Together they form a unique fingerprint.

Cite this