Counting Apples and Oranges with Deep Learning: A Data-Driven Approach

Steven W. Chen, Shreyas S. Shivakumar, Sandeep Dcunha, Jnaneshwar Das, Edidiong Okon, Chao Qu, Camillo J. Taylor, Vijay Kumar

Research output: Contribution to journalArticlepeer-review

300 Scopus citations


This paper describes a fruit counting pipeline based on deep learning that accurately counts fruit in unstructured environments. Obtaining reliable fruit counts is challenging because of variations in appearance due to illumination changes and occlusions from foliage and neighboring fruits. We propose a novel approach that uses deep learning to map from input images to total fruit counts. The pipeline utilizes a custom crowdsourcing platform to quickly label large data sets. A blob detector based on a fully convolutional network extracts candidate regions in the images. A counting algorithm based on a second convolutional network then estimates the number of fruits in each region. Finally, a linear regression model maps that fruit count estimate to a final fruit count. We analyze the performance of the pipeline on two distinct data sets of oranges in daylight, and green apples at night, utilizing human generated labels as ground truth. We also show that the pipeline has a short training time and performs well with a limited data set size. Our method generalizes across both data sets and is able to perform well even on highly occluded fruits that are challenging for human labelers to annotate.

Original languageEnglish (US)
Article number7814145
Pages (from-to)781-788
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number2
StatePublished - Apr 2017
Externally publishedYes


  • Agricultural automation
  • Visual learning
  • categorization
  • object detection
  • segmentation

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Counting Apples and Oranges with Deep Learning: A Data-Driven Approach'. Together they form a unique fingerprint.

Cite this