A hierarchical word-merging algorithm with class separability measure

Lei Wang, Luping Zhou, Chunhua Shen, Lingqiao Liu, Huan Liu

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


In image recognition with the bag-of-features model, a small-sized visual codebook is usually preferred to obtain a low-dimensional histogram representation and high computational efficiency. Such a visual codebook has to be discriminative enough to achieve excellent recognition performance. To create a compact and discriminative codebook, in this paper we propose to merge the visual words in a large-sized initial codebook by maximally preserving class separability. We first show that this results in a difficult optimization problem. To deal with this situation, we devise a suboptimal but very efficient hierarchical word-merging algorithm, which optimally merges two words at each level of the hierarchy. By exploiting the characteristics of the class separability measure and designing a novel indexing structure, the proposed algorithm can hierarchically merge 10,000 visual words down to two words in merely 90 seconds. Also, to show the properties of the proposed algorithm and reveal its advantages, we conduct detailed theoretical analysis to compare it with another hierarchical word-merging algorithm that maximally preserves mutual information, obtaining interesting findings. Experimental studies are conducted to verify the effectiveness of the proposed algorithm on multiple benchmark data sets. As shown, it can efficiently produce more compact and discriminative codebooks than the state-of-the-art hierarchical word-merging algorithms, especially when the size of the codebook is significantly reduced.

Original languageEnglish (US)
Article number6731380
Pages (from-to)417-435
Number of pages19
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number3
StatePublished - Mar 2014


  • Hierarchical word merge
  • bag-of-features model
  • class separability
  • compact codebook
  • object recognition

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'A hierarchical word-merging algorithm with class separability measure'. Together they form a unique fingerprint.

Cite this