Characterizing the high-level content of natural images using lexical basis functions

John A. Black, Kanav Kahol, Prem Kuchi, Gamal Fahmy, Sethuraman Panchanathan

Research output: Contribution to journalConference articlepeer-review

9 Scopus citations


The performance of content-based image retrieval using low-level visual content has largely been judged to be unsatisfactory. Perceived performance could probably be improved if retrieval were based on higher-level content. However, researchers have not been very successful in bridging what is now called the "semantic gap" between low-level content detectors and higher-level visual content. This paper describes a novel "top-down" approach to bridging this semantic gap. A list of primitive words (which we call "lexical basis functions") are selected from a lexicon of the English language, and are used to characterize the higher-level content of natural outdoor images. Visual similarity between pairs of images are then "computed" based on the degree of similarity between their respective word lists. These "computed" similarities are then shown to correlate with subjectively perceived similarities between pairs of images. This demonstrates that the chosen set of lexical basis functions is able to characterize the multidimensional content (and similarity) of these image pairs in a manner that parallels their subjectively perceived content (and similarity). If a retrieval system could be designed to automatically detect the visual content represented by these basis functions, it could compute a similarity measure that would correlate with human subjective similarity rankings.

Original languageEnglish (US)
Pages (from-to)378-391
Number of pages14
JournalProceedings of SPIE - The International Society for Optical Engineering
StatePublished - 2003
EventHuman Vision and Electronic Imaging VIII - Santa Clara, CA, United States
Duration: Jan 21 2003Jan 24 2003


  • Content-based retrieval
  • Image characterization
  • Image indexing
  • Lexical basis functions
  • Visual concepts
  • Visual content
  • Visual percepts

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Characterizing the high-level content of natural images using lexical basis functions'. Together they form a unique fingerprint.

Cite this