Can the high-level content of natural images be indexed using local analysis?

John A. Black, Mariano Phielipp, Greg Nielson, Sethuraman Panchanathan

Research output: Contribution to journalConference articlepeer-review


Early methods of image indexing relied heavily on color histograms, which characterize the global content of images. However, global indexing methods proved to be unsatisfactory, and researchers now employ more localized measures of image content, based on relatively small regions. At the same time, it has also become clear that image indexing should be based on higher-level visual content. This raises an important question: "Can the higher-level content of images be reliably indexed using local analysis?" In general, humans are better at indexing mid-level and high-level visual content than today's automated indexing algorithms. Therefore, it makes sense to ascertain how well humans can perform midlevel or high-level indexing, based on small regions. This paper describes research that employs a set of outdoor scenery images (called the NaturePix image set) to compare how successfully humans can label the visual content of small regions of natural images when (1) these regions are seen in the context of the larger image, and (2) when these regions are extracted from (and are seen in isolation from) that larger image. The results of these experiments indicate what types of higher-level image content can be recognized locally, and how successfully high-level image content can be indexed on the basis of local feature analysis.

Original languageEnglish (US)
Pages (from-to)414-425
Number of pages12
JournalProceedings of SPIE - The International Society for Optical Engineering
StatePublished - 2004
EventHuman Vision and Electronic Imaging IX - San Jose, CA, United States
Duration: Jan 19 2004Jan 21 2004


  • Content based image retrieval
  • Feature detectors
  • Image content
  • Image indexing
  • Lexical basis functions
  • Local content analysis
  • NaturePix
  • Semantic content
  • Semantic indexing
  • Visual content

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Can the high-level content of natural images be indexed using local analysis?'. Together they form a unique fingerprint.

Cite this