Towards semantic-based retrieval of visual information: A model-based approach

Youngchoon Park, Forouzan Golshani, Sethuraman Panchanathan

Research output: Contribution to journalConference articlepeer-review


This paper centers around the problem of automated visual content classification. To enable classification based image or visual object retrieval, we propose a new image representation scheme called visual context descriptor (VCD) that is a multidimensional vector in which each element represents the frequency of a unique visual property of an image or a region. VCD utilizes the predetermined quality dimensions (i.e., types of features and quantization level) and semantic model templates mined in priori. Not only observed visual cues, but also contextually relevant visual features are proportionally incorporated in VCD. Contextual relevance of a visual cue to a semantic class is determined by using correlation analysis of ground troth samples. Such co-occurrence analysis of visual cues requires transformation of a real-valued visual feature vector (e.g., color histogram, Gabor texture, etc.,) into a discrete event (e.g., terms in text). Good-feature to track, rule of thirds, iterative k-means clustering and TSVQ are involved in transformation of feature vectors into unified symbolic representations called visual terms. Similarity-based visual cue frequency estimation is also proposed and used for ensuring the correctness of model learning and matching since sparseness of sample data causes the unstable results of frequency estimation of visual cues. The proposed method naturally allows integration of heterogeneous visual or temporal or spatial cues in a single classification or matching framework, and can be easily integrated into a semantic knowledge base such as thesaurus, and ontology. Robust semantic visual model template creation and object based image retrieval are demonstrated based on the proposed content description scheme.

Original languageEnglish (US)
Pages (from-to)50-61
Number of pages12
JournalProceedings of SPIE - The International Society for Optical Engineering
StatePublished - Jan 1 2002
EventInternet Multimedia Management Systems III - Boston, MA, United States
Duration: Jul 31 2002Aug 1 2002


  • Contextual image matching
  • Ontology-based image retrieval
  • Semantic image classification
  • Visual cue co-occurrence

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Towards semantic-based retrieval of visual information: A model-based approach'. Together they form a unique fingerprint.

Cite this