Multi-class segmentation with relative location prior

Stephen Gould*, Jim Rodgers, David Cohen, Gal Elidan, Daphne Koller

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

340 Scopus citations


Multi-class image segmentation has made significant advances in recent years through the combination of local and global features. One important type of global feature is that of inter-class spatial relationships. For example, identifying "tree" pixels indicates that pixels above and to the sides are more likely to be "sky" whereas pixels below are more likely to be "grass." Incorporating such global information across the entire image and between all classes is a computational challenge as it is image-dependent, and hence, cannot be precomputed. In this work we propose a method for capturing global information from inter-class spatial relationships and encoding it as a local feature. We employ a two-stage classification process to label all image pixels. First, we generate predictions which are used to compute a local relative location feature from learned relative location maps. In the second stage, we combine this with appearance-based features to provide a final segmentation. We compare our results to recent published results on several multi-class image segmentation databases and show that the incorporation of relative location information allows us to significantly outperform the current state-of-the-art.

Original languageAmerican English
Pages (from-to)300-316
Number of pages17
JournalInternational Journal of Computer Vision
Issue number3
StatePublished - Dec 2008
Externally publishedYes

Bibliographical note

Funding Information:
Acknowledgements The authors would like to thank Geremy Heitz for his helpful suggestions, and David Vickrey and Haidong Wang for discussions on efficient CRF inference and learning. This research was supported by the Defense Advanced Research Projects Agency (DARPA) under contract number SA4996-10929-3 and the Department of Navy MURI under contract number N00014-07-1-0747.


  • Multi-class image segmentation
  • Relative location
  • Segmentation


Dive into the research topics of 'Multi-class segmentation with relative location prior'. Together they form a unique fingerprint.

Cite this