Harmony Potentials for Joint Classification and Segmentation

Josep M. Gonfaus, Xavier Boix, Joost van de Weijer, Andrew D. Bagdanov, Joan Serrat, Jordi Gonzalez

Research output: Contribution to journalArticleResearchpeer-review

130 Citations (Scopus)


Hierarchical conditional random fields have been successfully applied to object segmentation. One reason is their ability to incorporate contextual information at different scales. However, these models do not allow multiple labels to be assigned to a single node. At higher scales in the image, this yields an oversimplified model, since multiple classes can be reasonable expected to appear within one region. This simplified model especially limits the impact that observations at larger scales may have on the CRF model. Neglecting the information at larger scales is undesirable since class-label estimates based on these scales are more reliable than at smaller, noisier scales.

To address this problem, we propose a new potential, called harmony potential, which can encode any possible combination of class labels. We propose an effective sampling strategy that renders tractable the underlying optimization problem. Results show that our approach obtains state-of-the-art results on two challenging datasets: Pascal VOC 2009 and MSRC-21.

Original languageEnglish
Pages (from-to)3280-3287
Number of pages8
JournalIEEE Conference on Computer Vision and Pattern Recognition
Volume(GGS Rating A++, Class 1)
Publication statusPublished - 2010


Dive into the research topics of 'Harmony Potentials for Joint Classification and Segmentation'. Together they form a unique fingerprint.

Cite this