Multi-Class Segmentation with Relative Location Prior
- 1.2k Downloads
Multi-class image segmentation has made significant advances in recent years through the combination of local and global features. One important type of global feature is that of inter-class spatial relationships. For example, identifying “tree” pixels indicates that pixels above and to the sides are more likely to be “sky” whereas pixels below are more likely to be “grass.” Incorporating such global information across the entire image and between all classes is a computational challenge as it is image-dependent, and hence, cannot be precomputed.
In this work we propose a method for capturing global information from inter-class spatial relationships and encoding it as a local feature. We employ a two-stage classification process to label all image pixels. First, we generate predictions which are used to compute a local relative location feature from learned relative location maps. In the second stage, we combine this with appearance-based features to provide a final segmentation. We compare our results to recent published results on several multi-class image segmentation databases and show that the incorporation of relative location information allows us to significantly outperform the current state-of-the-art.
KeywordsMulti-class image segmentation Segmentation Relative location
Unable to display preview. Download preview PDF.
- Carbonetto, P., de Freitas, N., & Barnard, K. (2004). A statistical model for general contextual object recognition. In ECCV. Google Scholar
- Criminisi, A. (2004). Microsoft research Cambridge object recognition image database (version 1.0 and 2.0). http://research.microsoft.com/vision/cambridge/recognition.
- Fink, M., & Perona, P. (2003). Mutual boosting for contextual inference. In NIPS. Google Scholar
- Greig, D. M., Porteous, B. T., & Seheult, A. H. (1989). Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society. Series B (Methodological), 51(2), 271–279. Google Scholar
- He, X., Zemel, R., & Carreira-Perpinan, M. (2004). Multiscale conditional random fields for image labelling. In CVPR. Google Scholar
- He, X., Zemel, R. S., & Ray, D. (2006). Learning and incorporating top-down cues in image segmentation. Berlin: Springer. Google Scholar
- Kumar, M. P., Torr, P. H. S., & Zisserman, A. (2005). OBJ CUT. In CVPR. Google Scholar
- Kumar, S., & Hebert, M. (2005). A hierarchical field framework for unified context-based classification. In ICCV. Google Scholar
- Lafferty, J. D., McCallum, A., & Pereira, F. C. N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML. Google Scholar
- Minka, T. P. (2003). A comparison of numerical optimizers for logistic regression (Technical Report 758). Carnegie Mellon University, Department of Statistics. Google Scholar
- Mori, G., Ren, X., Efros, A. A., & Malik, J. (2004). Recovering human body configurations: combining segmentation and recognition. In CVPR. Google Scholar
- Murphy, K., Torralba, A., & Freeman, W. (2003). Using the forest to see the tree: a graphical model relating features, objects and the scenes. In NIPS. Google Scholar
- Opelt, A., Pinz, A., & Zisserman, A. (2006). Incremental learning of object detectors using a visual shape alphabet. In CVPR. Google Scholar
- Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Mateo: Morgan Kaufmann. Google Scholar
- Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., & Belongie, S. (2007). Objects in context. In ICCV. Google Scholar
- Ren, X., & Malik, J. (2003). Learning a classification model for segmentation. In ICCV. Google Scholar
- Schroff, F., Criminisi, A., & Zisserman, A. (2006). Single-histogram class models for image segmentation. In ICVGIP. Google Scholar
- Shental, N., Zomet, A., Hertz, T., & Weiss, Y. (2003). Learning and inferring image segmentations using the gbp typical cut. In ICCV. Google Scholar
- Shotton, J., Winn, J., Rother, C., & Criminisi, A. (2006). TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV’06. Google Scholar
- Singhal, A., Luo, J., & Zhu, W. (2003). Probabilistic spatial context models for scene content understanding. In CVPR. Google Scholar
- Sutton, C., & McCallum, A. (2005). Piecewise training of undirected models. In UAI. Google Scholar
- Torralba, A. B., Murphy, K. P., & Freeman, W. T. (2004). Contextual models for object detection using boosted random fields. In NIPS. Google Scholar
- Winn, J., Criminisi, A., & Minka, T. (2005). Object categorization by learned universal visual dictionary. In ICCV. Google Scholar
- Winn, J., & Shotton, J. (2006). The layout consistent random field for recognizing and segmenting partially occluded objects. In CVPR. Google Scholar
- Yang, L., Meer, P., & Foran, D. J. (2007). Multiple class segmentation using a unified framework over mean-shift patches. In CVPR. Google Scholar