Superpixel Correspondence for Non-parametric Scene Parsing of Natural Images

  • Veronica Naosekpam
  • Alexy BhowmickEmail author
  • Shyamanta M. Hazarika
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11941)


Scene parsing refers to the task of labeling every pixel in an image with the class label it belongs to. In this paper, we propose a novel scalable non-parametric scene parsing system based on superpixels correspondence. The non-parametric approach requires almost no training and can scale up to datasets with thousands of labels. This involves retrieving a set of images similar to the query image, followed by superpixel matching of the query image with the retrieval set. Finally, our system warps the annotation results of superpixel matching, and integrates multiple cues in a Markov Random Field (MRF) to obtain an accurate segmentation of the query image. Our non-parametric scene parsing achieves promising results on the LabelMe Outdoor dataset. The system has limited parameters, and captures contextual information naturally in the retrieval and alignment procedure.


Scene Scene parsing Non-parametric Label transfer Partial correspondence 


  1. 1.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 377–384. IEEE (1999)Google Scholar
  2. 2.
    Eigen, D., Fergus, R.: Nonparametric image parsing using adaptive neighbor sets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2799–2806. IEEE (2012)Google Scholar
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  4. 4.
    George, M.: Image parsing with a wide range of classes and scene-level context. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3622–3630 (2015)Google Scholar
  5. 5.
    Gould, S., Zhang, Y.: PatchMatchGraph: building a graph of dense patch correspondences for label transfer. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 439–452. Springer, Heidelberg (2012). Scholar
  6. 6.
    Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: CVPR 2010–23rd IEEE Conference on Computer Vision and Pattern Recognition, pp. 3304–3311. IEEE Computer Society (2010)Google Scholar
  7. 7.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2169–2178. IEEE (2006)Google Scholar
  8. 8.
    Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2368–2382 (2011)CrossRefGoogle Scholar
  9. 9.
    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: SIFT flow: dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008). Scholar
  10. 10.
    Malisiewicz, T., Gupta, A., Efros, A.A., et al.: Ensemble of exemplar-svms for object detection and beyond. In: ICCV, vol. 1, p. 6. Citeseer (2011)Google Scholar
  11. 11.
    Najafi, M., Taghavi Namin, S., Salzmann, M., Petersson, L.: Sample and filter: nonparametric scene parsing via efficient filtering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 607–615 (2016)Google Scholar
  12. 12.
    Razzaghi, P., Samavi, S.: A new fast approach to nonparametric scene parsing. Pattern Recogn. Lett. 42, 56–64 (2014)CrossRefGoogle Scholar
  13. 13.
    Tighe, J., Lazebnik, S.: SuperParsing: scalable nonparametric image parsing with superpixels. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 352–365. Springer, Heidelberg (2010). Scholar
  14. 14.
    Tighe, J., Lazebnik, S.: Finding things: Image parsing with regions and per-exemplar detectors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3001–3008 (2013)Google Scholar
  15. 15.
    Tung, F., Little, J.J.: CollageParsing: nonparametric scene parsing by adaptive overlapping windows. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 511–525. Springer, Cham (2014). Scholar
  16. 16.
    Yang, J., Price, B., Cohen, S., Yang, M.H.: Context driven scene parsing with attention to rare classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3294–3301 (2014)Google Scholar
  17. 17.
    Zhang, H., Fang, T., Chen, X., Zhao, Q., Quan, L.: Partial similarity based nonparametric scene parsing in certain environment. In: CVPR 2011, pp. 2241–2248. IEEE (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Veronica Naosekpam
    • 1
  • Alexy Bhowmick
    • 1
    Email author
  • Shyamanta M. Hazarika
    • 2
  1. 1.School of TechnologyAssam Don Bosco UniversityGuwahatiIndia
  2. 2.Indian Institute of TechnologyGuwahatiIndia

Personalised recommendations