Channel Coding for Joint Colour and Depth Segmentation

  • Marcus Wallenberg
  • Michael Felsberg
  • Per-Erik Forssén
  • Babette Dellen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6835)


Segmentation is an important preprocessing step in many applications. Compared to colour segmentation, fusion of colour and depth greatly improves the segmentation result. Such a fusion is easy to do by stacking measurements in different value dimensions, but there are better ways. In this paper we perform fusion using the channel representation, and demonstrate how a state-of-the-art segmentation algorithm can be modified to use channel values as inputs. We evaluate segmentation results on data collected using the Microsoft Kinect peripheral for Xbox 360, using the superparamagnetic clustering algorithm. Our experiments show that depth gradients are more useful than depth values for segmentation, and that channel coding both colour and depth gradients makes tuned parameter settings generalise better to novel images.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Björkman, M., Eklundh, J.-O.: Vision in the real world: Finding, attending and recognizing objects. Int. J. of Imag. Sys. and Technology 5(16), 189–209 (2006)CrossRefGoogle Scholar
  2. 2.
    Blatt, M., Wiseman, S., Domany, E.: Superparametric clustering of data. Physical Review Letters 76(18) (1996)Google Scholar
  3. 3.
    Bleiweiss, A., Werman, M.: Fusing time-of-flight depth and color for real-time segmentation and tracking. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 58–69. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Boulanger, P.: Simultaneous segmentation of range and color images based on bayesian decision theory. In: Computer and Robot Vision, CRV 2004 (2004)Google Scholar
  5. 5.
    Dellen, B., Alenya, G., Torras, C.: Segmenting color images into surface patches by exploiting sparse depth data. In: WACV, pp. 591–598 (2011)Google Scholar
  6. 6.
    Walter, A., et al.: Dynamics of seedling growth acclimation towards altered light conditions can be quantified via GROWSCREEN. New Phytol. 174, 447–455 (2007)CrossRefGoogle Scholar
  7. 7.
    Leens, J., Piérard, S., Barnich, O., Van Droogenbroeck, M., Wagner, J.-M.: Combining color, depth, and motion for video segmentation. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 104–113. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Felsberg, M.: Incremental computation of feature hierarchies. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 523–532. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Felsberg, M., Forssén, P.-E., Scharr, H.: Channel smoothing: Efficient robust smoothing of low-level signal features. IEEE TPAMI 28(2), 209–222 (2006)CrossRefGoogle Scholar
  10. 10.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE TPAMI 6, 721–741 (1984)MATHCrossRefGoogle Scholar
  11. 11.
    Granlund, G.H.: Does vision inevitably have to be active? In: SCIA 1999 (1999)Google Scholar
  12. 12.
    Granlund, G.H.: An associative perception-action structure using a localized space variant information representation. In: Sommer, G., Zeevi, Y.Y. (eds.) AFPAC 2000. LNCS, vol. 1888. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Kass, M., Solomon, J.: Smoothed local histogram filters. In: ACM SIGGRAPH 2010 papers, pages 100:1–100:10. ACM, New York (2010)CrossRefGoogle Scholar
  14. 14.
    Lakoff, G.: Women, Fire, and Dangerous Things - what categories reveal about the mind. University of Chicago Press, Chicago (1987)Google Scholar
  15. 15.
    Leibe, B., Schiele, B.: Interleaved object categorization and segmentation. In: British Machine Vision Conference, pp. 759–768 (2003)Google Scholar
  16. 16.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using brightness and texture. In: Advances in Neural Information Processing Systems (NIPS 2002). MIT Press, Cambridge (2002)Google Scholar
  17. 17.
    Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.B.: Image-based plant modeling. In: ACM SIGGRAPH 2006, New York, NY, USA, pp. 599–604 (2006)Google Scholar
  18. 18.
    Shpunt, A., Pesach, B.: Optical pattern projection. US Patent Application Publication, US 2010/0284082 A1 (November 2010)Google Scholar
  19. 19.
    Swendsen, R.H., Wang, S.: Nonuniversal critical dynamics in monte carlo simulations. Physical Review Letters 76(18), 86–88 (1987)CrossRefGoogle Scholar
  20. 20.
    Taguchi, Y., et al.: Stereo reconstruction with mixed pixels using adaptive over-segmentation. In: CVPR (2008)Google Scholar
  21. 21.
    Zemel, R.S., Dayan, P., Pouget, A.: Probabilistic interpretation of population codes. Neural Computation 10(2), 403–430 (1998)CrossRefGoogle Scholar
  22. 22.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1330–1334 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Marcus Wallenberg
    • 1
  • Michael Felsberg
    • 1
  • Per-Erik Forssén
    • 1
  • Babette Dellen
    • 2
  1. 1.Linköping UniversityLinköpingSweden
  2. 2.Institut de Robotica i Informatica Industrial (CSIC-UPC)BarcelonaSpain

Personalised recommendations