N-tuple Color Segmentation for Multi-view Silhouette Extraction

  • Abdelaziz Djelouah
  • Jean-Sébastien Franco
  • Edmond Boyer
  • François Le Clerc
  • Patrick Pérez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)


We present a new method to extract multiple segmentations of an object viewed by multiple cameras, given only the camera calibration. We introduce the n-tuple color model to express inter-view consistency when inferring in each view the foreground and background color models permitting the final segmentation. A color n-tuple is a set of pixel colors associated to the n projections of a 3D point. The first goal is set as finding the MAP estimate of background/foreground color models based on an arbitrary sample set of such n-tuples, such that samples are consistently classified, in a soft way, as ”empty” if they project in the background of at least one view, or ”occupied” if they project to foreground pixels in all views. An Expectation Maximization framework is then used to alternate between color models and soft classifications. In a final step, all views are segmented based on their attached color models. The approach is significantly simpler and faster than previous multi-view segmentation methods, while providing results of equivalent or better quality.


Color Model Foreground Object Color Distribution Background Pixel Foreground Pixel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Rother, C., Kolmogorov, V., Blake, A.: “grabcut”: interactive foreground extraction using iterated graph cuts. In: ACM SIGGRAPH (2004)Google Scholar
  2. 2.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: ICCV (1999)Google Scholar
  3. 3.
    Zeng, G., Quan, L.: Silhouette extraction from multiple images of an unknown background. In: ACCV (2004)Google Scholar
  4. 4.
    Lee, W., Woo, W., Boyer, E.: Silhouette Segmentation in Multiple Views. IEEE PAMI (2010)Google Scholar
  5. 5.
    Kolev, K., Brox, T., Cremers, D.: Fast joint estimation of silhouettes and dense 3d geometry from multiple images. IEEE PAMI (2011)Google Scholar
  6. 6.
    Campbell, N.D.F., Vogiatzis, G., Hernández, C., Cipolla, R.: Automatic 3d object segmentation in multiple views using volumetric graph-cuts. Image Vision Comput. (2010)Google Scholar
  7. 7.
    Feldmann, T., Dießelberg, L., Wörner, A.: Adaptive Foreground/Background Segmentation Using Multiview Silhouette Fusion. In: Denzler, J., Notni, G., Süße, H. (eds.) Pattern Recognition. LNCS, vol. 5748, pp. 522–531. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Gallego, J., Salvador, J., Casas, J., Pards, M.: Joint multi-view foreground segmentation and 3d reconstruction with tolerance loop. In: IEEE ICIP (2011)Google Scholar
  9. 9.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. IEEE PAMI (1997)Google Scholar
  10. 10.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: CVPR (1999)Google Scholar
  11. 11.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV (2001)Google Scholar
  12. 12.
    Snow, D., Viola, P., Zabih, R.: Exact voxel occupancy with graph cuts. In: CVPR (2000)Google Scholar
  13. 13.
    Franco, J.S., Boyer, E.: Fusion of Multi-View Silhouette Cues Using a Space Occupancy Grid. In: ICCV (2005)Google Scholar
  14. 14.
    Sarim, M., Hilton, A., Guillemaut, J.Y., Kim, H., Takai, T.: Wide-baseline multi-view video segmentation for 3d reconstruction. In: Proceedings of the 1st International Workshop on 3D Video Processing, 3DVP 2010 (2010)Google Scholar
  15. 15.
    Pham, V.Q., Takahashi, K., Naemura, T.: Foreground-background segmentation using iterated distribution matching. In: CVPR (2011)Google Scholar
  16. 16.
    Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Abdelaziz Djelouah
    • 1
    • 2
  • Jean-Sébastien Franco
    • 2
  • Edmond Boyer
    • 2
  • François Le Clerc
    • 1
  • Patrick Pérez
    • 1
  1. 1.TechnicolorCesson SevigneFrance
  2. 2.LJK - INRIA Rhône-AlpesFrance

Personalised recommendations