An Improved Occlusion Detection with Constraints Approach for Video Processing

  • Tuan-Anh Vu
  • Hung Ngoc Phan
  • Tu Kha Huynh
  • Synh Viet-Uyen HaEmail author
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 257)


The accurate understanding of occlusion region is critical for trustworthy estimation of optical flow to prevent the negative influence of occluded pixels on disocclusion regions. However, occlusion is the result of motion. In contrast, estimating accurate optical flow is necessary to locate reliable occlusions. Hence, one of the key challenges that required further exploration and research is the accuracy at the boundaries of the moving objects. This paper presents the work in process approach that can detect occlusion regions by using some constraints such as pixel-wise coherence, segment-wise confidence and edge-motion coherence. Comparing to the previous methods, our method achieves the same efficiency by solving only one Partial Differential Equation (PDE) problem. The proposed method is faster and provides better coverage rates for occlusion regions than variation techniques in various numbers of benchmark datasets. With these improved results, we can apply and extend our approach to a wider range of applications in computer vision, such as: motion estimation, object detection and tracking, robot navigation, 3D reconstruction, image registration.


Optical flow Unstable region Object boundaries Occlusion detection Video object extraction Video object segmentation 


  1. 1.
    Perez-Rua, J.-M., Crivelli, T., Bouthemy, P., Perez, P.: Determining occlusions from space and time image reconstructions. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1382–1391. IEEE (2016)Google Scholar
  2. 2.
    Alvarez, L., Deriche, R., Papadopoulo, T., Sánchez, J.: Symmetrical dense optical flow estimation with occlusions detection. Int. J. Comput. Vis. 75, 371–385 (2007)CrossRefGoogle Scholar
  3. 3.
    Strecha, C., Fransens, R., Van Gool, L.: A probabilistic approach to large displacement optical flow and occlusion detection. In: Comaniciu, D., Mester, R., Kanatani, K., Suter, D. (eds.) SMVP 2004. LNCS, vol. 3247, pp. 71–82. Springer, Heidelberg (2004). Scholar
  4. 4.
    Proesmans, M., Van Gool, L., Pauwels, E., Oosterlinck, A.: Determination of optical flow and its discontinuities using non-linear diffusion. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 294–304. Springer, Heidelberg (1994). Scholar
  5. 5.
    Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M.: Bilateral filtering-based optical flow estimation with occlusion detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 211–224. Springer, Heidelberg (2006). Scholar
  6. 6.
    Ayvaci, A., Raptis, M., Soatto, S.: Sparse occlusion detection with optical flow. Int. J. Comput. Vis. 97, 322–338 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Sadek, R., Ballester, C., Garrido, L., Meinhardt, E., Caselles, V.: Frame interpolation with occlusion detection using a time coherent segmentation. In: Proceedings of the International Conference on Computer Vision Theory and Applications, pp. 367–372. SciTePress - Science and and Technology Publications (2012)Google Scholar
  8. 8.
    Ballester, C., Garrido, L., Lazcano, V., Caselles, V.: A TV-L1 optical flow method with occlusion detection. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 31–40. Springer, Heidelberg (2012). Scholar
  9. 9.
    Tao, M., Bai, J., Kohli, P., Paris, S.: SimpleFlow: a non-iterative, sublinear optical flow algorithm. Comput. Graph. Forum. 31, 345–353 (2012)CrossRefGoogle Scholar
  10. 10.
    Lim, K.P., Das, A., Chong, M.N.: Estimation of occlusion and dense motion fields in a bidirectional Bayesian framework. IEEE Trans. Pattern Anal. Mach. Intell. 24, 712–718 (2002)CrossRefGoogle Scholar
  11. 11.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, pp. 508–515. IEEE Computer Society (2001)Google Scholar
  12. 12.
    Sun, J., Li, Y., Kang, S.B., Shum, H.-Y.: Symmetric stereo matching for occlusion handling. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), pp. 399–406. IEEE (2005)Google Scholar
  13. 13.
    Ince, S., Konrad, J.: Occlusion-aware optical flow estimation. IEEE Trans. Image Process. 17, 1443–51 (2008)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ben-Ari, R., Sochen, N.: Variational stereo vision with sharp discontinuities and occlusion handling. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–7. IEEE (2007)Google Scholar
  15. 15.
    Xu, L., Chen, J., Jia, J.: A segmentation based variational model for accurate optical flow estimation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 671–684. Springer, Heidelberg (2008). Scholar
  16. 16.
    Stein, A.N., Hebert, M.: Occlusion boundaries from motion: low-level detection and mid-level reasoning. Int. J. Comput. Vis. 82, 325–357 (2009)CrossRefGoogle Scholar
  17. 17.
    He, X., Yuille, A.: Occlusion boundary detection using pseudo-depth. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 539–552. Springer, Heidelberg (2010). Scholar
  18. 18.
    Gao, T., Packer, B., Koller, D.: A segmentation-aware object detection model with occlusion handling. In: CVPR 2011, pp. 1361–1368. IEEE (2011)Google Scholar
  19. 19.
    Humayun, A., Mac Aodha, O., Brostow, G.J.: Learning to find occlusion regions. In: CVPR 2011, pp. 2161–2168. IEEE (2011)Google Scholar
  20. 20.
    Sundberg, P., Brox, T., Maire, M., Arbelaez, P., Malik, J.: Occlusion boundary detection and figure/ground assignment from optical flow. In: CVPR 2011, pp. 2233–2240. IEEE, Washington, DC, USA (2011)Google Scholar
  21. 21.
    Shi, J., Tomasi, C.: Good features to track. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR 1994, pp. 593–600. IEEE Computer Society Press (1994)Google Scholar
  22. 22.
    Estellers, V., Soatto, S.: Detecting occlusions as an inverse problem. J. Math. Imaging Vis. 54, 181–198 (2016)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). Scholar
  24. 24.
    Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009)CrossRefGoogle Scholar
  25. 25.
    Fawcett, T.: An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861–874 (2006)CrossRefGoogle Scholar
  26. 26.
    Powers, D.M.W.: Evaluation: from precision, recall and F-measure To Roc, informedness, markedness & correlation. J. Mach. Learn. Technol. 2, 37–63 (2011)Google Scholar
  27. 27.
    Pont-Tuset, J.: Image segmentation evaluation and its application to object detection, Universitat Politècnica de Catalunya, UPC BarcelonaTech (2014)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Tuan-Anh Vu
    • 1
  • Hung Ngoc Phan
    • 1
  • Tu Kha Huynh
    • 1
  • Synh Viet-Uyen Ha
    • 1
    Email author
  1. 1.School of Computer Science and EngineeringInternational University, Vietnam National UniversityHo Chi Minh cityVietnam

Personalised recommendations