Advertisement

Real-time shadow detection using multi-channel binarization and noise removal

  • Márcio C. F. MacedoEmail author
  • Verônica P. Nascimento
  • Antonio C. S. Souza
Original Research Paper
  • 173 Downloads

Abstract

High-quality automatic shadow detection remains a challenging problem in image processing and computer vision. Existing techniques for shadow detection typically make use of deep learning strategies to obtain accurate shadow detection results, at the cost of demanding high processing time, making their use unsuitable for augmented reality and robotic applications. In this paper, we propose a novel approach to perform high-quality shadow detection in real time. To do so, we convert an input image into different color spaces to perform multi-channel binarization and detect different shadow regions in the image. Then, a filtering algorithm is proposed to remove the noisy false-positive shadow regions on the basis of their sizes. Experimental results evaluated in two different datasets show that the proposed approach may run entirely on the GPU, requiring only \(\approx\) 13 ms to detect shadows in an image with \(3840 \times 2160\) (4k) resolution. That makes our approach about 1.8 (66\(\times\)) to 4.6 (37,284\(\times\)) orders of magnitude faster than related work for 4k resolution images, at the cost of only \(\approx\) 5% of accuracy loss compared to the best results achieved for each dataset.

Keywords

Shadow detection Parallel processing Binarization Noise removal Real time 

Notes

Acknowledgements

We are thankful to Guo et al. [3], Hosseinzadeh et al. [19] and Le et al. [20] for gently sharing the source code of their shadow detection algorithms. This research is supported by the scholarship program of Coordenação de Aperfeiçoamento de Pessoal do Nível Superior (CAPES). The hardware used for processing time evaluation was provided by NVIDIA Corporation, through the GPU Education Center.

Supplementary material

Supplementary material 1 (mp4 18897 KB)

References

  1. 1.
    Liu, Y., Granier, X.: Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE Trans. Vis. Comput. Graph. 18, 573–580 (2012).  https://doi.org/10.1109/TVCG.2012.53 CrossRefGoogle Scholar
  2. 2.
    Zhu, J., Samuel, K.G.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: Proceedings of the CVPR, pp. 223–230. IEEE, San Francisco (2010).  https://doi.org/10.1109/CVPR.2010.5540209
  3. 3.
    Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. In: Proceedings of the CVPR, pp. 2033–2040. IEEE, Colorado Springs (2011).  https://doi.org/10.1109/CVPR.2011.5995725
  4. 4.
    Vicente, T.F.Y., Hou, L., Yu, C.P., Hoai, M., Samaras, D.: Proceedings of the ECCV. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Large-scale training of shadow detectors with noisily-annotated shadow examples, pp. 816–832. Springer International Publishing, Cham (2016)Google Scholar
  5. 5.
    Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the CVPR. Salt Lake City (2018)Google Scholar
  6. 6.
    Al-Najdawi, N., Bez, H.E., Singhai, J., Edirisinghe, E.: A survey of cast shadow detection algorithms. Pattern Recognit. Lett. 33(6), 752–764 (2012).  https://doi.org/10.1016/j.patrec.2011.12.013 CrossRefGoogle Scholar
  7. 7.
    Sanin, A., Sanderson, C., Lovell, B.C.: Shadow detection: a survey and comparative evaluation of recent methods. Pattern Recognit. 45(4), 1684–1695 (2012).  https://doi.org/10.1016/j.patcog.2011.10.001 CrossRefGoogle Scholar
  8. 8.
    Guo, R., Dai, Q., Hoiem, D.: Paired regions for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2956–2967 (2013).  https://doi.org/10.1109/TPAMI.2012.214 CrossRefGoogle Scholar
  9. 9.
    Vicente, T.F.Y., Yu, C.P., Samaras, D.: Single image shadow detection using multiple cues in a supermodular MRF. In: Proceedings of the BMVC. BMVA Press, Bristol (2013)Google Scholar
  10. 10.
    Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection. In: Proceedings of the ICCV, pp. 3388–3396. IEEE, Santiago (2015).  https://doi.org/10.1109/ICCV.2015.387
  11. 11.
    Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 682–695 (2018).  https://doi.org/10.1109/TPAMI.2017.2691703 CrossRefGoogle Scholar
  12. 12.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015).  https://doi.org/10.1038/nature14539 CrossRefGoogle Scholar
  13. 13.
    Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic feature learning for robust shadow detection. In: Proceedings of the CVPR, pp. 1939–1946. IEEE, Columbus (2014).  https://doi.org/10.1109/CVPR.2014.249
  14. 14.
    Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic shadow detection and removal from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 431–446 (2016).  https://doi.org/10.1109/TPAMI.2015.2462355 CrossRefGoogle Scholar
  15. 15.
    Shen, L., Chua, T.W., Leman, K.: Shadow optimization from structured deep edge detection. In: Proceedings of the CVPR, pp. 2067–2074. IEEE, Boston (2015).  https://doi.org/10.1109/CVPR.2015.7298818
  16. 16.
    Nguyen, V., Vicente, T.F.Y., Zhao, M., Hoai, M., Samaras, D.: Shadow detection with conditional generative adversarial networks. In: Proceedings of the ICCV, pp. 4510–4518. Venice (2017)Google Scholar
  17. 17.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc., Red Hook (2014)Google Scholar
  18. 18.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets (2014). arXiv preprint. arXiv:1411.1784
  19. 19.
    Hosseinzadeh, S., Shakeri, M., Zhang, H.: Fast shadow detection from a single image using a patched convolutional neural network (2017). arXiv preprint. arXiv:1709.09283
  20. 20.
    Le, H., Vicente, T.F.Y., Nguyen, V., Hoai, M., Samaras, D.: A+D-Net: shadow detection with adversarial shadow attenuation (2017). arXiv preprint. arXiv:1712.01361
  21. 21.
    Hu, X., Zhu, L., Fu, C.W., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection. In: Proceedings of the CVPR. Salt Lake City (2018)Google Scholar
  22. 22.
    Grana, C., Borghesani, D., Cucchiara, R.: Optimized block-based connected components labeling with decision trees. IEEE Trans Image Process. 19(6), 1596–1609 (2010).  https://doi.org/10.1109/TIP.2010.2044963 MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Harris, M., Sengupta, S., Owens, J.D.: Parallel prefix sum (scan) with CUDA. GPU Gems 3(39), 851–876 (2007)Google Scholar
  24. 24.
    Podlozhnyuk, V.: Image convolution with CUDA. NVIDIA Corporation White Paper 2097(3) (2007)Google Scholar
  25. 25.
    Chen, J., Nonaka, K., Watanabe, R., Sankoh, H., Sabirin, H., Naito, S.: Efficient parallel connected components labeling with a coarse-to-fine strategy (2017). arXiv preprint. arXiv:1712.09789
  26. 26.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision in C++ with the OpenCV Library, 2nd edn. O’Reilly Media, Inc., Sebastopol (2013)Google Scholar
  27. 27.
    Kirk, D.B., Hwu, W.M.W.: Programming Massively Parallel Processors: A Hands-On Approach, 2nd edn. Morgan Kaufmann Publishers Inc., San Francisco (2013)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Márcio C. F. Macedo
    • 1
    Email author
  • Verônica P. Nascimento
    • 2
  • Antonio C. S. Souza
    • 2
  1. 1.Federal University of BahiaSalvadorBrazil
  2. 2.Federal Institute of BahiaSalvadorBrazil

Personalised recommendations