Advertisement

Real-time shadow detection using multi-channel binarization and noise removal

  • 202 Accesses

Abstract

High-quality automatic shadow detection remains a challenging problem in image processing and computer vision. Existing techniques for shadow detection typically make use of deep learning strategies to obtain accurate shadow detection results, at the cost of demanding high processing time, making their use unsuitable for augmented reality and robotic applications. In this paper, we propose a novel approach to perform high-quality shadow detection in real time. To do so, we convert an input image into different color spaces to perform multi-channel binarization and detect different shadow regions in the image. Then, a filtering algorithm is proposed to remove the noisy false-positive shadow regions on the basis of their sizes. Experimental results evaluated in two different datasets show that the proposed approach may run entirely on the GPU, requiring only \(\approx\) 13 ms to detect shadows in an image with \(3840 \times 2160\) (4k) resolution. That makes our approach about 1.8 (66\(\times\)) to 4.6 (37,284\(\times\)) orders of magnitude faster than related work for 4k resolution images, at the cost of only \(\approx\) 5% of accuracy loss compared to the best results achieved for each dataset.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. 1.

    https://developer.nvidia.com/npp.

  2. 2.

    https://thrust.github.io/.

References

  1. 1.

    Liu, Y., Granier, X.: Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE Trans. Vis. Comput. Graph. 18, 573–580 (2012). https://doi.org/10.1109/TVCG.2012.53

  2. 2.

    Zhu, J., Samuel, K.G.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: Proceedings of the CVPR, pp. 223–230. IEEE, San Francisco (2010). https://doi.org/10.1109/CVPR.2010.5540209

  3. 3.

    Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. In: Proceedings of the CVPR, pp. 2033–2040. IEEE, Colorado Springs (2011). https://doi.org/10.1109/CVPR.2011.5995725

  4. 4.

    Vicente, T.F.Y., Hou, L., Yu, C.P., Hoai, M., Samaras, D.: Proceedings of the ECCV. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Large-scale training of shadow detectors with noisily-annotated shadow examples, pp. 816–832. Springer International Publishing, Cham (2016)

  5. 5.

    Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the CVPR. Salt Lake City (2018)

  6. 6.

    Al-Najdawi, N., Bez, H.E., Singhai, J., Edirisinghe, E.: A survey of cast shadow detection algorithms. Pattern Recognit. Lett. 33(6), 752–764 (2012). https://doi.org/10.1016/j.patrec.2011.12.013

  7. 7.

    Sanin, A., Sanderson, C., Lovell, B.C.: Shadow detection: a survey and comparative evaluation of recent methods. Pattern Recognit. 45(4), 1684–1695 (2012). https://doi.org/10.1016/j.patcog.2011.10.001

  8. 8.

    Guo, R., Dai, Q., Hoiem, D.: Paired regions for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2956–2967 (2013). https://doi.org/10.1109/TPAMI.2012.214

  9. 9.

    Vicente, T.F.Y., Yu, C.P., Samaras, D.: Single image shadow detection using multiple cues in a supermodular MRF. In: Proceedings of the BMVC. BMVA Press, Bristol (2013)

  10. 10.

    Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection. In: Proceedings of the ICCV, pp. 3388–3396. IEEE, Santiago (2015). https://doi.org/10.1109/ICCV.2015.387

  11. 11.

    Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 682–695 (2018). https://doi.org/10.1109/TPAMI.2017.2691703

  12. 12.

    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539

  13. 13.

    Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic feature learning for robust shadow detection. In: Proceedings of the CVPR, pp. 1939–1946. IEEE, Columbus (2014). https://doi.org/10.1109/CVPR.2014.249

  14. 14.

    Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic shadow detection and removal from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 431–446 (2016). https://doi.org/10.1109/TPAMI.2015.2462355

  15. 15.

    Shen, L., Chua, T.W., Leman, K.: Shadow optimization from structured deep edge detection. In: Proceedings of the CVPR, pp. 2067–2074. IEEE, Boston (2015). https://doi.org/10.1109/CVPR.2015.7298818

  16. 16.

    Nguyen, V., Vicente, T.F.Y., Zhao, M., Hoai, M., Samaras, D.: Shadow detection with conditional generative adversarial networks. In: Proceedings of the ICCV, pp. 4510–4518. Venice (2017)

  17. 17.

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc., Red Hook (2014)

  18. 18.

    Mirza, M., Osindero, S.: Conditional generative adversarial nets (2014). arXiv preprint. arXiv:1411.1784

  19. 19.

    Hosseinzadeh, S., Shakeri, M., Zhang, H.: Fast shadow detection from a single image using a patched convolutional neural network (2017). arXiv preprint. arXiv:1709.09283

  20. 20.

    Le, H., Vicente, T.F.Y., Nguyen, V., Hoai, M., Samaras, D.: A+D-Net: shadow detection with adversarial shadow attenuation (2017). arXiv preprint. arXiv:1712.01361

  21. 21.

    Hu, X., Zhu, L., Fu, C.W., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection. In: Proceedings of the CVPR. Salt Lake City (2018)

  22. 22.

    Grana, C., Borghesani, D., Cucchiara, R.: Optimized block-based connected components labeling with decision trees. IEEE Trans Image Process. 19(6), 1596–1609 (2010). https://doi.org/10.1109/TIP.2010.2044963

  23. 23.

    Harris, M., Sengupta, S., Owens, J.D.: Parallel prefix sum (scan) with CUDA. GPU Gems 3(39), 851–876 (2007)

  24. 24.

    Podlozhnyuk, V.: Image convolution with CUDA. NVIDIA Corporation White Paper 2097(3) (2007)

  25. 25.

    Chen, J., Nonaka, K., Watanabe, R., Sankoh, H., Sabirin, H., Naito, S.: Efficient parallel connected components labeling with a coarse-to-fine strategy (2017). arXiv preprint. arXiv:1712.09789

  26. 26.

    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision in C++ with the OpenCV Library, 2nd edn. O’Reilly Media, Inc., Sebastopol (2013)

  27. 27.

    Kirk, D.B., Hwu, W.M.W.: Programming Massively Parallel Processors: A Hands-On Approach, 2nd edn. Morgan Kaufmann Publishers Inc., San Francisco (2013)

Download references

Acknowledgements

We are thankful to Guo et al. [3], Hosseinzadeh et al. [19] and Le et al. [20] for gently sharing the source code of their shadow detection algorithms. This research is supported by the scholarship program of Coordenação de Aperfeiçoamento de Pessoal do Nível Superior (CAPES). The hardware used for processing time evaluation was provided by NVIDIA Corporation, through the GPU Education Center.

Author information

Correspondence to Márcio C. F. Macedo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 18897 KB)

Supplementary material 1 (mp4 18897 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Macedo, M.C.F., Nascimento, V.P. & Souza, A.C.S. Real-time shadow detection using multi-channel binarization and noise removal. J Real-Time Image Proc (2018). https://doi.org/10.1007/s11554-018-0799-3

Download citation

Keywords

  • Shadow detection
  • Parallel processing
  • Binarization
  • Noise removal
  • Real time