Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark

  • Bilal Iqbal
  • Waheed IqbalEmail author
  • Nazar Khan
  • Arif Mahmood
  • Abdelkarim Erradi


Nowadays, video cameras are increasingly used for surveillance, monitoring, and activity recording. These cameras generate high resolution image and video data at large scale. Processing such large scale video streams to extract useful information with time constraints is challenging. Traditional methods do not offer scalability to process large scale data. In this paper, we propose and evaluate cloud services for high resolution video streams in order to perform line detection using Canny edge detection followed by Hough transform. These algorithms are often used as preprocessing steps for various high level tasks including object, anomaly, and activity recognition. We implement and evaluate both Canny edge detector and Hough transform algorithms in Hadoop and Spark. Our experimental evaluation using Spark shows an excellent scalability and performance compared to Hadoop and standalone implementations for both Canny edge detection and Hough transform. We obtained a speedup of 10.8\(\times\) and 9.3\(\times\) for Canny edge detection and Hough transform respectively using Spark. These results demonstrate the effectiveness of parallel implementation of computer vision algorithms to achieve good scalability for real-world applications.


Hough transform Canny edge detection Video processing Spark Hadoop MapReduce 



This work was made possible by NPRP Grant # 7-481-1-088 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors.


  1. 1.
    Anjum, A., Abdullah, T., Tariq, M., Baltaci, Y., Antonopoulos, N.: Video stream analysis in clouds: an object detection and classification framework for high performance video analytics. IEEE Trans. Cloud Comput. (2016)Google Scholar
  2. 2.
    Arsh, S., Bhatt, A., Kumar, P.: Distributed image processing using hadoop and HIPI. In: 2016 International Conference on Advances in Computing, Communications and Informatics, ICACCI 2016, Jaipur, India, September 21–24, 2016, pp. 2673–2676 (2016)Google Scholar
  3. 3.
    Arthanari, J., Baskaran, R.: Enhancement of video streaming analysis using cluster-computing framework. Clust. Comput. 3 (2018)Google Scholar
  4. 4.
    Arunkumar, P., Shantharajah, S., Geetha, M.: Improved canny detection algorithm for processing and segmenting text from the images. Clust. Comput., pp. 1–7 (2018)Google Scholar
  5. 5.
    Chen, L., Chen, H., Pan, Y., Chen, Y.: A fast efficient parallel Hough transform algorithm on LARPBS. J. Supercomput. 29(2), 185–195 (2004)CrossRefzbMATHGoogle Scholar
  6. 6.
    Chen, J., Li, K., Tang, Z., Bilal, K., Yu, S., Weng, C., Li, K.: A parallel random forest algorithm for big data in a Spark cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 28(4), 919–933 (2017)CrossRefGoogle Scholar
  7. 7.
    Deanm, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  8. 8.
    Gentsos, C., Sotiropoulou, C.-L., Nikolaidis, S., Vassiliadis, N.: Real-time canny edge detection parallel implementation for fpgas. In: 2010 17th IEEE International Conference on Electronics, Circuits, and Systems (ICECS), pp. 499–502. IEEE (2010)Google Scholar
  9. 9.
    Halyo, V., LeGresley, P., Lujan, P., Karpusenko, V., Vladimirov, A.: First evaluation of the CPU, GPGPU and MIC architectures for real time particle tracking based on Hough transform at the LHC. J. Instrum. 9(04), P04005 (2014)CrossRefGoogle Scholar
  10. 10.
    Huang, W., Meng, L., Zhang, D., Zhang, W.: In-memory parallel processing of massive remotely sensed data using an Apache Spark on Hadoop YARN model. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 10(1), 3–19 (2017)CrossRefGoogle Scholar
  11. 11.
    Ismail El-Helw, R. H.: Scaling mapreduce vertically and horizontally. In: SC14: International Conference for High Performance Computing, Networking, Storage and Analysis (2014)Google Scholar
  12. 12.
    Jatrniko, W., Arsa, D.M.S., Wisesa, H., Jati, G., Ma’sum, M.A.: A review of big data analytics in the biomedical field. In: International Workshop on Big Data and Information Security (IWBIS), pp. 31–41. IEEE (2016)Google Scholar
  13. 13.
    Lv, J., Wu, B., Yang, S., Jia, B., Qiu, P.: Efficient large scale near-duplicate video detection base on Spark. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 957–962. IEEE (2016)Google Scholar
  14. 14.
    Mukhopadhyay, P., Chaudhuri, B.B.: A survey of hough transform. Pattern Recognit. 48(3), 993–1010 (2015)CrossRefGoogle Scholar
  15. 15.
    Rathore, M.M., Son, H., Ahmad, A., Paul, A., Jeon, G.: Real-time big data stream processing using GPU with Spark over Hadoop ecosystem. Int. J. Parallel Program. pp. 1–17 (2017)Google Scholar
  16. 16.
    Sweeney, C., Liu, L., Arietta, S., Lawrence, J.: HIPI: A Hadoop Image Processing Interface for Image-Based Mapreduce Tasks. University of Virginia, Chris (2011)Google Scholar
  17. 17.
    van den Braak, G.-J., Nugteren, C., Mesman, B., Corporaal, H.: Fast hough transform on gpus: Exploration of algorithm trade-offs. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 611–622. Springer (2011)Google Scholar
  18. 18.
    Waghule, D.R., Ochawar, R.S.: Overview on edge detection methods. In: 2014 International Conference on Electronic Systems, Signal Processing and Computing Technologies (ICESC), pp. 151–155. IEEE (2014)Google Scholar
  19. 19.
    Xu, Q., Varadarajan, S., Chakrabarti, C., Karam, L.J.: A distributed canny edge detector: algorithm and FPGA implementation. IEEE Trans. Image Process. 23(7), 2944–2960 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Yam-Uicab, R., Lopez-Martinez, J., Trejo-Sanchez, J., Hidalgo-Silva, H., Gonzalez-Segura, S.: A fast hough transform algorithm for straight lines detection in an image using gpu parallel computing with CUDA-C. J. Supercomput. 73(11), 4823–4842 (2017)CrossRefGoogle Scholar
  21. 21.
    Yaseen, M.U., Anjum, A., Rana, O., Hill, R.: Cloud-based scalable object detection and classification in video streams. Future Gener. Comput. Syst. 80, 286–298 (2018)CrossRefGoogle Scholar
  22. 22.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. HotCloud 10, 10–10 (2010)Google Scholar
  23. 23.
    Zaharia, M., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M.J., et al.: Apache spark: a unified engine for big data processing. Commun. ACM 59(11), 56–65 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Punjab University College of Information TechnologyLahorePakistan
  2. 2.Information Technology UniversityLahorePakistan
  3. 3.Department of Computer ScienceQatar UniversityDohaQatar

Personalised recommendations