Journal of Real-Time Image Processing

, Volume 15, Issue 2, pp 363–387 | Cite as

A parallel LEGION algorithm and cell-based architecture for real time split and merge video segmentation

  • Pradipta RoyEmail author
  • Prabir Kumar Biswas
Original Research Paper


Split and merge segmentation is a popular region-based segmentation scheme for its robustness and computational efficiency. But it is hard to realize for larger size images or video frames in real time due to its iterative sequential data flow pattern. A quad-tree data structure is quite popular for software implementation of the algorithm, where a local parallelism is difficult to establish due to inherent data dependency between processes. In this paper, we have proposed a parallel algorithm of splitting and merging which depends only on local operations. The algorithm is mapped onto a hierarchical cell network, which is a parallel version of Locally Excitory Globally Inhibitory Oscillatory Network (LEGION). Simulation results show that the proposed design is faster than any of the standard split and merge algorithmic implementations, without compromising segmentation quality. The timing performance enhancement is manifested in its Finite State Machine based VLSI implementation in VIRTEX series FPGA platforms. We have also shown that, though segmentation qualitywise split-and-merge algorithm is little bit behind the state-of-the-art algorithms, computational speedwise it over performs those sophisticated and complex algorithms. Good segmentation performance with minimal computational cost enables the proposed design to tackle real time segmentation problem in live video streams. In this paper, we have demonstrated live PAL video segmentation using VIRTEX 5 series FPGA. Moreover, we have extended our design to HD resolution for which the time taken is less than 5 ms rendering a processing throughput of 200 frames per second.


Split and merge segmentation VLSI Parallel architectures 


  1. 1.
    Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Associative hierarchical CRFs for object class image segmentation. In: IEEE 12 th International Conference on Computer Vision (ICCV 2009), pp. 739–746 (2009)Google Scholar
  2. 2.
    Lecumberry, F., Pardo, A., Sapiro, G.: Simultaneous object classification and segmentation with high-order multiple shape models. IEEE Trans. Image Process. 19(3), 625–635 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Peng, B., Zhang, D., Zhang, D.: Automatic image segmentation by dynamic region merging. IEEE Trans. Image Process. 20(12), 3592–3605 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Kimmel, R., Bruckstein, A.M.: Regularized Laplacian zero crossings and other optimal edge integrators. Int. J. Comput. Vis. 53(3), 225–243 (2003)CrossRefGoogle Scholar
  5. 5.
    Senthilkumaran, N., Rajesh, R.: Edge detection techniques for image segmentation—a survey of soft computing approaches. Int. J. Recent Trends Eng. 1(2), (2009)Google Scholar
  6. 6.
    Gonzalez, R.C., Woods, R.E.: Digital image processing, 3rd edn. Prentice Hall, NJ, USA (2008)Google Scholar
  7. 7.
    Pratt, W.K.: Digital Image Processing, 4th edn. John Wiley & Sons Inc, Los Altos (2007)CrossRefzbMATHGoogle Scholar
  8. 8.
    Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991)CrossRefGoogle Scholar
  9. 9.
    Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)CrossRefGoogle Scholar
  10. 10.
    Hunter, G.M., Steiglitz, K.S.: Operations on images using quad- tree. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 145–153 (1979)CrossRefGoogle Scholar
  11. 11.
    Haris, K., Efstratiadis, S.N., Maglaveras, N., Katsaggelos, A.K.: Hybrid image segmentation using watersheds and fast region merging. IEEE Trans. Image Process. 7(12), 1684–1699 (1998)CrossRefGoogle Scholar
  12. 12.
    Chen, S., Lin, W., Chen, C.: Split-and-merge image segmentation based on localized feature analysis and statistical tests. J CVGIP Gr Models Image Process. 53(5), 457–475 (1991)CrossRefGoogle Scholar
  13. 13.
    Serratosa, F., Millan, P.,Montseny, E.: Systolic processors applied to computer vision systems. In: Proceedings of the IEEE Computer Architectures for Machine Perception, (CAMP’95), pp 178–183 (1995)Google Scholar
  14. 14.
    Jonker, P.P.: An SIMD-MIMD architecture for image processing and pattern recognition. In: Proceedings of the IEEE Computer Architectures for Machine Perception, (CAMP’93), pp 222–230 (1993)Google Scholar
  15. 15.
    Masaki, I.: Parallel/pipelined processor dedicated to visual recognition. In Proceed. IEEE Int. Conf. Robot. Autom. 2, 100–107 (1985)Google Scholar
  16. 16.
    Kim, J., Chen, T.: A VLSI architecture for video-object segmentation. IEEE Trans. Circuits Syst. Video Technol. 13(1), 83–96 (2003)CrossRefGoogle Scholar
  17. 17.
    Arena, P., Basile, A., Bucolo, M., Fortuna, L.: An object oriented segmentation on analog CNN chip. IEEE Trans. Circuits Syst. I 50(7), 837–846 (2003)CrossRefGoogle Scholar
  18. 18.
    Chen, T., Sun, C., Su, H., Chien, S., Deguchi, D., Ide, I., Murase, H.: Power-efficient hardware architecture of K-means clustering with Bayesian-information-criterion processor for multimedia processing applications. IEEE J. Emerg. Sel. Topics Circuits Syst. 1(3), 357–368 (2011)CrossRefGoogle Scholar
  19. 19.
    Sirotkovic, J., Dujmic, H., Papic, V.: K-means image segmentation on massively parallel GPU architecture. In: Proceedings of the IEEE 35th International Convention MIPRO, 2012, pp. 489–484 (2012)Google Scholar
  20. 20.
    Horowitz, S., Pavlidis, T.: Picture segmentation by a tree traversal algorithm. J. ACM. 23, 368–388 (1976)CrossRefzbMATHGoogle Scholar
  21. 21.
    Kunlin, T., Lacassagne, L., Merigot, A.: A fast image segmentation scheme. In: Proceedings of the International Conference on Information and Communication Technologies: From Theory to Applications, (ICTTA2004), pp. 351–352 (2004)Google Scholar
  22. 22.
    Zhang, Z., Chen, C., Sun, J., Chan, K.L.: EM algorithms for Gaussian mixtures with split-and-merge operation. Pattern Recognit. 36(9), 1973–1983 (2003)CrossRefzbMATHGoogle Scholar
  23. 23.
    Panagiotakis, C., Grinias, I., Tziritas, G.: Natural image segmentation based on tree equipartition, Bayesian flooding and region merging. IEEE Trans. Image Process. 20(8), 2276–2287 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Willebeek-LeMair, M., Reeves, A.P.: Solving nonuniform problems on SIMD computers: case study on region growing. J. Parallel Distrib. Comput. 8(2), 135–149 (1990)CrossRefGoogle Scholar
  25. 25.
    Montoya, M.D.G., Gil, C., Garcia, I.: The load unbalancing problem for region growing image segmentation algorithms. J. Parallel Distrib. Comput. 63(4), 387–395 (2003)CrossRefzbMATHGoogle Scholar
  26. 26.
    Xuejing, G., Linlin, C., Kangze, Y.: Two parallel strategies of split-merge algorithm for image segmentation. IEEE Int. Conf. Wavelet Anal Pattern Recognit. 2, 840–845 (2007)CrossRefGoogle Scholar
  27. 27.
    Roy, P., Das, D., Biswas, P.K.: Real time VLSI implementation of a fast split and merge segmentation algorithm. In: Proceedings of the IEEE International Conference on Computational Intelligence and Computing Research (ICCIC 2012), pp. 114–121 (2012)Google Scholar
  28. 28.
    Faruquzzaman, A. B. M., Paiker, N. R., Arafat, J., Ali, M. A., Sorwar, G.: Literature on image segmentation based on split and merge techniques. In: Proceedings of the 5th International Conference on Information Technology and Applications (ICITA 2008), pp. 120–125 (2008)Google Scholar
  29. 29.
    Prabhashankar, L.: Parallel implementation of the split and merge algorithm on the hypercube machine. MS Thesis, Ohio University (1989)Google Scholar
  30. 30.
    Tyagi, A., Bayoumi, M.A.: Image segmentation on a 2D array by a directed split and merge procedure. IEEE Trans. Signal Process. 40(11), 2804–2813 (1992)CrossRefGoogle Scholar
  31. 31.
    Copty, N., Ranka, S., Fox, G., Shankar, R.: Solving the region growing problem on the connection machine. In Proceed. Int. Conf. Parallel Process. 3, 102–105 (1993)Google Scholar
  32. 32.
    Reyes, A, Martinez, V., Ramirez, A.: Parallel design the algorithm of digital images segmentation split and merge. In: Conferencia Latinoamericana de Computación de Alto Rendimiento, vol. 1, pp. 85–89 (2009)Google Scholar
  33. 33.
    Wang, D., Terman, D.: Image segmentation based on oscillatory correlation. J. Neural Comput. 9(4), 805–836 (1997)CrossRefGoogle Scholar
  34. 34.
    Ando, H., Miyake, M., Morie, T., Nagata, M., Iwata, A.: A nonlinear oscillator network circuit for image segmentation with double- threshold phase detection. IEEE Ninth Int. Conf. Artif Neural Netw. 2, 655–660 (1999)CrossRefGoogle Scholar
  35. 35.
    Morimoto, T., Harada, Y., Koide, T., Mattausch, H. J.: Pixel-parallel digital CMOS implementation of image segmentation by region growing. IEE Proceed. Circuits Devices Syst. 152(6), 579–589 (2005)CrossRefGoogle Scholar
  36. 36.
    Roy P., Biswas P. K.: VLSI implementation of fast connected component labeling using finite state machine based cell network. In: Proceedings of the Sixth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP’ 08), pp. 238–243 (2008)Google Scholar
  37. 37.
    Fernandes, D., Stedile, J., Navaux, P.: Architecture of oscillatory neural network for image segmentation. In: IEEE 14th Symposium on Computer Architecture and High Performance Computing (CAHPC 02), pp. 29–36 (2002).Google Scholar
  38. 38.
  39. 39.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the eighth IEEE International Conference on Computer Vision (ICCV01), vol. 2, pp. 416–423 (2001)Google Scholar
  40. 40.
    Kranthi Kumar, J. D., Srinivasan, S.: A novel VLSI architecture to implement region merging algorithm for image segmentation. In: IEEE Euromicro Symposium on Digital System Design, pp. 620–623 (2004)Google Scholar
  41. 41.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  42. 42.
    Khan, S.K., Ahmad, A.: Cluster center initialization algorithm for k-means clustering. Pattern Recogn. Lett. 25, 1293–1302 (2004)CrossRefGoogle Scholar
  43. 43.
    Pham, D.L., Prince, J.L.: An adaptive fuzzy C-means algorithm for image segmentation in the presence of intensity inhomogeneities. Pattern Recognit. Lett. 20(1), 57–68 (1999)CrossRefzbMATHGoogle Scholar
  44. 44.
    Wanga, X., Wanga, T., Bu, J.: Color image segmentation using pixel wise support vector machine classification. Pattern Recognit. 44(4), 777–787 (2011)CrossRefzbMATHGoogle Scholar
  45. 45.
    Deng, H., Clausi, D.A.: Unsupervised image segmentation using a simple MRF model with a new implementation scheme. In proceed. 17th Int. Conf Pattern Recognit. 2, 691–694 (2004)CrossRefGoogle Scholar
  46. 46.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)CrossRefGoogle Scholar
  47. 47.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  48. 48.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRefGoogle Scholar
  49. 49.
    Li, K., Song X.: A fast large size image segmentation algorithm based on spectral clustering. In: IEEE Fourth International Conference on Computational and Information Sciences (ICCIS 2012), pp. 345–348 (2012)Google Scholar
  50. 50.
    Feng, W., Jia, J., Liu, Z.: Self-validated labeling of Markov random fields for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1871–1887 (2010)CrossRefGoogle Scholar
  51. 51.
    Garrett, Z., Saito, H.: Live video object tracking and segmentation using graph cuts. In: 15th IEEE International Conference on Image Processing (ICIP 2008), pp. 1576–1579 (2008)Google Scholar
  52. 52.
    Liu, J., Sun J.: Parallel graph-cuts by adaptive bottom-up merging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010), pp. 2181–2188 (2010)Google Scholar
  53. 53.
    Kobori, D., Maruyama, T.: An acceleration of a graph cut segmentation with FPGA. In: 22nd International Conference on Field Programmable Logic and Applications (FPL 2012), pp. 407–413 (2012)Google Scholar
  54. 54.
  55. 55.
    Yang, A.Y., Wright, J., Ma, Y., Sastry, S.S.: Unsupervised segmentation of natural images via lossy data compression. Comput. Vis. Image Underst. 110(2), 212–225 (2008)CrossRefGoogle Scholar
  56. 56.
    Vineet, V., Narayanan, P.J.: CUDA cuts: fast graph cuts on the GPU. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 08), pp. 1–8 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Optronics CentreIntegrated Test RangeChandipurIndia
  2. 2.Department of E and ECEIndian Institute of TechnologyKharagpurIndia

Personalised recommendations