Abstract
Streaming video segmentation decreases processing time by creating supervoxels taking into account small parts of the video instead of using all video content. Thanks to the good performance of the Iterative Spanning Forest to compute Supervoxels (ISF2SVX) based on Dynamic Iterative Spanning Forest (DISF) for video segmentation framework we propose a new graph-based streaming video segmentation method for supervoxel generation by using dynamic iterative spanning forest framework, so-called StreamISF, based on a pipeline composed of six stages: (1) formation of the graph for each block of the video; (2) seed oversampling; (3) IFT-based supervoxel design; (4) reduction in the number of supervoxels; (5) spread of trees; and (6) creation of the segmented video. The difference in our proposed method is that it is unnecessary to have all the video in memory and the only previous information necessary to segment a block is the intersection frame between the blocks. Moreover, experimental results show that StreamISF creates supervoxels that maintain temporal coherence, producing very competitive measures compared to the state-of-the-art. Our code is publically available at https://github.com/IMScience-PPGINF-PucMinas/StreamISF.
The authors thank the Pontifícia Universidade Católica de Minas Gerais – PUC-Minas, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – CAPES – (Grant COFECUB 88887.191730/2018-00, Grant PROAP 88887.842889/2023-00 – PUC/MG and Finance Code 001), the Conselho Nacional de Desenvolvimento Científico e Tecnológico – CNPq (Grants 303808/2018-7, 407242/2021-0, 306573/2022-9) and Fundação de Apoio à Pesquisa do Estado de Minas Gerais – FAPEMIG (Grant APQ-01079-23).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)
Jerônimo, C., et al.: Graph-based supervoxel computation from iterative spanning forest. In: Lindblad, J., Malmberg, F., Sladoje, N. (eds.) DGMM 2021. LNCS, vol. 12708, pp. 404–415. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-76657-3_29
Belém, F., Guimarães, S., Falcão, A.: Superpixel segmentation using dynamic and iterative spanning forest. Signal Process. Lett. 27, 1440–1444 (2020)
Bragantini, J., Martins, S.B., Castelo-Fernandez, C., Falcão, A.X.: Graph-based image segmentation using dynamic trees. In: Vera-Rodriguez, R., Fierrez, J., Morales, A. (eds.) CIARP 2018. LNCS, vol. 11401, pp. 470–478. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13469-3_55
Chen, A., Corso, J.: Propagating multi-class pixel labels throughout video frames. In: Western New York Image Processing Workshop, pp. 14–17 (2010)
Ciesielski, C., Falcão, A., Miranda, P.: Path-value functions for which Dijkstra’s algorithm returns optimal mapping. J. Math. Imaging Vision 60(7), 1025–1036 (2018)
De Souza, K.J.F., et al.: Hierarchical video segmentation using an observation scale. In: Hirata, N., Nedel, L., Silva, C., Boyer, K. (eds.) SIBGRAPI 2013 (XXV Conference on Graphics, Patterns and Images), Arequipa (Aug 2013)
De Souza, K.J.F., De Albuquerque Araújo, A., Do Patrocínio, Jr., Z.K.G., Guimarães, S.J.F.: Graph-based hierarchical video segmentation based on a simple dissimilarity measure. Pattern Recogn. Lett. 47, 85–92 (2014). https://doi.org/10.1016/j.patrec.2014.02.016
Falcão, A., Stolfi, J., Lotufo, R.: The image foresting transform: theory, algorithms, and applications. Trans. Pattern Anal. Mach. Intell. 26(1), 19–29 (2004)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. Int. J. Comput. Vision 59(2), 167–181 (2004)
Griffin, B.A., Corso, J.J.: Video object segmentation using supervoxel-based gerrymandering. arXiv preprint arXiv:1704.05165 (2017)
Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: Computer Vision and Pattern Recognition (CVPR), pp. 2141–2148. IEEE (2010)
Oneata, D., Revaud, J., Verbeek, J., Schmid, C.: Spatio-temporal object detection proposals. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 737–752. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_48
Papon, J., Abramov, A., Schoeler, M., Worgotter, F.: Voxel cloud connectivity segmentation-supervoxels for point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2027–2034 (2013)
Sheng, H., Zhang, X., Zhang, Y., Wu, Y., Chen, J., Xiong, Z.: Enhanced association with supervoxels in multiple hypothesis tracking. IEEE Access 7, 2107–2117 (2018)
de Souza, K.J.F., de Albuquerque Araújo, A., Guimarães, S.J.F., do Patrocínio, Z.K.G., Cord, M.: Streaming graph-based hierarchical video segmentation by a simple label propagation. In: 28th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI 2015), Salvador, 26–29 August 2015, pp. 119–125. IEEE Computer Society (2015). https://doi.org/10.1109/SIBGRAPI.2015.23
Vargas-Muñoz, J., Chowdhury, A., Alexandre, E., Galvão, F., Miranda, P., Falcão, A.: An iterative spanning forest framework for superpixel segmentation. Trans. Image Process. 28(7), 3477–3489 (2019)
Verdoja, F., Thomas, D., Sugimoto, A.: Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understanding. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 1285–1290. IEEE (2017)
Wang, B., et al.: Real-time hierarchical supervoxel segmentation via a minimum spanning tree. Trans. Image Process. 29, 9665–9677 (2020)
Wu, F., et al.: Rapid localization and extraction of street light poles in mobile lidar point clouds: a supervoxel-based approach. IEEE Trans. Intell. Transp. Syst. 18(2), 292–305 (2016)
Xu, C., Xiong, C., Corso, J.J.: Streaming hierarchical video segmentation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 626–639. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_45
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this paper
Cite this paper
Vieira, D., Barcelos, I.B., Belém, F., Patrocínio, Z.K.G., Falcão, A.X., Guimarães, S.J.F. (2024). Streaming Graph-Based Supervoxel Computation Based on Dynamic Iterative Spanning Forest. In: Vasconcelos, V., Domingues, I., Paredes, S. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2023. Lecture Notes in Computer Science, vol 14470. Springer, Cham. https://doi.org/10.1007/978-3-031-49249-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-49249-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49248-8
Online ISBN: 978-3-031-49249-5
eBook Packages: Computer ScienceComputer Science (R0)
