Skip to main content
SpringerLink
Log in

Graph convolutional network-based semi-supervised feature classification of volumes

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

Feature classification has always been one of the research hotspots in scientific visualization. However, conventional interactive feature classification methods rely on prior knowledge and typically require trial and error, whereas feature classification based on data mining is generally based on local features; therefore, obtaining good results with traditional methods is difficult. In this paper, we first map a volume to the super-voxel graph using a 3D extension of the simple linear iterative clustering algorithm and then construct a graph convolutional neural network to implement node classification in a semi-supervised way, i.e., a small number of user-labeled super-voxels. We transform the feature classification of a volume into the classification task of nodes of a super-voxel graph, which is a novel approach and broadens the application scope of graph neural network to volumes. Experiments on different volumes have demonstrated the strong learning ability and reasoning ability of the proposed method.

Graphic abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado G.S, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/. Software available from tensorflow.org

  • Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transac Pattern Analy Machine Intell 34(11):2274–2282

    Article  Google Scholar 

  • Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203

  • Caban JJ, Rheingans P (2008) Texture-based transfer functions for direct volume rendering. IEEE Transact Visuali Comput Graphics 14(6):1364–1371

    Article  Google Scholar 

  • Cai LL, Nguyen BP, Chui CK, Ong SH (2015) Rule-enhanced transfer function generation for medical volume visualization. Comput Graph Forum 34(3):121–130

    Article  Google Scholar 

  • Correa CD, Ma K (2009) The occlusion spectrum for volume classification and visualization. IEEE Transact Visuali Comput Graphics 15(6):1465–1472

    Article  Google Scholar 

  • Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Transact Inf Theory 13(1):21–27

    Article  Google Scholar 

  • Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? The J Machine Learn Res 15(1):3133–3181

    MathSciNet  MATH  Google Scholar 

  • Gao H, Ji S (2019) Graph u-nets. In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, pp. 2083–2092

  • Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl, GE (2017) Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning, 70: 1263–1272

  • Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: IEEE International Joint Conference on Neural Networks, vol. 2, pp. 729–734. IEEE

  • Hagberg AA, Schult DA, Swart PJ (2008) Exploring network structure, dynamics, and function using networkx. In: Proceedings of the 7th Python in Science Conference, pp. 11–15

  • Haidacher M, Patel D, Bruckner S, Kanitsar A, Gröller M.E (2010) Volume visualization based on statistical transfer-function spaces. In: IEEE Pacific Visualization Symposium PacificVis 2010, Taipei, Taiwan, March 2-5, 2010, pp. 17–24. IEEE Computer Society

  • Hamilton WL, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: I. Guyon, U. von Luxburg, S. Bengio, H.M. Wallach, R. Fergus, S.V.N. Vishwanathan, R. Garnett (eds.) Advances in Neural Information Processing Systems, pp. 1024–1034

  • He X, Tao Y, Wang Q, Lin H (2018) Biclusters based visual exploration of multivariate scientific data. Proc IEEE Scient Visuali Conf (SciVis) 2018:40–45

    Google Scholar 

  • He X, Tao Y, Wang Q, Lin H (2018) A co-analysis framework for exploring multivariate scientific data. Visual Inf 2(4):254–263

    Article  Google Scholar 

  • He X, Tao Y, Wang Q, Lin H (2019) Multivariate spatial data visualization: a survey. J Visualization 22(5):897–912

    Article  Google Scholar 

  • Jadhav S, Nadeem S, Kaufman AE (2019) Featurelego: volume exploration using exhaustive clustering of super-voxels. IEEE Transact Visualization Comput Graphics 25(9):2725–2737

    Article  Google Scholar 

  • Kehrer J, Hauser H (2012) Visualization and visual analysis of multifaceted scientific data: a survey. IEEE Transact Visualization Comput Graphics 19(3):495–513

    Article  Google Scholar 

  • Kindlmann G, Durkin J.W (1998) Semi-automatic generation of transfer functions for direct volume rendering pp. 79–86

  • Kindlmann GL, Whitaker RT, Tasdizen T, Möller T (1996) Curvature-based transfer functions for direct volume rendering: Methods and applications. In: 14th IEEE Visualization Conference, pp. 513–520

  • Kingma D.P, Ba J (2015) Adam: a method for stochastic optimization. In: International Conference on Learning Representation, pp. 1–15

  • Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  • Kniss J, Kindlmann GL, Hansen CD (2002) Multidimensional transfer functions for interactive volume rendering. IEEE Transact Visualization Comput Grap 8(3):270–285

    Article  Google Scholar 

  • Kohonen T (1990) The self-organizing map. Proc IEEE 78:1464–1480

    Article  Google Scholar 

  • Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, 32

  • Ljung P, Krüger J, Gröller E, Hadwiger M, Hansen CD, Ynnerman A (2016) State of the art in transfer functions for direct volume rendering. Comput Graph Forum 35(3):669–691

    Article  Google Scholar 

  • Micheli A (2009) Neural network for graphs: a contextual constructive approach. IEEE Transact Neural Netw 20(3):498–511

    Article  MathSciNet  Google Scholar 

  • Patel D, Haidacher M, Balabanian J, Gröller M.E (2009) Moment curves. In: IEEE Pacific Visualization Symposium PacificVis, pp. 201–208

  • Praßni J, Ropinski T, Mensmann J, Hinrichs K.H (2010) Shape-based transfer functions for volume visualization. In: IEEE Pacific Visualization Symposium PacificVis, pp. 9–16

  • Quan TM, Choi J, Jeong H, Jeong W (2018) An intelligent system approach for probabilistic volume rendering using hierarchical 3d convolutional sparse coding. IEEE Transact Visualization Comput Graph 24(1):964–973

    Article  Google Scholar 

  • Ren X, Malik J (2003) Learning a classification model for segmentation. In: IEEE International Conference on Computer Vision, pp. 10–17

  • Simpson A.L, Antonelli M, Bakas S, Bilello M, Farahani K, Van Ginneken B, Kopp-Schneider A, Landman B.A, Litjens G, Menze B, et al (2019) A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063

  • Soundararajan KP, Schultz T (2015) Learning probabilistic transfer functions: a comparative study of classifiers. Comput Graph Forum 34(3):111–120

    Article  Google Scholar 

  • Tzeng F, Lum EB, Ma K (2005) An intelligent system approach to higher-dimensional classification of volume data. IEEE Transact Visualization Comput Graph 11(3):273–284

    Article  Google Scholar 

  • Tzeng, FY, Ma KL (2004) A Cluster-Space Visual Interface for Arbitrary Dimensional Classification of Volume Data. In: Eurographics / IEEE VGTC Symposium on Visualization. The Eurographics Association

  • Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv preprint arXiv:1710.10903

  • Wu F, Chen G, Huang J, Tao Y, Chen W (2015) Easyxplorer: a flexible visual exploration approach for multivariate spatial data. Comput Graph Forum 34(7):163–172

    Article  Google Scholar 

  • Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K.i, Jegelka S (2018) Representation learning on graphs with jumping knowledge networks. In: International Conference on Machine Learning, pp. 5453–5462. PMLR

  • Yushkevich PA, Piven J, Cody Hazlett H, Gimpel Smith R, Ho S, Gee JC, Gerig G (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128

    Article  Google Scholar 

  • Zhou L, Hansen C (2014) Guideme: slice-guided semiautomatic multivariate exploration of volumes. Comput Graph Forum 33(3):151–160

    Article  Google Scholar 

  • Zhou L, Schott M, Hansen C (2012) Transfer function combinations. Comput Graph 36(6):596–606

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (61890954 and 61972343) and Key Research and Development Program of Zhejiang Province (2021C03032).

Author information

Authors and Affiliations

  1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China

    Xiangyang He, Shuoliu Yang, Yubo Tao, Haoran Dai & Hai Lin

Authors
  1. Xiangyang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuoliu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yubo Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haoran Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hai Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yubo Tao or Hai Lin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, X., Yang, S., Tao, Y. et al. Graph convolutional network-based semi-supervised feature classification of volumes. J Vis 25, 379–393 (2022). https://doi.org/10.1007/s12650-021-00787-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-021-00787-7

Keywords

Search

Navigation