Abstract
Medical images, especially volumetric images, are of high resolution and often exceed the capacity of standard desktop GPUs. As a result, most deep learning-based medical image analysis tasks require the input images to be downsampled, often substantially, before these can be fed to a neural network. However, downsampling can lead to a loss of image quality, which is undesirable especially in reconstruction tasks, where the fine geometric details need to be preserved. In this paper, we propose that high-resolution images can be reconstructed in a coarse-to-fine fashion, where a deep learning algorithm is only responsible for generating a coarse representation of the image, which consumes moderate GPU memory. For producing the high-resolution outcome, we propose two novel methods: learned voxel rearrangement of the coarse output and hierarchical image synthesis. Compared to the coarse output, the high-resolution counterpart allows for smooth surface triangulation, which can be 3D-printed in the highest possible quality. Experiments of this paper are carried out on the dataset of AutoImplant 2021 (https://autoimplant2021.grand-challenge.org/), a MICCAI challenge on cranial implant design. The dataset contains high-resolution skulls that can be viewed as 2D manifolds embedded in a 3D space. Codes associated with this study can be accessed at https://github.com/Jianningli/voxel_rearrangement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The test set is further split into \(D_{100}\), which contains 100 defective skulls with similar defect shapes to those of the training set and \(D_{10}\), which includes 10 defective skulls with varied defect patterns.
- 2.
The coarse output (size: \(128\,\times \,128\,\times \,64\)) by the shape completion autoencoder network shown in Fig. 1.
- 3.
For example, in a typical \(512 \times 512 \times 256\) volume containing the skull, there may only be around three million of such voxels, which is, however, still impractical for a linear search, as over three million \(\times \) three million comparisons are needed to update all voxels.
- 4.
Not to be confused with the \(3^3\) and \(5^3\) voxel neighbors in the image pyramid.
- 5.
The visualized implants in these figures, as well as the implants used for calculating the DSCs and HDs (Table 1, Fig. 5) are post-processed using the denoise script from the repository: https://github.com/Jianningli/voxel_rearrangement.
- 6.
The underlying network used to produce the coarse completed skull (\(128 \times 128 \times 64\)) is the same as that of the interpolation and voxel rearrangement-based method.
- 7.
E.g., 12% and 59% CPU memory consumption for the hash table and kd-tree-based method, respectively.
- 8.
The feature space script can be found in the following GitHub repository: https://github.com/Jianningli/voxel_rearrangement.
References
Akil, M., Saouli, R., Kachouri, R., et al.: Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy. Med. Image Anal. 63, 101692 (2020)
Chang, Y.Z., Wu, C.T., Yang, Y.H.: Three-dimensional deep learning to automatically generate cranial implant geometry (2021)
Dai, A., Ruizhongtai Qi, C., Nießner, M.: Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5868–5877 (2017)
Díaz García, J., Brunet Crosa, P., Navazo Álvaro, I., Vázquez Alcocer, P.P.: Downsampling methods for medical datasets. In: Proceedings of the International Conferences Computer Graphics, Visualization, Computer Vision and Image Processing 2017 and Big Data Analytics, Data Mining and Computational Intelligence 2017, Lisbon, Portugal, 21–23 July 2017, pp. 12–20. IADIS Press (2017)
Graham, B., Engelcke, M., Van Der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9224–9232 (2018)
Graham, B., van der Maaten, L.: Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307 (2017)
Han, X., Li, Z., Huang, H., Kalogerakis, E., Yu, Y.: High-resolution shape completion using deep neural networks for global structure and local geometry inference. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 85–93 (2017)
Kodym, O., Španěl, M., Herout, A.: Cranial defect reconstruction using cascaded CNN with alignment. In: Li, J., Egger, J. (eds.) AutoImplant 2020. LNCS, vol. 12439, pp. 56–64. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64327-0_7
Kodym, O., Španěl, M., Herout, A.: Skull shape reconstruction using cascaded convolutional networks. Comput. Biol. Med. 123, 103886 (2020)
Li, J., et al.: Automatic skull defect restoration and cranial implant generation for cranioplasty. Med. Image Anal. 73, 102171 (2021)
Li, J., Egger, J.: Dataset descriptor for the AutoImplant cranial implant design challenge. In: Li, J., Egger, J. (eds.) AutoImplant 2020. LNCS, vol. 12439, pp. 10–15. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64327-0_2
Li, J., Pepe, A., Gsaxner, C., von Campe, G., Egger, J.: A baseline approach for AutoImplant: the MICCAI 2020 cranial implant design challenge. arXiv preprint arXiv:2006.12449 (2020)
Li, J., et al.: AutoImplant 2020-first MICCAI challenge on automatic cranial implant design. IEEE Trans. Med. Imaging 40(9), 2329–2342 (2021)
Riegler, G., Osman Ulusoy, A., Geiger, A.: OctNET: learning deep 3D representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3577–3586 (2017)
Wang, P.S., Liu, Y., Guo, Y.X., Sun, C.Y., Tong, X.: O-CNN: octree-based convolutional neural networks for 3D shape analysis. ACM Trans. Graph. (TOG) 36(4), 1–11 (2017)
Acknowledgement
This work was supported by the following funding agencies:
\(\bullet \) CAMed (COMET K-Project 871132, see also https://www.medunigraz.at/camed/), which is funded by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) and the Austrian Federal Ministry for Digital and Economic Affairs (BMDW), and the Styrian Business Promotion Agency (SFG);
\(\bullet \) The Austrian Science Fund (FWF) KLI 678-B31 (enFaced).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, J., Pepe, A., Gsaxner, C., Jin, Y., Egger, J. (2021). Learning to Rearrange Voxels in Binary Segmentation Masks for Smooth Manifold Triangulation. In: Li, J., Egger, J. (eds) Towards the Automatization of Cranial Implant Design in Cranioplasty II. AutoImplant 2021. Lecture Notes in Computer Science(), vol 13123. Springer, Cham. https://doi.org/10.1007/978-3-030-92652-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-92652-6_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92651-9
Online ISBN: 978-3-030-92652-6
eBook Packages: Computer ScienceComputer Science (R0)
