Abstract
Rock assemblies, such as ballasts and cobbles, are typically composed of granular particles of various sizes and shapes that significantly affect their mechanical behavior. Currently, it remains challenging to generate realistic and identical three-dimensional (3D) models for rock particles with a limited number of their individual two-dimensional (2D) projections. This study proposes a novel and effective deep-learning-enhanced approach to achieve the realistic reconstruction of 3D particle models, which includes the following sequential steps: (1) automatically detecting 2D projected images of each 3D particle individually during dynamic scanning of a rock particle assembly; (2) predicting 3D particle size indexes based on 2D images for each particle; (3) generating a corresponding 3D skeleton network from 2D outlines; and (4) reconstructing the particle surface of individual particle by spatial interpolations. Sensitivity analysis and verification are finally performed on both randomized superball particles and real cobbles/ballasts. The results demonstrated that the proposed approach is robust and efficient in reconstructing 3D rock particle models and offers a rapid and low-cost practical solution for convincible numerical studies of granular materials.
Similar content being viewed by others
Data availability
All data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Alhani IJ, Noor MJBM, Al-Bared MAM, Harahap ISH, Albadri WM (2020) Mechanical response of saturated and unsaturated gravels of different sizes in drained triaxial testing. Acta Geotechnica 15(11):3075
Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. In: 2016 IEEE international conference on image processing (ICIP), Phoenix, USA, 25–28 September, pp 3464–3468
Brechbühler C, Gerig G, Kübler O (1995) Parametrization of closed surfaces for 3D shape description. Comput Vis Image Underst 61(2):154–170
Chen Z, Han Z, Hao J, Zhu Q, Soh YC (2015) Fusion of wifi, smartphone sensors and landmarks using the kalman filter for indoor localization. Sensors 15:715–732
Chen M, Li M, Li Y (2021) Rock particle motion information detection based on video instance segmentation. Sensors 21(12):4108
Cho GC, Dodds J, Santamarina JC (2006) Particle shape effects on packing density, stiffness, and strength: natural and crushed sands. J Geotechn Geoenviron Eng 132(5):591–602
Dan HC, Bai GW, Zhu ZH (2021) Application of deep learning-based image recognition technology to asphalt–aggregate mixtures. Methodol Constr Build Mater 297:123770
Deng J, Guo J, Xue N, Zafeiriou S (2019) Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Long Beach, CA, USA, 16–20 June, pp 4690–4699
Farber L, Tardos G, Michaels JN (2003) Use of X-ray tomography to study the porosity and morphology of granules. Powder Technol 132(1):57–63
Galiyawala H, Raval MS, Dave S (2019) Visual appearance based person retrieval in unconstrained environment videos. Image Vis Comput 92:103816
Ganju E, Kılıç M, Prezzi M, Salgado R, Parab N, Chen W (2021) Effect of particle characteristics on the evolution of particle size, particle morphology, and fabric of sands loaded under uniaxial compression. Acta Geotech 16(11):3489–3516
Jiang H, Bian X, Cheng C, Chen Y, Chen R (2016) Simulating train moving loads in physical model testing of railway infrastructure and its numerical calibration. Acta Geotech 11(2):231–242
Kim Y, Ma J, Lim SY, Song JY, Yun TS (2022) Determination of shape parameters of sands: a deep learning approach. Acta Geotech 17:1–11
Kuo CY, Frost JD, Lai JS, Wang LB (1996) Three-dimensional image analysis of aggregate particles from orthogonal projections. Transp Res Rec 1526(1):98–103
Lai Z, Chen Q (2019) Reconstructing granular particles from X-ray computed tomography using the TWS machine learning tool and the level set method. Acta Geotech 14(1):1–18
Liang Z, Nie Z, An A, Gong J, Wang X (2019) A particle shape extraction and evaluation method using a deep convolutional neural network and digital image processing. Powder Technol 353:156–170
List J, Köhler U, Witt W (2011) Dynamic image analysis extended to fine and coarse particles. Part Syst Anal 2011:1–5
Lu M, McDowell GR (2007) The importance of modelling ballast particle shape in the discrete element method. Gr Matter 9(1–2):69
Mahalanobis PC (1936) On the generalized distance in statistics. National Institute of Science of India
Mahawish A, Bouazza A, Gates WP (2018) Effect of particle size distribution on the bio-cementation of coarse aggregates. Acta Geotech 13(4):1019–1025
Marsaglia G (1972) Choosing a point from the surface of a sphere. Ann Math Stat 43(2):645–646
Nguyen H, Bui X-N, Tran Q-H, Nguyen D-A, Hoa LTT, Le Q-T et al (2021) Predicting blast-induced ground vibration in open-pit mines using different nature-inspired optimization algorithms and deep neural network. Nat Resour Res. https://doi.org/10.1007/s11053-021-09896-4
Nie Z, Liang Z, Wang X (2018) A three-dimensional particle roundness evaluation method. Gr Matter 20(2):32
Nurzynska K, Iwaszenko S (2020) Application of texture features and machine learning methods to grain segmentation in rock material images. Image Anal Stereol 39(2):73–90
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Chintala S (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:1
Rahman MM, Dafalias YF (2021) Modelling undrained behaviour of sand with fines and fabric anisotropy. Acta Geotech 17:2305
Rao C, Tutumluer E, Kim IT (2002) Quantification of coarse aggregate angularity based on image analysis. Transp Res Rec 1787(1):117–124
Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1):157–173
Su D, Wang X, Yang H, Hong C (2019) Roughness analysis of general-shape particles, from 2D closed outlines to 3D closed surfaces. Powder Technol 356:423
Su D, Yan WM (2018) 3D characterization of general-shape sand particles using microfocus X-ray computed tomography and spherical harmonic functions, and particle regeneration using multivariate random vector. Powder Technol 323:8–23
Su D, Yan WM (2019) Prediction of 3D size and shape descriptors of irregular granular particles from projected 2D images. Acta Geotech 15:1–23
Sun Z, Wang C, Hao X, Li W, Zhang X (2020) Quantitative evaluation for shape characteristics of aggregate particles based on 3D point cloud data. Constr Build Mater 263:120156
Sun Q, Zheng Y, Li B, Zheng J, Wang Z (2019) Three-dimensional particle size and shape characterisation using structural light. Géotech Lett 9(1):72–78
Ueda T, Oki T, Koyanaka S (2019) 2D–3D conversion method for assessment of multiple characteristics of particle shape and size. Powder Technol 343:287–295
Wang X, Gong J, An A, Zhang K, Nie Z (2019) Random generation of convex granule packing based on weighted Voronoi tessellation and cubic-polynomial-curve fitting. Comput Geotech 113:103088
Wang ZY, Gu DM, Zhang WG (2020) Influence of excavation schemes on slope stability: a DEM study. J Mt Sci 17(6):1509–1522
Wang X, Tian K, Su D, Zhao J (2019) Superellipsoid-based study on reproducing 3D particle geometry from 2D projections. Comput Geotech 114:103–131
Wang Z, Wang L, Zhang W (2019) A random angular bend algorithm for two-dimensional discrete modeling of granular materials. Materials 12(13):2169
Wang X, Yin ZY, Su D, Wu X, Zhao JD (2021) A novel approach of random packing generation of complex-shaped 3D particles with controllable sizes and shapes. Acta Geotechnica 17:1–22
Wang X, Yin ZY, Su D, Xiong H, Feng YT (2021) A novel Arcs-based discrete element modeling of arbitrary convex and concave 2D particles. Comput Methods Appl Mech Eng 386:114071
Wang X, Yin ZY, Xiong H, Su D, Feng YT (2021) A spherical-harmonic-based approach to discrete element modeling of 3D irregular particles. Int J Numer Methods Eng 122(20):5626–5655
Williams JR, Pentland AP (1992) Superquadrics and modal dynamics for discrete elements in interactive design. Eng Comput 9(2):115–127
Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP), Beijing, China, 17–20 Sep, pp 3645–3649
Xu YR, Xu Y (2021) Numerical simulation of direct shear test of rockfill based on particle breaking. Acta Geotechnica 16:1–12
Yamamoto KI, Inoue T, Miyajima T, Doyama T, Sugimoto M (2002) Measurement and evaluation of three-dimensional particle shape under constant particle orientation with a tri-axial viewer. Adv Powder Technol 13(2):181–200
Yan WM, Su D (2018) Inferring 3D particle size and shape characteristics from projected 2D images: lessons learned from ellipsoids. Comput Geotech 104:281–287
Yan WM, Su D (2018) Evaluation of three-dimensional particle shape index from projected two-dimensional image. Géotech Lett 8(4):336–343
Yang HW, Lourenço SD, Baudet BA, Choi CE, Ng CW (2019) 3D Analysis of gravel surface texture. Powder Technol 346:414–424
Yang L, Nguyen H, Bui X-N, Nguyen-Thoi T, Zhou J, Huang J (2021) Prediction of gas yield generated by energy recovery from municipal solid waste using deep neural network and moth-flame optimization algorithm. J Clean Prod 311:127672
Yang L, Fan Y, Xu N (2019) Video instance segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5188–5197
Zhang H, Nguyen H, Bui X-N, Nguyen-Thoi T, Bui T-T, Nguyen N et al (2020) Developing a novel artificial intelligence model to estimate the capital cost of mining projects using deep neural network-based ant colony optimization algorithm. Resour Policy 66:101604
Zhang H, Nguyen H, Bui X-N, Pradhan B, Asteris PG, Costache R et al (2021) A generalized artificial intelligence model for estimating the friction angle of clays in evaluating slope stability using a deep neural network and Harris Hawks optimization algorithm. Eng Comput. https://doi.org/10.1007/s00366-020-01272-9
Zhao S, Zhou X (2017) Effects of particle asphericity on the macro-and micro-mechanical behaviors of granular assemblies. Gr Matter 19(2):38
Zhou W, Liu J, Ma G, Chang X (2017) Three-dimensional DEM investigation of critical state and dilatancy behaviors of granular materials. Acta Geotech 12(3):527–540
Zhou B, Wang J (2017) Generation of a realistic 3D sand assembly using X-ray micro-computed tomography and spherical harmonic-based principal component analysis. Int J Numer Anal Methods Geomech 41(1):93–109
Zhou B, Wang J, Zhao B (2015) Micromorphology characterization and reconstruction of sand particles using micro X-ray tomography and spherical harmonics. Eng Geol 184:126–137
Zou Y, Ma G, Mei J, Zhao J, Zhou W (2021) Microscopic origin of shape-dependent shear strength of granular materials: a granular dynamics perspective. Acta Geotechnica. https://doi.org/10.1007/s11440-021-01403-6
Acknowledgements
This research was financially supported by the Research Grants Council (RGC) of Hong Kong Special Administrative Region Government (HKSARG) of China (Grant Nos. 15220221), Natural Science Foundation of Shenzhen, China (No. JCYJ20210324094607020) and the National Natural Science Foundation of China (Grant No. 51878416).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
To generate the 3D model for a superball particle, the polar representation is used in this study. Given the local spherical coordinate \(\left(\theta ,\varphi \right)\) of a point on the surface, the corresponding local Cartesian coordinate \(\left(x,y,z\right)\) is expressed as:
where term \(\mathrm{Sign}\left(x\right)\) is the signum function defined as:
Appendix 2
The \(\theta\) and \(\varphi\) for a random unit vector in 3D space can be easily generated through the following steps:
-
Step 1: Generate a pair of random variables \({\xi }_{1}\) and \({\xi }_{2}\) that are uniformly distributed on [− 1,1].
-
Step 2: Calculate \({r}^{2}\) by Eq. (47). If \({r}^{2}<1\), abandon this pair and repeat step 1.
$${r}^{2}={{\xi }_{1}}^{2}+{{\xi }_{2}}^{2}$$(47) -
Step 3: Calculate \({n}_{x},{n}_{y},{n}_{z}\) by the following expression
$$\left\{ {\begin{array}{*{20}l} {n_{x} = 2\xi _{1} \sqrt[2]{{1 - r^{2} }}} \hfill \\ {n_{y} = 2\xi _{2} \sqrt[2]{{1 - r^{2} }}} \hfill \\ {n_{x} = 1 - r^{2} } \hfill \\ \end{array} } \right.$$(48)
Appendix 3
The process to search all the surface points on the 3D particle that corresponds to the projected points in plane M is implemented based on the following steps:
-
1.
Randomly select one point \({Q}_{1}\) from plane M such that vector \(\mathop \rightharpoonup \limits_{{OQ_{1} }} \left({x}_{q1}, {y}_{q1},{z}_{q1}\right)\) is perpendicular to the projection vector \(\mathbf{O}\mathbf{V}\), as shown in Fig. 6b.
-
2.
Find the surface point \({P}_{1}\), as shown in Fig. 6c that has a normal vector \(\mathop \rightharpoonup \limits_{{N_{1} }}\) parallel to vector \(\mathop \rightharpoonup \limits_{{OQ_{1} }}\) as follows: Given a normal vector \(\mathop \rightharpoonup \limits_{{N_{1} }}\left({n}_{x},{n}_{y},{n}_{z}\right)\) on the surface, the corresponding local spherical coordinates \(\left(\theta ,\varphi \right)\) of \({P}_{1}\) are obtained using the following equations:
$$\theta =\mathrm{atan}2\left[{\mathrm{Sign}\left({n}_{y}\right)\left|b{n}_{y}\right|}^{\frac{1}{2-{\varepsilon }_{1}}},{\mathrm{Sign}\left({n}_{x}\right)\left|a{n}_{x}\right|}^{\frac{1}{2-{\varepsilon }_{1}}}\right]$$(49)$$\varphi =\mathrm{atan}2\left[{\mathrm{Sign}\left({n}_{z}\right)\left|c{n}_{z}\right|\mathrm{cos}\theta }^{\left|2-{\varepsilon }_{2}\right|\frac{1}{2-{\varepsilon }_{2}}},{\left|a{n}_{x}\right|}^{\frac{1}{2-{\varepsilon }_{2}}}\right]$$(50)where the term \(\mathrm{atan}2\left(x,y\right)\) is the arctangent function of \(\frac{x}{y}\) producing results in the range \((-\uppi ,\uppi ]\). After the local spherical coordinates \(\left(\theta ,\varphi \right)\) of \({P}_{1}\) are obtained, the corresponding local Cartesian coordinates \(\left(x,y,z\right)\) of \({P}_{1}\) can be computed using Eq. (45).
-
3.
Rotate \(\mathop \rightharpoonup \limits_{{OQ_{1} }}\) at a discretized angle \(\Delta\updelta =\uppi /\mathrm{m}\) around axis \(\mathbf{O}\mathbf{V}\) to obtain a new vector \(\mathop \rightharpoonup \limits_{{OQ_{i}^{\prime } }}\) and find the corresponding surface point \({P}_{i}\) with \(\mathop \rightharpoonup \limits_{{N_{i} }} \left(\updelta \right)\) parallel to \(\mathop \rightharpoonup \limits_{{OQ_{i}^{\prime } }}\);
-
4.
Repeat (3) until \(\updelta =2\uppi\), and connect all acquired points, in turn, to form the full projected outline, as shown in Fig. 6d.
Rights and permissions
About this article
Cite this article
Wang, X., Zhang, H., Yin, ZY. et al. Deep-learning-enhanced model reconstruction of realistic 3D rock particles by intelligent video tracking of 2D random particle projections. Acta Geotech. 18, 1407–1430 (2023). https://doi.org/10.1007/s11440-022-01616-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11440-022-01616-3