Advertisement

Automatic Crack Detection and Analysis for Biological Cellular Materials in X-Ray In Situ Tomography Measurements

  • Ziling Wu
  • Ting Yang
  • Zhifei Deng
  • Baokun Huang
  • Han Liu
  • Yu Wang
  • Yuan Chen
  • Mary Caswell Stoddard
  • Ling LiEmail author
  • Yunhui ZhuEmail author
Open Access
Thematic Section: 3D Materials Science
  • 154 Downloads
Part of the following topical collections:
  1. 3D Materials Science 2019

Abstract

We introduce a novel methodology, based on in situ X-ray tomography measurements, to quantify and analyze 3D crack morphologies in biological cellular materials during damage process. Damage characterization in cellular materials is challenging due to the difficulty of identifying and registering cracks from the complicated 3D network structure. In this paper, we develop a pipeline of computer vision algorithms to extract crack patterns from a large volumetric dataset of in situ X-ray tomography measurement obtained during a compression test. Based on a hybrid approach using both model-based feature filtering and data-driven machine learning, the proposed method shows high efficiency and accuracy in identifying the crack pattern from the complex cellular structures and tomography reconstruction artifacts. The identified cracks are registered as 3D tilted planes, where 3D morphology descriptors including crack location, crack opening width, and crack plane orientation are registered to provide quantitative data for future mechanical analysis. This method is applied to two different biological materials with different levels of porosity, i.e., sea urchin (Heterocentrotus mamillatus) spines and emu (Dromaius novaehollandiae) eggshells. The results are verified by experienced human image readers. The methodology presented in this paper can be utilized for crack analysis in many other cellular solids, including both synthetic and natural materials.

Keywords

Cellular material Crack detection Computer vision X-ray tomography Machine learning 

Introduction

Cellular structures or foams are widely found in natural material systems with a variety of microstructure types, including honeycombs (e.g., cork), closed-cell foams (e.g., porcupine quill and plant parenchyma), and open-cell foams (e.g., trabecular bone and sea urchins spines) [1, 2]. These natural cellular materials usually function as mechanical support—a bony skeleton must support body weight and prevent external mechanical damage, for instance. Compared to the fully organic (e.g., wood, cork, porcupine quill, etc.) and partially mineralized (e.g., trabecular bone) biological cellular materials, the porous microstructure of echinoderm skeletal elements represents a unique class of biological ceramic cellular materials due to its high mineral density (> 98.4 wt%) [3] and excellent damage tolerance [4, 5, 6]. This is in stark contrast to many synthetic ceramic cellular solids, which often exhibit immediate structural degradation when loading exceeds the structure’s strength, leading to limited energy absorption capabilities [7, 8, 9].

Echinoderms (including sea stars, sea urchins, sand dollars, brittle stars, etc.) are a group of marine invertebrates (phylum Echinodermata) characterized by their highly organized and protective mineralized skeletons [10, 11, 12, 13]. For example, the spines of sea urchins provide protection from the impact, wear, and fracture resulting from hydrodynamic forces of waves (many sea urchins live in the intertidal zone) as well as from predators [14]. The echinoderm skeletons are among the most amazing biomineralized structures in nature: a complex bicontinuous porous structure with a controlled gradient in porosity and structural parameters [13, 15, 16]. Despite their single-crystal nature based on inherently weak, brittle magnesium calcite [15, 17], echinoderm skeletons exhibit high strength, and, more importantly, graceful failure through the so-called conchoidal fracture, where cracks are believed to be scattered by the complex porous structure to prevent catastrophic failure [4, 5, 6].

Unlike polymeric and metallic foams, crack formation and propagation are the primary deformation mode for ceramic foams. Therefore, precise detection of crack generation and quantitative analysis of their characteristics, such as sizes, crack opening (and related crack tip opening displacement), and propagation direction, are critical for understanding deformation behavior and the entire damage process of these materials. In this regard, in situ X-ray tomography is an excellent characterization platform in comparison to previous studies that are primarily based on postmortem analysis. In situ X-ray tomography has recently been widely utilized to investigate the mechanical behavior of a variety of synthetic structural materials [18, 19], including cellular solids. However, few studies have used in situ X-ray tomography to investigate the deformation and fracture mechanisms of biological cellular structures, particularly echinoderm porous skeletons. Traditionally, 2D damage characterization is implemented by observing and tracking the propagation of cracks from 2D images of sample surfaces. This process is usually conducted manually by expert human image readers, and the samples are often limited to be solid materials with a few cracks. It is very challenging to apply the manual 2D analysis methods for investigating crack characteristics in ceramic cellular solids, where a high-density cracks of various sizes form and propagate simultaneously inside the complex 3D porous structure. We need an efficient crack detection algorithm to account for the complexity and 3D nature of bio-cellular material systems.

In the past few years, several studies have attempted to identify damage using computer vision approaches to replace the slow and subjective manual inspection procedures for fast and reliable defect analysis. Many computer vision algorithms have been proposed, including thresholding [18, 19], segmentation [20], edge detector [21, 22], and filter-based algorithms [23]. These have been used to detect damage on pavement surfaces [20, 21, 22, 23, 24, 25], bridge surfaces [26], wood samples [27, 28], and steel [29]. These automated algorithms, while being significantly faster than manual inspection, have been mainly developed under specific hypotheses for 2D images; they cannot be generalized to 3D applications with various environmental background, irregularly illuminated conditions, shading, and blemishes. Recently, advanced machine learning techniques have been developed to solve the image classification [30], reconstruction [31, 32, 33], and object detection [34, 35] problems by learning from the database adaptively and fine-tuning on the basis of exhaustive examples. A successful example has been the development of R-CNN method by Girshick et al [34]. This method identifies a targeted objects by proposing the regions of interest and classifying them via a convolutional neural network (CNN) of feature vectors. The dual-step approach has been successfully applied to face recognition [35, 36], animal identification [37], and car localization [38, 39, 40]. The promising results have motivated the application of deep learning techniques to damage detection. For example, Cha et al. [41] use a sliding window to divide the image into blocks and apply CNN to predict the existence of cracks in pavement damage. Zhang et al. [42] use CNN to identify pixels in the pavement image that belongs to a crack, based on local patch information. Later, Li et al. [43] developed an algorithm similar to R-CNN on electron microscopic images to study irradiation damage of metal alloys, which uses object detection to propose a damage region first and then categorizes damage types with a CNN classifier. However, these learning-based methods have been mainly applied to 2D images, where cracks are sparsely located on a solid background with minor noise or texture. In the context of cellular materials with complex 3D structures as the background, these damage detection algorithms are no longer suitable. To solve the problem, we need to first recognize the crack patterns against a strong and structured background that cannot be easily erased or filtered out. Second, we need to implement the analysis in 3D to include the 3D morphology features and damage distribution. For instance, to understand how the 3D porous network of sea urchin spines regulates the orientation of the crack planes and the propagation of crack to prevent catastrophic failure [4, 5, 6], we must register the position and orientation of the cracks in reference to the hosting branches on the cellular networks. Such 3D information is critical for understanding the structure–mechanical property relationship of this class of materials, which may provide guidelines for the design of bio-inspired low-density materials.

In this paper, we report a 3D automatic crack detection and analysis methodology based on in situ synchrotron-based X-ray micro-computed tomography (µ-CT), which identifies, registers, and analyzes cracks in 3D in complicated biological ceramic cellular solids. This computer vision-based pipeline for 3D crack analysis consists of four steps, including (1) µ-CT imaging reconstruction and enhancement, (2) the initial crack extraction based on a feature map library, (3) crack identification based on supervised machine learning, and (4) 3D crack registration and analysis. The process presented here is fast, only requiring approximately 20 min on a GPU (graphics processing unit)-based laptop computer to detect and register all cracks over a field of view of 1 mm3, with over 200 mega voxels in the volume. We validate this approach on two types of biological materials with different levels of porosity (sea urchin spines and emu eggshells), and we compare the end product of 3D crack visualization with human labeling. Results of the tested samples demonstrate successful elimination of the background structure and the CT reconstruction artifacts. Our method generates comparable or better results than that of human readers.

Methods

The pipeline for 3D crack detection, registration, and analysis is schematically described in Fig. 1 with four steps of (I) CT data acquisition and reconstruction, (II) feature-based crack detection, (III) machine learning refinement, and (IV) 3D crack registration and analysis. The following sections describe the 4-step process.
Fig. 1

Pipeline of crack characterization, including four steps of CT data acquisition (I), crack feature detection (II), refinement via machine learning (III), and 3D crack registration (IV)

Step I: CT Data Acquisition and Reconstruction

The in situ mechanical measurement in compression mode on the biological material samples was conducted at beamline 2-BM at the Advanced Photon Source at Argonne National Laboratory. The biological cellular material models used in this work include a sea urchin spine from Heterocentrotus mamillatus and the shell fragments from commercially sourced emu eggs (Dromaius novaehollandiae). Sea urchin spine is a representative porous biological material system with a porosity of ~ 60–80%. Such cellular structures are widely found in nature. It is reported to be simultaneously lightweight, strong and damage-tolerant. In the CT scans, X-ray resolves the pores and cracks with the same intensity, making it difficult to distinguish cracks from the background via traditional threshold-based segmentation methods. Emu eggshell is a structural composite with an upper porous layer and inner dense layer. The complexity of crack detection originates from the lack of contrast for small initial cracks (microvoids), which are generated in large numbers during the crack initiation and early crack propagation stages. These sample systems are chosen to demonstrate the efficiency of detecting cracks of various sizes and contrast in cellular material systems.

The in situ experimental setup used in this work is shown in Fig. 2a. The samples were cut in cube shapes and placed between compression platens, where the top compression platen was driven by the linear actuator on the top of the device to apply the compression force in the \( z \) direction. A minimum displacement rate of 0.008 mm/s was used in this work. A time-lapse measurement was adopted, where the actuator was stopped multiple times during the compression test for the tomographic scan. We implement the damage detection in datasets at early stages to focus on crack initiation and early propagation, which are more significant to the materials’ mechanical properties and are more technically challenging. Figure 2b and c shows two typical projection images of the sample at different stages of compression. Tomography scanning was conducted by rotating the specimen 180° (rotation speed, 0.5°/s), where 1500 projections with an exposure time of 0.24 s each were acquired. The energy of the X-ray beam used in the µ-CT measurement was 27.4 keV. A PCO Edge high-speed camera equipped with a 5 × objective lens was used to collect the projection images and the resolution of the reconstructed volumetric data is (1.3 µm)3/voxel. The dimensions of samples used in the in situ tests were ~ 2.2 mm \( \times \) 2.2 mm \( \times \) 1.7 mm for sea urchin spines and ~ 2.0 mm \( \times \) 2.0 mm \( \times \) 1.5 mm for emu eggshells.
Fig. 2

Synchrotron-based in situ X-ray tomography measurement. a Experiment setup for conducting in situ compression tests. The samples are placed in between two compression platens and compressed by controlling the actuator on top. Side view projection images of a tested sea urchin spine sample before (b) and during compression (c). d A representative horizontal slice of reconstructed tomography image of the same sample. The yellow line in c indicates the location of the horizontal slice

The open software TomoPy [44] was utilized for the image reconstruction. Several de-noising algorithms implemented in TomoPy were applied to suppress noise and artifacts of reconstruction, including smoothing filtering of the sinogram to correct for bad pixels on the detector, back-projection filtering to suppress high-frequency artifacts, and phase diffraction correction based on transport of intensity equation [44]. After the reconstruction, a 3D volumetric dataset containing 2560 \( \times \) 2560 \( \times \) 1280 voxels was generated for each tomography scan. Figure 2d shows a typical reconstruction slice for the sample of sea urchin spines. In this study, representative volumes from the central regions of the samples (1880 \( \times \) 1620 \( \times \) 300 voxels) at compression strains of 18% and 22.8% for the sea urchin spine and the emu eggshell, respectively, were extracted and used for the development of crack detection and analysis algorithms.

Step II: Feature-Based Crack Detection

The automatic crack detection approach defines the crack patterns with the feature map library, which describes the crack locally as a high contrast notch representing a narrow gap. The profile of a notch pattern in a local patch in the \( xy \) plane is defined as a rectangular function via
$$ f\left( {x,y} \right) = \left\{ {\begin{array}{ll} {0,} & {{\text{if}} \left( { - \frac{w}{2} \le x\sin \theta + y\cos \theta \le \frac{w}{2} - 1} \right) \cap \left( { - \frac{L}{2} \le x\cos \theta + y\sin \theta \le \frac{L}{2} - 1} \right)} \\ {1,} & {\text{otherwise}} \\ \end{array} } \right. $$
(1)
where w is the width of the crack opening, \( \theta \) is the tilting angle of the crack, and \( L \) is the length of the crack. In biological cellular materials, cracks form with various sizes and lengths. We establish the crack library only for short local crack patterns in a small patch area of \( L \times L \) pixels and identify the extended cracks as a series of smaller local notches aligned and attached together. This localized method registers all types of cracks and their possible wavy shapes based on the reduced crack pattern library, which substantially reduces the computational cost of detecting large numbers of small and twisty cracks in the cellular materials.

We build the local crack feature libraries considering all possible orientations, widths, and lengths in the small patch of size \( L \times L \), based on the observation of crack patterns from the databases and resolution limits. For the sea urchin spine samples, a total of 432 notch patterns are included with 36 orientations across the 180° span at 5° intervals, 4 different widths from one to four pixels incremented by one-pixel each, and 3 different aspect ratios. For the emu eggshell samples, which demonstrate larger width spread, a total of 648 notch patterns are included with 36 orientations across the 180° at 5° intervals, 6 different widths from one to six pixels incremented by one-pixel each, and 3 different aspect ratios. Several notch pattern examples in the crack library are shown in Fig. 1 for Step II.

We then correlate the CT reconstructed slices with the crack library in \( xy \), \( yz \), \( xz \) planes. The 432 feature maps for sea urchin spine sample and 648 feature maps for the emu eggshell sample are generated by scanning the notch patterns as correlation filters across the CT reconstructed slices with normalized cross-correlation. We evaluate the local patch of each pixel to see if it matches any of the 432/648 crack patterns. An initial crack filtering is then implemented by thresholding with the maximum correlation value for each pixel. The highly correlated locations indicate a high probability of finding a crack, and the highly correlated filter at this location reveals the shape of the potential crack.

Step III: Refinement of Crack Detection Based on Supervised Machine Learning

Step II is a model-based 2D crack detection method based on feature filtering. The proposed crack candidates from Step II are refined in the machine learning Step III to enhance the accuracy of detection based on nonstandard features beyond the feature library. In this step, we calibrate the crack detection from manually labeled crack data, which is used to train a convolutional neural network (CNN) classifier. The combination of the model-based pre-filtering step and the data-driven refinement step proves to provide high accuracy with minimum computational cost and a small dataset size, as compared to model-only or data-only approaches.

The feature-based filtering in Step II is based on 2D images and prone to 2D reconstruction artifacts and noises. These artifacts and noises, however, are uncorrelated across different reconstruction slices, while the crack formation is continuous in the 3D volume. Combining adjacent slices along the \( z \) axis can, therefore, provide additional structural information that helps the CNN to distinguish the reconstruction artifacts from the actual structure and cracking. Our CNN classifier extend the 2D input to semi-3D by stacking the adjacent three slices around the proposed 2D cracks. Isolated crack notches without extension to the neighboring slices were deleted. This 3D continuity validation is shown to efficiently remove false detections efficiently from the artifacts of the CT reconstruction.

Data Preparation

The CNN classifiers in Step III were trained on two databases, the sea urchin spine and emu eggshell, respectively. For sea urchin spine samples, we cropped two different regions without overlapping for training and testing. The first region contains 201 slices with the image size of \( 396 \times 438 \) and the second one contains 181 slices with the image size of \( 560 \times 312 \). The emu eggshell database contains 100 slices with the size of \( 706 \times 575 \). The first 10 slices were used for testing, and the last 80 slices were used for training. Each testing slice was annotated and checked by three experienced annotators manually. The input of the network contains patches of manually identified cracks with the size of \( 32 \times 32 \times 3 \) voxels. We set the crack/non-crack ratio in the training data as 1:3, as suggested in Ref. [45]. A total of 40,000 image patches were used for the sea urchin sample and 20,000 patches for the emu eggshell sample.

Network Architecture and Implementation

As illustrated in Fig. 1, we used a simple CNN network with three convolutional (conv) layers, four Relu (rectified linear unit) layers and three max-pooling layers, two fully connected (FC) layers, and one softmax layer. Extracted image patches with the sizes of 32 \( \times \) 32 \( \times \) 3 with zero center normalization were used as the input of the network, followed by three blocks of Conv–Relu–Maxpolling layers. The convolutional layers were equipped with a kernel of 3 × 3 and a stride of 1, whose effectiveness has been verified in VGG-Net. No padding was used since each crack patch is centered at the input. The number of kernels was conventionally set as 32, 64, and 128 filters, respectively. The number of Conv–Relu–Maxpolling blocks was chosen to be 3 to adapt to the complexity of the classification problem. All convolution layers were constructed with a stride of 1 and padded by 2. The Relu function was applied to provide nonlinearity after the convolution layer, and we used a max-pooling layer with the filter size of 3 \( \times \) 3, stride [2, 2] and no puffing after the Relu layer. After three-block operations, we applied a fully connected layer of 64 nodes followed by a Relu layer and another fully connected layer of 2 nodes as the classification layer to detect crack and non-crack. The softmax function was used as the final output layer to give the output category for each input image.

The training dataset was split into training and validation sets with the ratio of 9:1 to prevent over-fitting. We implemented the training on MATLAB 2018b with the Deep Learning Toolbox. The machine used for our experiments was a PC with Intel Core i7-6700 K 4.0-GHz CPU, 32-GB RAM, GeForce GTX 960 2 GB GPU. All the parameters in this neural network were initialized using a Glorot initializer, which independently samples from a uniform distribution with zero mean and variance 2/(\( n_{\text{in}} \) + \( n_{\text{out}} \)), where \( n_{\text{in}} \) and \( n_{\text{out}} \) indicate the number of input and output units in each layer. Data were fed into the CNN in a mini-batch size of 32. The cross-entropy loss for binary classification was optimized with ADAM algorithm with \( L_{2} \) regularization set to 0.01. The initial learning rate was 0.001 with a learning rate drop factor of 0.01 every 10 epochs. The maximum number of epochs was set as 30. The dropout method was used after the first fully connected layers to reduce over-fitting by preventing complex coadaptions on the training data. The validation dataset was used to monitor the network quality during training and provide the stop criterion. In all experiments, the \( l_{1} \)-norm over the validation dataset, i.e., the validation error, was computed every 100 gradient steps. The training was stopped once no improvement in the validation error was found for 10,000 gradient steps.

Step IV: 3D Crack Registration

The refined crack results detected in Step III are then registered as 3D objects by combining with the shape information identified in Step II. In this step, we used cluster analysis to combine spatially connected 2D cracks as one 3D crack. Each of the crack clusters was then registered as a 3D object and fitted into a 3D plane with a principal component analysis (PCA) algorithm. The principal component with the least variation is identified as the crack plane orientation. By projecting every surface point of one 3D crack, the average projection distance is defined as crack opening width and the projection area is defined as crack plane area. We registered the center location, crack opening width, crack plane orientation, and crack plane area. The end result was a set of 3D registered crack planes throughout the volume.

Results and Discussion

In this study, we selected two model natural ceramic materials with different porosity levels and morphologies, sea urchin spines and emu eggshells, to demonstrate the proposed methodology for crack detection analysis by using the micro-X-ray computed tomography coupled with in situ compression tests.

Structures of Model Material Systems and Observed Damage Behavior

Figure 3a shows a typical cellular structure of the sea urchin spines from H. mamillatus, with its characteristic bicontinuous network, also known as stereom, and highly curved structures. A high-magnification scanning electron micrograph in Fig. 3b shows the classical non-cleavage fractured surface, although sea urchin spines are known to consist of single crystals [15, 17]. The fracture planes do not coincide with a specific internal feature, such as branch connection points or the middle of branches. Some cracks even propagate along the longitudinal direction of branches, resulting in a large fracture plane compared to the transverse fracture of individual branches. This so-called conchoidal fracture is believed to contribute to the graceful failure of the biomineralized cellular structure [5, 6, 45]. Although these 2D imaging approaches are important for examining the morphology of the fracture surfaces, their 3D information, as referenced in the underlying cellular network, cannot be obtained with this method.
Fig. 3

Cellular structures in sea urchin spines (Heterocentrotus mamillatus) as revealed by scanning electron micrographs at different magnifications (a, b) and µ-CT (c)

Similar to other eggshells, the main structural component in emu eggshells is a calcified layer consisting of vertical columnar crystals, where isolated microscopic pores and vertical pore canals are present (Fig. 4a). We utilize this shell structure to further evaluate our crack detection and analysis algorithm. As shown in Fig. 4b–e, upon compression the eggshell developed both several primary cracks as well as a large number of secondary cracks. The cracks are affected by the presence of microscopic pores and pore canals.
Fig. 4

Microstructure of emu eggshell and representative in situ compression test results. a A reconstruction slice of the original emu eggshell structure, indicating the presence of microscopic pores and pore canals. The location of this region is shown in c. b, d Projection images of an emu eggshell specimen before and during a compression test, and (c, e) corresponding horizontal reconstruction slices. The yellow dashed lines in b and d indicate the locations of the reconstruction slices in c and e

Crack Detection Results

We first show the crack detection in 2D slices for the tested samples. The different microstructure configurations and chemical compositions lead to differences in crack morphology and distribution, ranging from sparse and scattered small cracks in our sea urchin spines, to dense and distributed cracks in the emu eggshells. The material differences also produce different artifacts and noises in the CT reconstruction. Our approach succeeds in detecting cracks for both materials, as shown in Figs. 5 and 6.
Fig. 5

2D crack detection for sea urchin spine. a Crack detection from expert human readers with cross-validations, considered as the ground truth. b High-risk potential crack detection results from feature filtering in Step II, which identified all 14 cracks among many other candidates. c Crack detection results by the CNN classifier in Step III alone, with significant false detections, d Crack detection results by refining filtering results with the CNN classifier, showing improved accuracy and e Crack detection results by refining filtering results with the CNN classifier and 3D correlation in step III, achieving a high degree of accuracy with one missed detection (green box), and one false detection (yellow box) out of 14 cracks

Fig. 6

2D and 3D crack detection for emu eggshell. a Input 2D slice of the original CT reconstruction b Highlighted 2D crack detection. c Input 3D volume of the original CT reconstruction. d Detected 3D cracks in this volume

Figure 5 shows the pipeline of crack detection for one 2D slice in the testing dataset in our sea urchin spine sample. Ground truth based on manual labeling is shown in Fig. 5a. We first use the feature map filtering in Step II to list the high-risk targets. These potential crack locations are shown in Fig. 5b with large quantities of false-positive (incorrectly classified cracks), as expected. Most of the false-positive cracks are misclassified background reconstruction artifacts: in particular, the streak effect from CT reconstruction. There are also falsely identified cracks at the boundary of the cellular structures. These proposed crack candidates are then refined with the 3D classifier trained in Step III. Three results of Step III are shown for comparison. Figure 5c shows the detection results solely based on the CNN classifier without the initial filtering in Step II. The performance is significantly improved as shown in Fig. 5d by integrating Step II with a 2D CNN classifier trained without adjacent patches. Finally, the integration of filtering with the 3D classifier yields the best results shown in Fig. 5e, showing a high degree of correlation with the ground truth. The accuracy is evaluated by the \( F_{1} \) score, defined as an average of precision and recall. The definitions of the precision and recall are
$$ P = \frac{\text{True positive}}{{{\text{True positive}} + {\text{False positive}}}} $$
$$ R = \frac{\text{True positive}}{{{\text{True positive}} + {\text{False negative}}}} $$
and the \( F_{1} \) score is expressed as
$$ F_{1} = \frac{2PR}{P + R}. $$
The accuracy of the result is indicated by a high \( F_{1} \) score, which is based on both good precision and good recall. For example, the initial filtering in Step II achieved \( R = 1 \) by finding all cracks, but resulted in a low \( P \) value of 0.0700 due to the many false-positives, leading to a low \( F_{1} \) score of 0.1308. The performance scores of results in Fig. 5 are given in Table 1. The \( F_{1} \) score of our final results in Fig. 5e is 0.9286, based on one false-positive (highlighted in the yellow box) and one false-negative (highlighted in the green box) out of 14 cracks. This crack detection results on one slice took 5 s to generate on a GPU-based laptop computer. As a comparison, it took around 15 min for the human image readers to generate the ground truth in Fig. 5a. The results show that the hybrid approach provides good performance while minimizing computational time and memory expense. The 2D feature library provides fast filtering in the complex background structure and noisy reconstruction, which selects and specifies just a few high-risk locations for the CNN classifier to process. At the same time, the threshold of the filtering is selected to ensure the inclusion of all cracks. The initial filtering also reduces the diversity of the input image patches, which is expected to contribute to the success of the CNN classifier with a relatively small number of layers.
Table 1

Quantitative metrics of the proposed workflow

Metrics\method

Step II

CNN classifier

Step II + CNN classifier

Step II + CNN classifier + 3D correlation

F1

0.1308

0.5000

0.6667

0.9286

The methodology was further tested with another biological ceramic material: emu eggshells. As shown in Fig. 6, the resultant crack patterns are successfully captured by our method, despite the significant structural difference between sea urchin spines and emu eggshells. Cracks of different sizes and shapes are identified against the reconstruction artifacts and noises. Figure 6d shows the 3D crack detection results by stacking 2D crack maps.

Figure 7 shows the 3D visualization of cracks detected from Step III for the sea urchin spine sample in the 3D volumetric dataset. Scattered cracks are observed throughout the volume. We observe a major crack concentration band where nearby cracks are roughly aligned. This result is compared with human labeling shown in Fig. 7c, which took around 3 h to generate for this volume. A high correlation is observed between the automatic detection and human reading.
Fig. 7

3D crack visualization in sea urchin spines. a The original 3D \( \upmu \)-CT reconstruction volumetric data, (b) with detected 3D cracks, which is compared with (c) the human labeling results. d Here the cracks are shown as embedded in the cellular structure

3D Crack Analysis Results

To further characterize the cracks in the sea urchin spines, we register the cracks as 3D objects and represent the 3D morphology and orientation information quantitatively for mechanical analysis. Figure 8a shows the registered 3D cracks in the sea urchin spines. The crack notch patterns on each 2D slice are compiled into connected 3D objects, which are registered as tilted 3D planes characterized by orientation, crack opening width, and surface area. The crack orientation is characterized by two angles, the misorientation angle, \( \theta \) and the in-plane rotation angle, \( \varphi \), illustrated in the top right corner of Fig. 8. \( \theta \) is defined as the angle between the normal direction of each crack plane and a reference global direction. For calculation of the in-plane rotation angle \( \varphi \), the normal direction is first projected to the plane perpendicular to the defined global direction. \( \varphi \) is then defined as the angle between the projection direction and another selected in-plane reference direction. The misortinetation angle, \( \theta \), ranges from \( 0^\circ \)\( 90^\circ \) and the in-plane rotation angle, \( \varphi \), ranges from \( 0^\circ \)\( 360^\circ \). The crack opening width \( t \) is calculated by doubling the average distance of all surface points to the fitted crack planes. The surface area \( s \) is then calculated by summing up the triangular mesh area based on the surface mesh data for each individual crack. Figure 8 shows one registered crack with the registered crack opening width \( t = 4.49 \; \upmu{\text{m}} \), surface area \( s = 540.76 \; \upmu{\text{m}}^{2} \), and cracking plane orientation \( \theta = 80.67^\circ \), \( \varphi = 23.74^\circ \). The blue arrow shows the normal direction of fitted planes.
Fig. 8

3D crack registration results in sea urchin spine. Registered different crack components are labeled with different colors. The zoomed-in figure shows the 3D registration of one titled crack plane with registered crack opening width \( t = 4.49 \upmu{\text{m}} \), surface area \( s = 540.76 \upmu{\text{m}}^{2} \), and cracking plane orientation \( \theta = 80.67^\circ \), \( \varphi = 23.74^\circ \), where tilt angle \( \theta \) is the angle between the normal direction of the crack plane and the z-axis, the in-plane angle \( \varphi \) is the angle between the projected plane normal direction on the \( xy \) plane and the \( x \)-axis

This 3D crack characterization enables a statistical analysis for all the cracks. The results are displayed in Fig. 9a–c, where orientation of the crack planes, crack opening width and surface area are displayed in histograms.
Fig. 9

3D crack morphology statistics for the sea urchin spine sample, showing the orientation angle distribution (a), the histogram of crack opening width \( t \) (b), and the surface area \( s \) (c)

The result proves that our algorithm can extract and separate the 3D pattern of the damage cracks from the hosting environment and reconstruction artifacts. The framework introduces myriad opportunities for 3D damage characterization and is broadly applicable to other types of abnormality detection in complicated structured material systems.

Conclusion

In this paper, we demonstrate a 3D crack characterization method that can be applied to various cellular material systems. This feature-map-based method utilizes a large collection of crack features to rapidly eliminate the background of hosting structures or reconstruction artifacts. A refinement based on supervised machine learning further improves the detection accuracy from the training data. In addition, the method allows for the registering of 3D information for each individual crack, including the location, area, orientation, and crack opening width. Such information can be integrated with the original cellular network structural, providing valuable insights about the structural-property relationship of cellular materials from the individual branch to the full network level. Compared to human labeling, this 3D computer vision approach is both significantly faster and more sensitive, providing robust quantitative 3D characterization of cracks on various material systems subject to noise and CT reconstruction artifacts.

Notes

Acknowledgements

We gratefully acknowledge the National Science Foundation under award number CMMI-1825646. This research used resources of the Advanced Photon Source, a U.S. Department of Energy (DOE) Office of Science User Facility operated for the DOE Office of Science by Argonne National Laboratory under Contract No. DE-AC02-06CH11357. L.L. thanks the Department of Mechanical Engineering at Virginia Tech for the support. We thank Dr. X. Xiao, Pavel Shevchenko, and Dr. Francesco De Carlo for their technical assistance with synchrotron measurement.

References

  1. 1.
    Gibson L, Ashby M, Harly B (2010) Cellular materials in nature and medicine. Cambridge University Press, CambridgeGoogle Scholar
  2. 2.
    Gibson LJ, Ashby MF (1999) Cellular solids: structure and properties. Cambridge University Press, CambridgeGoogle Scholar
  3. 3.
    Albe M et al (2018) Interplay between calcite, amorphous calcium carbonate, and intracrystalline organics in sea urchin skeletal elements. Cryst Growth Des.  https://doi.org/10.1021/acs.cgd.7b01622 CrossRefGoogle Scholar
  4. 4.
    Presser V, Schultheiß S, Berthold C, Nickel KG (2009) Sea Urchin Spines as a model-system for permeable, light-weight ceramics with graceful failure behavior. Part I. Mechanical behavior of Sea Urchin Spines under compression. J. Bionic Eng. 6:203–213CrossRefGoogle Scholar
  5. 5.
    Seto J et al (2012) Structure-property relationships of a biological mesocrystal in the adult sea urchin spine. Proc Natl Acad Sci U. S. A. 109:3699–3704CrossRefGoogle Scholar
  6. 6.
    Moureaux C et al (2010) Structure, composition and mechanical relations to function in sea urchin spine. J Struct Biol 170:41–49CrossRefGoogle Scholar
  7. 7.
    Yamada Y et al (2000) Compressive deformation behavior of Al2O3 foam. Mater Sci Eng A 277:213–217CrossRefGoogle Scholar
  8. 8.
    Brezny R, Green DJ (1993) Uniaxial strength behavior of brittle cellular materials. J Am Ceram Soc 76:2185–2192CrossRefGoogle Scholar
  9. 9.
    Tulliani JM, Montanaro L, Bell TJ, Swain MV (1999) Semiclosed-cell mullite foams: preparation and macro- and micromechanical characterization. J Am Ceram Soc 82:961–968CrossRefGoogle Scholar
  10. 10.
    Telford M (1985) Domes, arches and urchins: the skeletal architecture of echinoids (Echinodermata). Zoomorphology 105(2):114–124CrossRefGoogle Scholar
  11. 11.
    Towe KM (1967) Echinoderm calcite: single crystal or polycrystalline aggregate. Science 157:1048–1050CrossRefGoogle Scholar
  12. 12.
    Dubois P, Ameye L (2001) Regeneration of spines and pedicellariae in echinoderms: a review. Microsc Res Tech 55:427–437CrossRefGoogle Scholar
  13. 13.
    Smith AB (1980) Stereom microstructure of the echinoid test. Spec Pap Palaeontology 25:1–81Google Scholar
  14. 14.
    Chen TT (2011) Microstructure and micromechanics of the sea urchin, Colobocentrotus atratus. Doctoral dissertation, Massachusetts Institute of TechnologyGoogle Scholar
  15. 15.
    Donnay G et al (1969) X-ray diffraction studies of echinoderm plates. Science 80(166):1147–1150CrossRefGoogle Scholar
  16. 16.
    Lowenstam HA, Weiner S (1989) On biomineralization. Oxford University Press, OxfordGoogle Scholar
  17. 17.
    Weber JN (1969) The incorporation of magnesium into the skeletal calcites of echinoderms. Am J Sci 267:537–566CrossRefGoogle Scholar
  18. 18.
    Kim E et al (2016) Suppressed instability of a-igzo thin-film transistors under negative bias illumination stress using the distributed bragg reflectors. IEEE Trans Electron Devices 63:1066–1071CrossRefGoogle Scholar
  19. 19.
    Cheng HD, Shi XJ, Glazier C (2003) Real-time image thresholding based on sample space reduction and interpolation approach. J Comput Civ Eng 17:264–272CrossRefGoogle Scholar
  20. 20.
    Ying L, Salari E (2010) Beamlet transform-based technique for pavement crack detection and classification. Comput Civ Infrastruct Eng 25:572–580CrossRefGoogle Scholar
  21. 21.
    Santhi B, Krishnamurthy G, Siddharth S, Ramakrishnan PK (2012) Automatic detection of cracks in pavements using edge detection operator. J Theor Appl Inf Technol 36:199–205Google Scholar
  22. 22.
    Nisanth A, Mathew A (2014) Automated visual inspection of pavement crack detection and characterization. Int J Technol Eng Syst 6:14–20Google Scholar
  23. 23.
    Xie X, Xia Y, Liu B, Li K, Wang T (2019) The multichannel integration active contour framework for crack detection. Int J Adv Robot Syst 16:1–13Google Scholar
  24. 24.
    Sinha SK, Karray F (2002) Classification of underground pipe scanned images using feature extraction and neuro-fuzzy algorithm. IEEE Trans Neural Netw 13:393–401CrossRefGoogle Scholar
  25. 25.
    Lei H, Cheng J, Xu Q (2019) Cement pavement surface crack detection based on image processing. Mech Eng Sci 1:63–70CrossRefGoogle Scholar
  26. 26.
    Li G, Zhao X, Du K, Ru F, Zhang Y (2017) Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Autom Constr 78:51–61CrossRefGoogle Scholar
  27. 27.
    Cavalin P, Oliveira LS, Koerich AL, Britto AS (2006) Wood defect detection using grayscale images and an optimized feature set. In: IECON conference on industrial electronics. pp 3408–3412.  https://doi.org/10.1109/iecon.2006.347618
  28. 28.
    Gu IYH, Andersson H, Vicen R (2010) Wood defect classification based on image analysis and support vector machines. Wood Sci Technol 44:693–704CrossRefGoogle Scholar
  29. 29.
    Kong X, Li J (2018) Vision-based fatigue crack detection of steel structures using video feature tracking. Comput Civ Infrastruct Eng 33:783–799CrossRefGoogle Scholar
  30. 30.
    Wang J. et al. (2016) CNN–RNN: a unified framework for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2285–2294 Dec 2016Google Scholar
  31. 31.
    Wu Z, Yang T, Li L, Zhu Y (2019) A hierarchical reconstruction of x-ray phase tomography based on transferred non-local structural features.  https://doi.org/10.1117/12.2519055
  32. 32.
    Wu Z, Yang T, Li L, Zhu Y (2019) Hierarchical convolutional network for sparse-view X-ray CT reconstruction.  https://doi.org/10.1117/12.2521239
  33. 33.
    Wu, Z., Alorf, A., Yang, T., Li, L. & Zhu, Y. Robust X-ray Sparse-view Phase Tomography via Hierarchical Synthesis Convolutional Neural Networks. 1–9Google Scholar
  34. 34.
    Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448Google Scholar
  35. 35.
    Jiang H, Learned-Miller E (2017) Face detection with the faster R-CNN. In: Proc 12th IEEE Conf Autom Face Gesture Recognition, pp 650–657.  https://doi.org/10.1109/FG.2017.82
  36. 36.
    Sun X, Wu P, Hoi SCH (2018) Face detection using deep learning: an improved faster RCNN approach. Neurocomputing 299:42–50CrossRefGoogle Scholar
  37. 37.
    Li X, Shang M, Qin H, Chen L (2015) Fast accurate fish detection and recognition of underwater images with Fast R-CNN. In: Ocean 2015-MTS/IEEE Washington, 1–5 2017.  https://doi.org/10.23919/oceans.2015.7404464
  38. 38.
    Hoang Ngan Le T, Zheng Y, Zhu C, Luu K, Savvides M (2016) Multiple scale faster-RCNN approach to driver’s cell-phone usage and hands on steering wheel detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 46–53, 2016.  https://doi.org/10.1109/cvprw.2016.13
  39. 39.
    Braun M, Rao Q, Wang Y, Flohr F (2016) Pose-RCNN: Joint object detection and pose estimation using 3D object proposals. In: IEEE conference on intelligent transportation systems, ITSC.  https://doi.org/10.1109/itsc.2016.7795763
  40. 40.
    Kundu A, Li Y, Rehg JM (2018) 3D-RCNN: instance-level 3D object reconstruction via render-and-compare. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 3559–3568.  https://doi.org/10.1109/cvpr.2018.00375
  41. 41.
    Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Comput Civ Infrastruct Eng 32:361–378CrossRefGoogle Scholar
  42. 42.
    Zhang L, Yang F, Daniel Zhang Y, Zhu YJ (2016) Road crack detection using deep convolutional neural network. In: Proceedings of international conference on image processing, ICIP, 3708–3712 Aug 2016Google Scholar
  43. 43.
    Li W, Field KG, Morgan D (2018) Automated defect analysis in electron microscopic images. npj Comput Mater 4:1–9CrossRefGoogle Scholar
  44. 44.
    Gürsoy D, De Carlo F, Xiao X, Jacobsen C (2014) TomoPy: a framework for the analysis of synchrotron tomographic data. J Synchrotron Radiat 21:1188–1193CrossRefGoogle Scholar
  45. 45.
    Presser V et al (2009) Sea Urchin Spines as a model-system for permeable, light-weight ceramics with graceful failure behavior. Part II. Mechanical behavior of sea urchin spine inspired porous aluminum oxide ceramics under compression. J Bionic Eng 6:357–364CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.The Department of Electrical & Computer EngineeringVirginia Tech (Virginia Polytechnic Institute and State University)BlacksburgUSA
  2. 2.The Department of Mechanical EngineeringVirginia Tech (Virginia Polytechnic Institute and State University)BlacksburgUSA
  3. 3.The Department of Computer ScienceVirginia TechBlacksburgUSA
  4. 4.Department of Ecology and Evolutionary BiologyPrinceton UniversityPrincetonUSA

Personalised recommendations