A study on skeletonization of complex petroglyph shapes

In this paper, we present a study on skeletonization of real-world shape data. Skeletonization is a crucial prerequisite for the robust description and indexing of shapes, and for further search and shape retrieval [24]. The investigated data represent manually performed tracings of prehistoric petroglyphs that pose novel challenges to skeletonization due to their complex topology and structure. Petroglyphs are human-made markings on rock surfaces, which were pecked, scratched or carved into rocks [8]. They can be found all over the world and are preserved, studied, and interpreted by archeologists to gain knowledge about early human history. Petroglyphs are an interesting testbed for skeletonization as they exhibit a number of challenges. Depicted motifs range from simple geometric shapes (e.g. crosses) up to compositions of complex hunting, fighting, and dancing scenes.¹ The tracings of the petroglyph shapes may be incomplete due to partial abrasion of the rock surface. Since the petroglyphs are made of individual peck marks they exhibit a complex boundary as well as numerous holes in their interior (see Fig. 1a). Additionally, complex figures may consist of several disconnected parts. Petroglyph shapes can either show filled bodies or just the silhouette of a figure depending on their artistic style. Finally over the years figures have been pecked on top of each other which results in merged shapes.

Petroglyphs are important artifacts that document early human life and development. The digitization and thus permanent preservation of petroglyphs recently gained increasing attention [23, 34]. Recent effort is put into the building of retrieval systems that enable the search for similar shapes as well as the automated classification of petroglyphs into predefined shape classes according to archeological typologies [25]. Following the segmentation of photographs of petroglyphs to get the shapes of the figures [23], the work in this publication is an essential prerequisite for later automated recognition of the shapes based on skeletal descriptors [24, 25].

Existing skeletonization algorithms are not directly applicable to this type of material and yield poor skeletons as shown in Fig. 1b. One reason for the poor performance is that existing methods are usually developed on perfectly segmented shapes with continuous contour and continuously filled regions originating, for example, from public datasets such as MPEG-7 Core Experiment CE-Shape-1 and Kimia-99 [3, 4, 28, 33]. Thus most methods do not fulfill the requirements of noisy real-world data such as that employed in this work. Other algorithms are designed for special tasks (e.g. fingerprint recognition) and therefore rely on specific image properties such as parallel ridges and furrows with well-defined frequency, orientation, and line width [2]. This makes such algorithms inappropriate for our material which show a high variety in composition, line width, and complexity. Aside from different applications (e.g. shape retrieval) enabled by robust skeletonization, the employed petroglyph shapes pose a powerful testbed for the further development of skeletonization algorithms.

Our contribution is a comprehensive investigation of skeletonization in a new application domain that provides noisy real-world data and has rarely been explored so far [29]. We study the applicability of existing skeletonization methods and evaluate their strengths and weaknesses. We identify two major requirements for robust skeletonization: (i) pre-processing of the shapes is necessary to smooth the contour and fill holes in the interior; (ii) existing methods require adaptions to handle the complex structure of the shapes. We propose several improvements of recent skeletonization methods to make them applicable to the investigated material. Finally, we come up with a novel shape pre-processing method that improves the shapes by smoothing and hole filling. The goal of the pre-processing is to reduce the complexity of the shapes and thereby to facilitate the subsequent skeletonization. Our method balances the degree of smoothing and hole filling by an adaptive threshold to yield a suitable tradeoff between preserving the topology and removing noise. An evaluation on our dataset shows that pre-processing is a crucial step for robust skeletonization of noisy and complex shapes like petroglyph tracings (see Fig. 1). Furthermore the proposed improvements of the skeletonization algorithms yield more accurate and complete skeletons.

Open image in new window

The paper is structured as follows: In Section 2 we review related work on skeletonization and skeleton pruning and identify suitable algorithms for our task. Section 3 presents our real-world material and its characteristics. We describe our pre-processing approach and the improvements of skeletonization algorithms in Section 4. Experimental setup and results are presented in Sections 5 and 6. Finally we draw conclusions in Section 7.

In this section we review skeletonization and skeleton pruning algorithms, analyze their properties and identify suitable methods for our task. In the literature the usage of terminology for skeletonization is highly ambiguous. Skeletonization, thinning, medial axis transform, distance transform as methodologies and skeleton, medial axis or medial line as their results are used inconsistently [5]. According to Arcelli and Baja [1] algorithms for skeleton computation in discrete space can generally be partitioned into two categories: Methods that perform skeletonization by medial axis transform produce skeletons following Blum’s definition of the medial axis [6] and techniques employing skeletonization by thinning derive a thin version of a shape [9, 13]. A third category of approaches performs the medial axis transform to polygonal shapes in continuous space [12, 20, 21]. Additionally, there is a group of more recent skeletonization algorithms that utilize physics-based modeling of the shapes [14, 22] for which we suggest a fourth category.

All skeletonization methods are sensitive to boundary noise, i.e. small perturbations of a shape may have large influence on the skeleton (see Fig. 1). To overcome this problem some form of regularization is required [26]. This regularization process is generally referred to as “skeleton pruning”. Shaked and Bruckstein observe that pruning is an essential part of skeletonization algorithms and most recent developments combine skeletonization and skeleton pruning in one algorithm. Skeleton pruning methods can be consolidated in two major categories: The first covers the pruning of skeleton branches based on a significance value calculated for every single skeleton point. This results in a shortening of all skeleton branches. The second class of skeleton pruning algorithms calculate a significance measure for each branch. Based on its significance value a branch is either removed completely or remains in the skeleton [18].

The state-of-the-art technique in archeology to document petroglyphs is to manually trace them on transparent foils peck mark by peck mark. Subsequently these foils are scanned and stitched together to large images that cover entire rock surfaces. The tracings show different types of figures and also complex scenes that document ancient life. To extract the individual petroglyph shapes from the digitized tracings, we built a web tool that facilitates the semi-automatic segmentation and manual classification of individual petroglyphs into semantic shape classes according to an archeological typology, e.g. antropomorph, hut, deer, and bird [25]. In a joint effort, a team of archeologists compiled a large data set of more than 1100 petroglyph shapes classified according to two different archeological typologies. Figure 2 shows a small subset of the dataset.

Open image in new window

Initial experiments of Takaki et al. showed that skeletonization is a useful abstraction of shapes [29]. Thus it enables higher level applications such as similarity search and automated shape classification, which is our ultimate goal. Petroglyphs pose a challenge for skeletonization as they are made of single peck marks and thus have neither a continuous contour nor continuously filled regions. Figure 2 shows that the shapes have highly varying complexity, contain numerous holes due to incompletely pecked areas, often contain very fine structures (horns of deer, feathers of birds, etc.), and have disconnected parts.

As already discussed in Section 1 the characteristics of petroglyph shapes impede skeletonization which leads to unsatisfactory results (see for example Fig. 1). For robust skeletonization a pre-processing of the shapes is necessary as well as improvements of skeletonization techniques. In the following sections we present a fully automated shape pre-processing method and propose a number of improvements for the selected skeletonization algorithms to make them applicable to petroglyph shapes.

4.1 Adaptive shape pre-processing

The major challenge of pre-processing is to find a solution that forms a good tradeoff across all different types of petroglyphs in our dataset. We define a minimum set of requirements that a suitable pre-processing needs to fulfill to enable more robust skeletonization: (i) close small holes, (ii) smooth the contour, (iii) connect nearby parts, and (iv) avoid the unwanted decomposition of the shapes. We design a method that automatically improves the shapes according to these requirements and that terminates autonomously when a good tradeoff amongst the four aims is achieved. The intuition behind our approach is to combine different structural and morphological filters in a way that they join their strengths. An overview of the method is shown in Fig. 3.

Open image in new window

Initially we resize and pad all shapes to normalize the inputs. Next we apply a median filter with size s_med to the input. Median filtering removes small holes in foreground and background (salt and pepper noise) and at the same time slightly smooths the contour. Apart from this, however, the median filter may generate artifacts by disconnecting weakly connected blobs. To compensate for these artifacts, we apply an area opening and closing as proposed by Vincent [32]² as well as a dilation operation. We use an area size of t_aoc pixels as threshold for area opening and closing, and combine it with a dilation by a disc with a radius r_dil to reconnect disjoint parts. We iterate these steps with increasing median filter size until a stopping criterion is met.

The stopping criterion requires a robust indicator function that is suitable for the differently complex shapes in the dataset. We evaluate different indicator functions such as solidity and circularity of the shape, the number and size of foreground and background blobs, and the number of endpoints in the thinning skeleton. Our preliminary experiments show that the most robust criterion is a combination of the number of foreground blobs and the number of background blobs. If the number of both do not change over a certain number of iterations n_it, we terminate the pre-processing. The intuition behind this stopping criterion is that if the number of foreground and background blobs remains constant for some time, the figure is likely to be in a robust state where the influence of noise is low.

After the iterative pre-processing has terminated, we smooth the contour by a convolution filter with a Hanning window of size s_conv. We smooth the x- and y-coordinates of the shapes’ contour points separately which removes contour perturbations and thus reduces the likelihood to get spurious (insignificant) skeleton branches in subsequent skeletonization.

Figure 4 illustrates multiple iterations of the proposed pre-processing method for a given shape. The numbers of foreground and background blobs decrease rapidly during the initial iterations as holes in the shape are removed and closely spaced shape parts are merged. After a few (4) iterations the shape reaches a robust state which is indicated by the fact that the number of foreground and background blobs do not change for a certain number of iterations, i.e. the two indicator functions reach a plateau. We observe further plateaus (e.g. starting at iteration 7, 12, and 15). As iterations proceed the plateaus usually get longer. The stopping criterion finally decides after which plateau length to stop by parameter n_it. For the given example a n_it=3 would give satisfactory results. After the first plateau the number of foreground blobs increases again and the shape starts to decompose.

Open image in new window

4.2 Improvements of existing skeletonization methods

We take the automatically pre-processed shapes as basis for the computation of the skeletons with the chosen algorithms (thinning, BPR, DCE, and SPT). In our preliminary experiments we observe that two of these algorithms (DCE and SPT) suffer from severe problems (see Fig. 5a, left column): The original DCE algorithm [4] is not able to compute the skeleton of multiple blobs in one image, it does not preserve the topology of the shape (loses inner connections), and sometimes generates incomplete skeletons. We propose improvements to compensate for these shortcomings. First, we apply skeleton computation in all blobs of the input instead of on the first blob only (see Fig. 5a, 1^st row). To preserve the inner connections, we extend the contour tracing of DCE in a way that all inner contours are traced (see Fig. 5a, 2^nd row). The original DCE-algorithm employs either only the concave or only the convex contour points (depending on which set of points is larger) for skeletonization and misses significant shape parts when the concave are selected (see Fig. 5a, 3^rd row). We remove the selection criterion and employ all (convex and concave) points for skeletonization. As a result we obtain complete skeletons even for complex shapes. The effects of our improvements are shown in Fig. 5a, right column. The SPT algorithm [28] utilizes DCE for a coarse pruning of the shape and thus benefits from our improvements as well. A second pruning step in SPT refines the skeleton by deleting branches whose reconstruction contribution to the shape is below a certain threshold. In the original reconstruction step the skeleton path tracing starts at one seed point and traces the skeleton to every endpoint. Thereby inner connections and partly outer connections are missed and thus removed from the skeleton (see Fig. 5b, left). We improve the skeleton path tracing by setting several seed points at the skeleton branch points. As a result all inner connections are preserved in the final skeleton (see Fig. 5b, right).

Open image in new window

From the ongoing annotation process of the investigated material described in Section 3, we derive a dataset of 1181 petroglyph shapes to carry out our experiments with.³ In the following, we describe the selection of the parameters for our adaptive shape pre-processing method and the investigated skeletonization methods, and present our evaluation criteria.

5.1 Selection of parameters

For the selection of suitable values for the parameters defined in Section 4.1, we evaluate the pre-processing method on a reduced dataset consisting of 150 representative shapes from the entire dataset. In the absence of a ground truth of pre-processed shapes, we face difficulties in the selection of suitable parameter values. Hence we choose a heuristic approach to estimate robust parameters for the proposed method.

To determine the area opening and closing threshold t_aoc [32], we examine the histogram of all blob sizes in the reduced dataset. The histogram is strongly skewed to the left and exhibits a peak between blob sizes of 5 and 20 pixels. Blobs of this size mostly represent noise (e.g. small holes) that usually does not contribute to a figure’s shape significantly. With a threshold t_aoc between 5 and 20 pixels about 20 % of all blobs are removed and a large portion of noise is filtered out. For the remaining experiments with the entire dataset, we set t_aoc to 10 pixels. For the size of the dilation disk r_dil we select a rather low value of 3 pixels. This facilitates the reconnection of nearby parts and minimizes the amount of region growing so that the likelihood of merging unrelated shape parts is reduced. From the reduced dataset we observe that typical plateau lengths of the indicator functions are between 2 and 5 iterations. A value of n_it=3 iterations yields a good tradeoff between the sufficient smoothing of perturbed shapes, and the decomposition of thin and sparsely connected figures. For some figures a higher value for this stopping criterion leads to a better smoothed shape (see Fig. 6a), whereas for others a higher value leads to a decomposition (see Fig. 6b). To estimate s_conv, we apply thinning to the shapes in the reduced dataset and count the number of endpoints. We repeat this process with increasing values of s_conv. For filter sizes s_conv between 30 and 70 contour points the number of the endpoints remains mostly constant. This shows that the parameter has low sensitivity. We set s_conv to 51 pixels for our experiments. The size of the median filter is a non-critical parameter as it is simply increased every iteration. We start with a minimum size of s_med=3 pixels and increase it by 2 pixels every iteration.

Open image in new window

Additional parameters have to be selected for the skeletonization methods. For DCE [4], we estimate the parameter for the number of vertices adaptively by counting the number of endpoints of the respective thinning skeleton. For the BPR-algorithm [27] and the SPT-algorithm [28], we take the parameter values as proposed by their authors. The computation of the thinning skeleton is parameter free as it is a simple morphological operation.

First, we evaluate the effect of the proposed pre-processing method quantitatively. The intuition behind this evaluation is to obtain an indication of how effectively the pre-processing simplifies the shapes (and thus improves them for subsequent skeletonization). To assess this ability, we utilize a simple observation, namely that typical petroglyph skeletons should usually not exhibit more than 20 endpoints. This can be derived by looking at the shapes in Fig. 2. A consequence of this observation is that skeletons with more than 20 endpoints are likely to contain spurious branches. We use the number of endpoints as a simple heuristic to evaluate the performance of the pre-processing method. For evaluation we count the number of endpoints of the thinning skeletons before and after pre-processing. Figure 7 shows the distribution of the number of endpoints before and after pre-processing. Pre-processing strongly reduces the number of endpoints in the resulting skeletons and maps them to a reasonable range. Thus this evaluation provides a first indication that the proposed pre-processing is beneficial for subsequent skeletonization.

Open image in new window

The pre-processing method performs well for 79.8 % of all figures. Examples are shown in Fig. 8a where unimportant holes in the shapes are removed successfully and the contours are properly smoothed. For the remaining 20.2 % of the shapes we identify different types of errors (see Fig. 8b and Table 2). 9.4 % of all shapes are not smoothed sufficiently and their contours remain perturbed, which impedes skeletonization. For 6.9 % of the shapes closely spaced parts are merged although it would be important for skeletonization that they remain separated. Only 3.9 % of the shapes are smoothed too strongly and thus important shape parts are lost. The proposed method thus forms a tradeoff between the filtering of noise and removal unimportant shape parts, and the smoothing of important shape parts.

Open image in new window

Similarly to the evaluation of the pre-processing, we assessed the quality of the skeletonization methods according to the criteria presented in Section 5.2. The performance evaluation of the skeletonization methods on the pre-processed shapes for the entire dataset is shown in Table 3. On average (over all skeletonization algorithms) we obtained satisfactory results for the majority of our shapes. Figure 9 provides examples of the obtained skeletons and demonstrates that the skeletons are accurate to a high degree. We comprehensively investigated different types of errors that arise during skeletonization and observed that all skeletonization algorithms have deficiencies in certain situations, but none of them completely fails. See Fig. 10 for examples of improper skeletonization.

Open image in new window

Open image in new window

Table 3

Evaluation of the four selected skeletonization algorithms on the entire dataset. BPR and thinning perform best on the pre-processed shapes

Open image in new window	Spurious branches	Lost parts	Sum errors	Sum correct
Thinning	12.5 %	4.0 %	16.5 %	83.5 %
BPR [27]	6.7 %	6.4 %	13.1 %	86.9 %
DCE [4]	47.7 %	3.7 %	51.4 %	48.6 %
SPT [28]	65.8 %	1.4 %	67.2 %	32.8 %

The skeletons pruned with DCE are in most cases complete (96.3 %), but the algorithm additionally produces a lot of spurious branches (47.7 %). This is related to the fact that the algorithm requires prior knowledge about the shapes (the number of DCE vertices). If this number is set too low, significant branches are deleted even before spurious branches (3.7 % of all shapes). If the number is set too high, a lot of spurious branches remain (47.7 %). We set this parameter adaptively (depending on the number of endpoints of the corresponding thinning skeleton) since a unique number that is suitable for all shapes does not exist.

SPT builds upon DCE and performs similarly. Although it produces even more spurious branches (65.8 % of all shapes), it deletes spurious branches before important ones. Thus for only 1.4 % of all shapes important parts are lost. An advantage of SPT over DCE is that it does not require a priori information about the shapes.

The BPR algorithm outperforms DCE and SPT and produces satisfactory skeletons for 86.9 % of all shapes. BPR generates much fewer spurious branches than DCE and SPT (only 6.7 % of all shapes). The BPR pruning is, however, in some situations too strong and thus important branches are removed in 6.4 % of all shapes (see for example tail of the bird in Fig. 10, 2nd row).

A simple thinning results in notably good skeletons for 83.5 % of all shapes which is nearly as good as the performance of the more sophisticated BPR algorithm. Since petroglyph shapes often resemble stick-like figures, they can be well modeled by the thinning algorithm. Additionally, the contour smoothing in the pre-processing avoids the generation of spurious branches by thinning (in 87.5 % of all cases). The results obtained for thinning show that a proper pre-processing can replace an additional skeleton pruning. In only 4.0 % of all cases important branches are missed by thinning.

In this paper, we presented a study on skeletonization of petroglyph shapes. We introduced a large heterogeneous dataset of real-world shapes that exhibit numerous challenges to existing skeletonization algorithms and thus poses an interesting testbed. We studied the applicability of existing skeletonization methods and evaluated their strengths and weaknesses. Existing skeletonization methods were developed and evaluated mainly on ideal shapes and are thus not directly applicable to our real-world data. Therefore we improved several skeletonization algorithms to compensate for their shortcomings that became apparent. Additionally, we proposed an adaptive shape pre-processing method that enables the computation of robust skeletons for the complex and diverse shapes under investigation. We performed a large-scale experiment and showed that a proper pre-processing is crucial for the skeletonization of petroglyph shapes. Experiments on skeletonization showed that pre-processing in combination with a simple thinning yields a good tradeoff for robust skeletonization, whereas more sophisticated skeletonization techniques either generate more spurious branches (DCE, SPT) or delete important ones (BPR). Our experiments clearly demonstrated that the presented pre-processing method and the proposed improvements of recent skeletonization methods solve the additional challenges introduced by our complex and noisy real-world shape data for more than 86% of all investigated shapes.