Introduction

Ancient monuments and artifacts are often damaged because of exposure to natural or artificial environmental factors, which can lower their historical value or significance [1,2,3,4,5,6,7,8,9]. Traditionally, functional repair and aesthetic restoration are undertaken to recover the original form of damaged monuments and artifacts [10, 11]. The Athens Charter (1931), Venice Charter (1964), and Nara Document on Authenticity (1992) have been prepared to establish restoration principles for cultural heritages, and countries around the world have restored their cultural heritages according to their own rules following international recommendation [12]. Generally, damaged artifacts can be restored only when their historical and cultural value can be retrieved through sufficient and direct historical research. Any restoration that may distort the value or cause a controversy is restricted. Fundamentally, a traditional technique and original materials are used for restoration. However, if the traditional technique is not applicable and the original materials cannot be supplied, modern technologies and materials that have been scientifically and experientially established can be used.

Recently, various conservation methods using advanced technologies and new materials have been used in the restoration process of cultural artifacts [13,14,15,16]. In particular, when a cultural artifact loses its historical value or changes its shape, the restoration effort focuses on recovering the entire or partial shape of a certain period via thorough historical research and consultation with experts [17,18,19]. The restoration of a cultural artifact is usually realized using a manual method that relies on conservators’ intuition and techniques [20, 21]. Using this method, physical contact with the material is inevitable. Moreover, if the restored parts look inappropriate, it would have to be removed. In this case, the original artifact can suffer a secondary damage either from frequent physical contact or excessive force applied during the removal of the parts.

With the advancement in digital technology, many approaches that complement traditional restoration methods have emerged. The most prominent technologies include three-dimensional (3D) scanning and printing. 3D scanning is used in areas of digital documentation [22,23,24,25], shape analysis [26], conservation evaluation [27], and digital restoration [28,29,30]. In particular, this method is actively used for digitalizing and documenting rock materials such as rock arts [31], inscribed stones [32, 33], knapped stones [34], dolmen [35], and stone pagoda [36]. Moreover, 3D printing has been employed in a wide area, including the establishment of restoration planning, supplementing the lost area, replication, and exhibition. Because of the availability of a wide variety of printing materials, 3D printing is used for the actual restoration of damaged artifacts [37,38,39,40,41,42,43,44,45].

In this study, the restoration of a stone-seated Bodhisattva (stone Buddha statue) was performed using various digital technologies. This statue is composed of zeolitic tuff, which has rarely been reported to be used as a Buddha material around the world. Thus, its unique value as a representative style of the Buddha statue has been appreciated between the seventeenth and nineteenth century in Korea. However, this statue was mostly damaged because of the missing parts. According to the ICOMOS-ISCS glossary, the missing part is defined as the empty space, obviously located in the place of some formerly existing stone part. The protruding and particularly exposed parts of sculptures are typical locations for material loss, which result in the missing parts [46]. The missing parts of the Buddha statue are attributed to the soft and weak characteristics of the material.

Therefore, noncontact restoration was realized using 3D scanning, virtual restoration modeling, and 3D printing technologies. To achieve this goal, 3D high-precision scanning was used to record the original shape of the stone Buddha statue and the missing parts were virtually restored using a haptic modeling system. Moreover, the restoration of the stone Buddha statue was completed using the 3D printed outputs of the virtual restoration model. In particular, a systematic design mockup and simulation approach as well as the convergence of digital and analog technologies were used to enhance the heuristic-based assembly suitability of the virtually restored model.

Status of the study object

Wood, clay, and metal are the commonly used materials to create a small statue because they are sufficiently easy to obtain in Korea. In particular, wood and metal have been the most commonly used universal materials since a very long time; however, clay has mostly been used after the fifteenth century. In contrast to the commonly used materials, the study object, which is a stone-seated Bodhisattva excavated in Pyeongchang, Gangwon province in the Republic of Korea, is composed of zeolitic tuff (Fig. 1).

Fig. 1
figure 1

Status of the Buddha statue

Full size image

Based on visual observation, the rock exhibits a white microcrystalline substance with a dark porphyritic texture and its rock-forming minerals include albite, biotite, and quartz [47]. This rock was traditionally used to sculpt a Buddha statue, because it is very soft and easy to handle. The statue was discovered in several fragments in 1974 and has been maintained in almost its entire form through a conservation treatment. However, four fragments that are estimated to be parts of the crown and one fragment of an ear have been separately stored without joining [48].

A huge excavated hole exists inside the statue from below the chest to the bottom, which is believed to have stored relics. Traces of coloring using mineral pigments and gold gilding are found on the exterior of the statue; however, its exact form is difficult to discern as most of them were peeled off and deteriorated. Moreover, the crown, face, and left finger are missing, and several physical damages are observed because of stone crack and exfoliation. Owing to its vulnerable conservation status, this statue has been managed in the national museum storage with stable environmental conditions for decades and has not been used for exhibition. Therefore, aesthetic and functional restoration of the damaged parts is necessary to recover its historical value and use the statue for exhibition or display.

Devices and software

The digital model of the statue was created using a fixed 3D high-precision scanner and handheld mid-precision scanner. Among these, the fixed high-precision scanner (HDI Advance R3X, LMI TECHNOLOGIES, Canada) is based on triangulation and uses white structured light and two stereo cameras (2.8 MP). In this study, scanning was performed with a field of view (FOV) of 400 mm and an accuracy of 65 μm. Furthermore, the handheld mid-precision scanner (Eva, Artec3D, Luxembourg) that uses white structured light and camera (1.3 MP) has an accuracy of 0.1 mm and a maximum point resolution of 0.5 mm. Software used for operating the scanners included FlexScan3D and Artec Studio 13, and the 3D model was edited using Geomagic Design X 2019. The high-resolution digital scanning results were used to identify the manufacturing techniques and analyze the shape of the statue.

A haptic device (Geomagic Touch X, 3D SYSTEMS, USA) and voxel-based software (Geomagic Freeform Plus, 3D SYSTEMS, USA) were used to virtually restore the missing parts. The device provides haptic feedback when a user interacts with the virtual environment. In particular, voxel-based modeling using a virtual clay material provides design flexibility. Therefore, this system was used for both 3D modeling and virtual assembly simulation, and the suitability of the 3D restoration model was evaluated.

Material extrusion and photopolymerization 3D printing technologies were used to obtain the output of the virtual restoration model. A material extrusion 3D printer (Ultimaker 3 Extended, Ultimaker) was used to verify the modeling result through the output of a design mockup, and the mockup was printed using polylactic acid (PLA) material at a resolution of 0.10–0.15 mm. Moreover, the 3D printer (ProJet 6000 HD, 3D SYSTEMS, USA) used for the final restoration employed photopolymerization that provides a very high level of surface precision. The 3D model was printed using UV-hardened plastic material with a layer thickness of 10 mm. The final printed output was used to restore the statue and its exhibition after strengthening and joining.

Methods and data processing

3D scanning

The missing parts were restored by a noncontact approach using digital technologies such as 3D scanning, modeling, and printing (Fig. 2). Fixed and handheld scanning was performed to record the 3D shape of the statue and its missing parts (Fig. 3). First, the exterior of the statue (scanned 101 times) and five fragments (scanned 34 times) were digitized using the fixed high-precision scanner based on stereo vision. The total 3D scans were performed by considering the structure and appearance of the statue, scope of survey, and degree of overlapping. However, the fixed high-precision scanner based on the binocular lens could not reach sufficient depths to record the shape of the excavated hole from below the chest to the bottom of the statue. Therefore, a handheld mid-precision scanner based on a single lens with depth-scanning capability was used to record the shape of the excavated hole.

Fig. 2
figure 2

Methods and data processing

Full size image
Fig. 3
figure 3

3D scanning process and the result of convergence modeling of the statue

Full size image

Each image acquired by on-site scanning provides only misaligned raw shape information. Therefore, the raw scans were post processed to complete a single model from the raw scan data. The processing was executed in the following order: filtering, aligning, registering, merging, filling holes, and RGB texture mapping. The reliability of a 3D model depends on the registering result based on the iterative closet point algorithm [49]. Therefore, registering, which is the most important stage in scan processing, was performed with an average overlap ratio of 66% and within an error vector of 45 μm. This error vector value was very similar to the distance accuracy of the 3D scanner, indicating high reliability of the registering results.

Moreover, high-precision scanning of the statue exterior and mid-precision scanning of the excavated hole needed to be integrated into a single model because the scanning yielded separate 3D models. Accordingly, a deviation analysis between the two scanning models to evaluate the registering error revealed that most deviations were within ± 0.1 mm and the average RMS was 0.28 mm. These values are similar to the accuracy of general precision scanners; thus, the convergence model was created based on an ICP algorithm.

Virtual modeling and simulation

The general rule of the conservation treatment and restoration of cultural artifacts states only minimum and passive intervention under inevitable conditions. Therefore, in this study, among the damaged areas, minimum scope was selected for aesthetic restoration and exhibition. The cracks are very fine, and their depths are visually conspicuous. These cracks seemingly do not cause any severe physical and structural problems. Thus, the cracks in the statue were left unattended without conservation treatment. However, the missing parts have a critical impact on the aesthetic value of the statue. Therefore, some parts that could be recreated using symmetry modeling or assured significant evidence through historic research were restored. In particular, the entire stone fragments that were stored separately were used for the restoration.

The haptic modeling system used to create a virtual model of the missing parts enables intuitive modeling because it helps recognize conflicts and interference of data. Compared with the existing method that uses a mouse, this method is considerably excellent for creating a complex shape. In particular, the system uses a voxel engine to create a model, providing a user an experience of force feedback similar to that faced in actually carving of a virtual object. This system is mainly used in the medical field, graphic design, car industry, exhibition technology, and the restoration of cultural artifacts [50,51,52,53].

In this study, the digital virtual restoration of the four selected parts was performed using a haptic modeling system. In the process of modeling, Boolean operation was used to combine several complex models into a new complete model. Boolean operation is an essential method for original-based modeling requirements such as the restoration of the cultural artifacts; therefore, the operation was mainly used for symmetrization of the missing parts. Boolean operation was performed to modify the model, matching it exactly to the original shape. Because symmetry modeling could not be applied to the center of the crown part, a statue similar to the study object was selected as a reference and estimated restoration was performed.

A design mockup helps to intuitively understand the influence of interference caused by the components and reduces the time required for completion by solving a making-related problem in advance. Preliminary verification through design mockup minimizes the unnecessary consumption of materials and time in the restoration process and maximizes the work efficiency [54]. In this study, to enhance restoration completeness using the noncontact approach, digital–analog-based assembly simulation was performed by creating a design mockup. Material extrusion 3D printing, which can minimize time, shape error, and costs, was used to create the design mockup.

Results

3D documentation and shape analysis

The converged 3D model comprises 36,556,604 polyfaces with an average point distance resolution of 0.2 mm. According to Fig. 4a, the 3D scanning result clearly revealed the complete shape of the statue and the polygon mesh and RGB texture-mapping models showed high reality effects. In particular, the final 3D scanning result exhibited a high resolution with millions of polygons, which well revealed the surface texture, manufacturing technique, and detailed shape of the damaged area.

Fig. 4
figure 4

Result of 3D scanning and shape analysis of the statue. a The 3D polygon mesh and RGB texture-mapping model. b A 3D deterioration map of the stone Buddha statue, and the mapping of the restoration ranges

Full size image

Based on the 3D scanning, the stone Buddha statue was measured to be 410-mm wide, 310-mm thick, and 580-mm high. Its calculated area and volume were 7,250 and 20,040 cm3, respectively. Moreover, the excavated hole was measured to be 188-mm wide, 155-mm thick, and 195-mm high, and its area and volume were 1,228 and 2,401 cm3, respectively. In particular, the cone-shaped excavated hole showed traces of chisels whose average sizes were 10.5 and 15.4 mm. Therefore, the excavated hole may have been carved using these two chisels.

A 3D deterioration map was created to define the scope of the conservation treatment and restoration. According to the map, cracks and missing parts were observed around the head; the body was mainly damaged owing to the cracks (Fig. 4b). Out of the total area, the deterioration rates of the missing parts were 4.8% (348.3 cm2) in the eye and head and 0.7% (52.9 cm2) in the body.

Virtual restoration modeling

For restoration, the big missing parts in the head, surrounding area of the Baekho, right ear, and right eye were selected, accounting for 64.2% (257.1 cm2) of the total surface area of the missing parts (400.1 cm2).During the examination of the virtual restoration modeling process of the missing parts in the head (Fig. 5), first, the original stone fragments were placed by referring to the shape and pattern of the stone Buddha statue. Then, a reference model for the missing parts was selected and copied using symmetry modeling. However, because the original statue was manually created, it lacked perfect symmetry.

Fig. 5
figure 5

Virtual restoration modeling process of the head of the statue

Full size image

The restoration of the right ear has aesthetic as well as functional purposes to support the original fragment. Accordingly, for virtual restoration, the original fragment was placed in the right ear and the shape of the missing parts was acquired using the symmetry of the well-preserved left ear. Next, the angle and height from the front view were adjusted to match those in the left, and completed after partial revision (Fig. 6a, b). Finally, aesthetic restoration of the surrounding area of Baekho and right ear was performed by filling the inner part and retouching the outer part. In particular, their restoration was completed after several modifications in consultation with art history experts because the overall impression of the stone Buddha statue can change owing to minor shape differences (Fig. 6c).

Fig. 6
figure 6

Result of the virtual restoration modeling of the face of the stone Buddha statue. a, b The right ear. c The surrounding area of Baekho and right eye

Full size image

Design mockup and assembly simulation

To summarize the assembly simulation process, first, an alternative model of the original statue was printed on the basis of the 3D scanning model to minimize direct physical contact with the statue. Moreover, heuristic-based assembly suitability was examined several times by printing the virtual restoration model. When an error was detected during verification, the virtual restoration model was modified and reprinted to enhance assembly completeness. However, when no error or aesthetic issue was detected, a design mockup was directly applied to the statue for further verification. After using photopolymerization 3D printing, the restoration process was completed (Fig. 7).

Fig. 7
figure 7

Heuristic-based design mockup and assembly simulation process

Full size image

When the printed virtual restoration model was applied to the mockup statue, the interference between the two surfaces was severe owing to the nonstructured form (Fig. 8a). This indicates the high quality of the virtual restoration model; however, it was insufficient for restoring the original artifact. Therefore, the area in the virtual model where interference occurs should be corrected so that the printed output fits the missing parts of the statue. In addition to the revision of the virtual model, assembly suitability to the 3D-printed mockup statue was intuitively checked and verified by directly modifying the mockup of the virtual restoration model (Fig. 8b). The 3D model, which was verified for digital–analog assembly suitability, was completed as the final design mockup through the outer design modification (Fig. 8c).

Fig. 8
figure 8

Simulation process using the design mockup. a Verifying assembly suitability using a 3D-printed alternative model. b Direct modification of the design mockup printing output. c Final design mockup after revising the joining surface. d The modification result of interface modeling. e Deviation analysis and mapping result of the virtual restoration model and final design mockup model

Full size image

In a quantitative analysis of the shape difference between the virtual restoration model and final design mockup of the head, the volume of the virtual restoration model was measured to be 461 cm3 and that of the design mockup assembled on the original stone Buddha statue was measured to be 437 cm3 (Fig. 8d). The volume reduction of 24 cm3 (5.2%) equals the amount of the removed interfering surface along the assembly interface. In the deviation analysis of the final design mockup against the virtual restoration model, the size of the removed joining surface was nearly 5.00 mm, with RMS and standard deviation of 0.67 and 0.56 mm, respectively. Approximately 66.6% of the virtual restoration model, whose tolerance range was within ± 0.1 mm, was not modified. The interfering surface of the joining interface was removed for 33.4% of the virtual restoration model (Fig. 8e). This deviation-mapping result enabled the visualization of the modification range and deviation amount of the initial virtual restoration model, providing quantitative information about the digital–analog simulation process.

Restoration of the missing parts

For restoration, 3D printing technology was used to facilitate an educational effect in exhibition and enhance aesthetic and functional completeness. The material extrusion 3D printing used to create the design mockup played a critical role in analyzing the shape of the missing parts to design the assembly planning. However, this technology uses a PLA material with relatively high surface roughness and weak physical properties to functionally support the light-weight original stone fragments [55]. To complement this, photopolymerization 3D printing technology was used to create the final restoration model.

The model was printed using opaque UV-hardened plastic whose color was similar to that of the original statue, with a layer thickness of 0.1 mm (Fig. 9a). After approximately 15 h of printing, the supporters were removed from the printed output and the surface that was in contact with the supporter was carefully smoothed using soft sandpaper with more than 3,000 mesh. Additional UV hardening was used to enhance the surface strength of the printed output (Fig. 9b).

Fig. 9
figure 9

3D printing and conservation treatment processes for the restoration of the stone Buddha statue. a 3D-printed outputs using photopolymerization. b UV hardening to enhance surface strength. c 3D-printed outputs of the four selected parts, including the head, surrounding area of the Baekho, and right ear and eye. d 3D printing output colored with acrylic painting to modify the surface texture and prevent yellowing. e The interfaces of the missing parts of the original stature strengthened using an ethyl silicate-based consolidant. f Joining of the missing parts of the statue using acrylic resin and cyanoacrylate instant adhesives

Full size image

The finished 3D-printed output exhibited a smooth and glowing surface, which was disparate with that of the original statue (Fig. 9c). The initiator used in photopolymerization can cause yellowing owing to the surrounding light. It is known that the efficiency of polymerization can influence discoloration because a higher degree of conversion corresponds to a smaller amount of residual monomers available to form the colored degraded products [56]. By observing the light source as an isolated factor, it was noted that this can alter the color of the studied composite resin [57]. Therefore, to modify the surface texture of the 3D-printed output and prevent its yellowing, the surface was colored with acrylic painting and varnished to prevent discoloration of the colored layer (Fig. 9d).

The missing parts of the stone Buddha statue were restored using the conservation treatment and 3D-printed output. As described, the Buddha statue is composed of zeolitic tuff. This material is very soft and has high absorption ratio owing to its porous structure. Accordingly, the interfaces of the missing parts of the original statue were strengthened using an ethyl silicate-based consolidant before joining the 3D-printed output (Fig. 9e). Then, the 3D-printed output was joined to the missing parts of the statue using acrylic resin and cyanoacrylate instant adhesives (Fig. 9f).

In this study, various digital data and mockup of the original statue obtained using 3D scanning and printing technologies transformed the exhibition, which comprised simple display of artifacts, into an educational exhibition focusing on the restoration process. Combined with unique storytelling, the exhibition has provided a new experience to museum visitors (Fig. 10). This is an important case, which shows that established modern technologies and materials can be used for the restoration, education, and exhibition of cultural artifacts.

Fig. 10
figure 10

Restoration of the stone Buddha statue and its use in an exhibition. a The restoration result of the missing parts. b A special exhibition on the restoration process of the stone Buddha statue

Full size image

Discussion

The stone Buddha statue, owned by the Chuncheon museum, was largely damaged and not in its original shape because many stone fragments were stored separately. The value of the stone Buddha statue has been appreciated from the perspective of art history; accordingly, the necessity of the restoration of its missing parts based on historic research has been raised. Therefore, in this study, a noncontact restoration of the stone Buddha statue was performed by converging various digital technologies, including 3D scanning, virtual restoration modeling, assembly simulation, and printing restoration.

Furthermore, 64.2% of the total missing parts were selected as the restoration scope, which was deemed necessary for the aesthetic and functional restoration. Moreover, the separately retained stone fragments were all used for the restoration. A digital virtual restoration of the missing parts was performed using the original-based symmetry modeling [45]. For the partial area in the head where symmetrization was difficult owing to damage to more than half of its part, restoration was performed through historical research and verification. In particular, throughout the modeling process, a haptic device and voxel software were used, facilitating an intuitive restoration process.

The assembly simulation using the design mockup was the most focused stage in the whole restoration process. Generally, virtual assembly simulation is widely used in medical and industrial fields to evaluate the impact of surface interference in advance [58], and virtual assembly based on various algorithms is used in the field of cultural heritage [59,60,61,62,63]. However, virtual assembly has its limitations when applied to the stone Buddha statue, the study object, whose joining surface is nonstructured and complicated.

Therefore, in this study, an alternative model for the original and virtual restoration models were 3D printed to minimize physical contact with the original stone statue. In particular, the 3D-printed alternative model was greatly useful for identifying the problems of the virtual restoration model by verifying the design mockup in real time, thus overcoming the physical constraints that did not allow assembly with the original statue. Moreover, the virtual restoration model was 3D printed and used for heuristic-based assembly simulation, thereby intuitively completing the restoration of the missing parts. This study serves as a remarkable case to shift toward the use of the noncontact digital method in restoration. It can also prevent secondary damage by minimizing physical contact with the original statue. In particular, the heuristic-based assembly simulation helped modify and improve the restoration plan and enhanced the restoration reliability, as the assembly result was intuitively reviewed and assessed.

Recently, several solutions for digital virtual assembly have been widely explored, mainly focused on the development of automatic strategies for matching fragments [64,65,66], measure of the surface profile [67], fusion of multiple features [63], and alignment using bridges [68]. Thus, state-of-the-art approaches have their weaknesses and strengths according to the application aims. The stone Buddha statue was not complex and had few fragments. In particular, most assembly parts were created through Boolean operation based on the original statue. Therefore, automatic and semiautomatic approaches for assembly are not important issues in this study. However, the development of an algorithm that could be used to modify the modeling of the interference between fractured surfaces must significantly reduce the frequency of design mockup assembly, and the whole process will be accomplished in a digital virtual environment.

Conclusions

1. In this study, digital recording of the original statue was performed using 3D scanning of the physically damaged stone Buddha statue and digital virtual restoration of the missing parts using a haptic modeling system. Moreover, the missing parts were restored using 3D printing technology after design mockup creation and assembly simulation.

2. The 3D model of the stone Buddha statue was created by converging the fixed high-precision scanning of the exterior and the handheld mid-precision scanning of the interior excavated hole. The 3D model showed a high resolution with an average point density of 0.2 mm. In particular, the 3D scanning result of the cone-shaped excavated hole revealed that two chisels, whose sizes were 10.5 and 15.4 mm on average, were used to carve the hole.

3. Using a 3D deterioration map of the stone Buddha statue, the area of the missing parts was measured to be 400.1 cm2 (5.5% of the total) of the total area (7,250 cm2). The restoration was performed only for 64.2% (257.1 cm2) of the total area of the missing parts, for which symmetry modeling was applicable or original forms could be estimated based on historical research. The restoration objects included four parts: the head, surrounding area of the Baekho, right ear, and right eye.

4. The virtual restoration of the missing parts of the stone Buddha statue was performed using a haptic modeling system: first, the location of the original fragments was determined; second, a reference model was selected and symmetry modeling was conducted; finally, the virtual restoration model was modified and improved, and estimation modeling through historical research and description of the outer shape was also used.

5. The created virtual restoration model was verified for its heuristic-based assembly suitability by design mockup printing and digital–analog simulation. Accordingly, the initial virtual restoration model decreased in volume by 5.2%. Moreover, the removed surface owing to the interference effect between the virtual restoration model and final design mockup was 33.4%, and their RMS was calculated to be 0.67 mm.

6. Photopolymerization 3D printing technology was used for the actual restoration of the stone Buddha statue, and considering the surface roughness, the material layer thickness was set at 0.10 mm. In addition, the surface of the printed output was colored to prevent yellowing, and the final restoration was completed by joining the missing parts of the statue.

7. This study serves as a remarkable case to shift from the traditional manual-contact method to the noncontact digital method in the restoration of artifacts, and heuristic-based assembly simulation helps in modifying and improving the restoration plan and enhances the restoration reliability. In the future, it is assumed that digital–analog-based restoration technology will be extensively used in the conservation of cultural artifacts.