Using Pix2Pix to Achieve the Spatial Refinement and Transformation of Taihu Stone

Abstract

Under the impact of globalization, the transformation of traditional architectural space is particularly important for the development of local architecture. As an important spatial component of traditional gardens, Taihu stone has the image characteristics of “thin, wrinkled, leaky and transparent”. The “transparency” and “ leaky” of Taihu stone reflect the connectivity and irregularity of the holes of Taihu stone, which are in line with the ideas of flowing space and transparency in contemporary architectural design. However, there are relatively few theoretical studies on the spatial analysis and design transformation of Taihu stone. The Pix2Pix model extracts the 3D spatial variation pattern by learning the variation pattern between two adjacent slices of Taihu stone. The trained Pix2Pix model can generate a series of continuous spatial sections with the spatial variation pattern of Taihu stone. Finally, the 2D sections are transformed into 3D building volumes to complete the spatial translation of Taihu stone in contemporary architectural design. In addition, this paper also provides a new idea for machine learning to master the continuous 3D spatial change pattern.

1 Introduction

After entering the twenty-first century, with the continuous development and construction of cities, Chinese urban architecture style is losing its own cultural personality. The reason is that the local architectural culture has lost its own individuality in the modern architectural design trend. Thus, the translation of local architectural culture in contemporary architectural design is particularly important for the inheritance and development of local architectural culture in contemporary times.

Classical gardens play an important role in traditional Chinese architectural culture. Taihu stone is an important spatial component in classical gardens, a sculptural language in Chinese gardening art, and an aesthetic expression of Eastern philosophical concepts. Traditional Chinese gardens are known for the subtlety of “though made by man, just like opening from heaven”, and their space is therefore ambiguous and unqualified. The interior space of Taihu stone is a representative of this kind of space, and its unrepeatability and irreproducibility make it very precious. Taihu stone has the dual properties of building material and space, so this paper focuses on the transformation of Taihu stone into a design element of architectural space. However, the complex spatial relationship of Taihu stone makes the translation in contemporary architectural design a challenge.

The transformation of the space of Taihu stone in contemporary architectural design is of great importance. On a cultural level, the spatial translation of Taihu stone can advance the establishment of Chinese architectural systems in contemporary era. At the level of contemporary architectural design, the transformation of Taihu stone can bring more creative ideas to contemporary architectural design. At the same time the architecture transformed by the space of Taihu stone can be improved in performance. The reason is that the internal spaces of the building are connected to each other, which will bring better light and ventilation to the building. By this method, building energy consumption can be reduced.

With the rapid development of artificial intelligence technology, Generative adversarial network (GAN) has become a popular research direction in artificial intelligence, and the basic idea of GAN is derived from the two-person zero-sum game of game theory. The purpose is to estimate the potential distribution of complex data samples and generate new data samples. In this paper, we try to extract the logical relations of the complex space of Taihu stone with the help of GAN, so as to generate the architecture with the spatial change pattern of Taihu stone. The translation of the space of Taihu stone in contemporary times is accomplished through this way.

2 Background

In previous studies, research has focused on traditional gardening techniques and appreciation of Taihu stones. Ji [4] in “The Craft of Gardens” introduced the reasons for the formation of Taihu stones, materials and their role in the garden. Li [5] in “The idle feeling is sent occasionally” advocated the appreciation of Taihu stones in terms of “translucency”, “thinness” and “leakiness”. Feng et al. [2] in “Environmental Data-Driven Performance-Based Topological Optimisation for Morphology Evolution of Artificial Taihu Stone” presented a combination of CFD and BESO algorithm to topologically optimize the generation of Taihu stones. These studies have contributed to the development of gardening, especially in the selection of stone strategies for gardening and the creation of different spatial effects. However, these works rarely address the quantification of the internal space of Taihu stones and the transformation of spatial design.

For complex spatial logic relationships such as Taihu stone, its spatial distribution pattern can be extracted with the help of machine learning. Many recent studies have applied GAN models to layout generation and demonstrated that GAN can quickly grasp and generate complex spatial layouts. Huang and Zheng [3] proposed to use GAN to recognize and generate apartment floor plans. In the field of 3D machine learning, Zheng et al. [6] put forward to cut the 3D model into plans, then perform style transfer with the given style image and finally re-stack it into a 3D model. Del Campo et al. [1] proposed to express the 3D model as a 2D depth map, train it by CNN and style transfer, and then express the generated results back to the 3D state. However, the current research on 3D machine learning is mainly in the field of style transfer, and there is fewer research in 3D continuous space sequences.

From the above studies, it can be seen that currently in the field of machine learning mainly contains 2D machine learning and 3D machine learning. For the machine learning of such 3D spatial variation pattern of Taihu stone, this paper tries to propose a new idea to solve it. In this paper, the 3D Taihu stone model is extracted into multiple slices in sequence, and two adjacent slices are used as a set of original training samples. The Pix2Pix model obtains the overall 3D spatial change pattern by analyzing the variation trend of adjacent slices in each group of samples.

3 Methodology

The main process of the experiment to extract the internal spatial variation pattern of Taihu stone by Pix2Pix model is as follows (Fig. 1):

• (1) Dataset establishment. Firstly, we selected the images of Taihu stone with “transparent” and “leaky” characteristics from the web. Then, the 2D Taihu stone images are converted into 3D Taihu stone models by running Rhino and Grasshopper plug-ins. The profile slices extracted from the 3D Taihu stone model are served as the training dataset.

• (2) Sample processing and labeling. Firstly, the spatially different elements in the samples are labeled with different colors, and then the two adjacent Taihu stone slices that have been labeled are used as a set of training samples.

• (3) Training and testing. This experiment was conducted with a total of 520 sets of samples, including 500 sets for training and 20 sets for testing.

• (4) Generation of architecture. By inputting one section, the trained Pix2Pix model is able to generate a series of consecutive sections with the spatial relationship of the Taihu stone. Finally, all the sections are combined to generate the architecture.

• (5) Experimental evaluation and analysis. The experimental model is evaluated by the generation effect of the test set and the spatial effect of the generated 3D architecture.

  Fig. 1.

  

  Workflow of research.

3.1 Network Architecture

The traditional GAN consists of two parts, Generator and Discriminator. The Generator is designed to generate samples and the Discriminator is used to determine the authenticity of this generated sample.

During the training process, the goal of Generator is to generate as realistic images as possible to deceive Discriminator, whose goal is to try to distinguish the images generated by Generator from the real ones. As the two networks play against each other, both networks become more and more capable. The images generated by Generator become more and more like real images, and Discriminator becomes more and more capable of judging the authenticity of the images (Fig. 2). The Pix2Pix model adopted in this experiment is based on GAN to implement image-to-image translation. Therefore, we can generate very realistic images with generator once the Pix2Pix model is trained.

Fig. 2.

The network of architecture of Pix2Pix.

3.2 Dataset

Taihu stone has the image characteristics of “thin, wrinkled, leaky and transparent”, among which the “leaky and transparent” characteristics are more relevant to the contemporary architectural space. Therefore, in order to make the sections generated by the trained model have a better spatial effect, we selected the images of Taihu stones with obvious “leaky and transparent” features as the original samples.

3.2.1 Sample Processing

In this work, we have collected a total of 12 2D images of Taihu stone as the original samples. By running Rhino and Grasshopper plug-ins, the 2D original samples are converted into 3D Taihu stone models. Then, the 3D Taihu stone models are transformed into multiple sequential 2D profile slices (Fig. 3). The two adjacent slices are used as a set of original training samples.

Fig. 3.

Preparation of original samples of Taihu stone, left: 3D Taihu stone model, right: Grouped profile slices of the Taihu stone model.

3.2.2 Augmentation

For the purpose of better grasping the spatial distribution pattern of the Taihu stone by Pix2Pix model, 104 sets of original samples are rotated and mirrored, etc. Finally, 520 sets of samples are obtained, of which 500 sets of samples for training and 20 sets of samples for testing.

3.3 Labelling Based on Analysis

By analyzing the spatial elements of the Taihu stone, the Taihu stone profile is divided into three components: “transparent” space, “leaky” space and solid space. In this research, these three spatial elements are labeled with different colors in the sample processing (Fig. 4). The adjacent slices that are labeled are used as a set of training samples (Fig. 5).

Fig. 4.

Labeling based on spatial analysis, left: original slice, middle: labelled slice, right: labelling rule.

Fig. 5.

Adjacent slices as a set of training samples.

4 Training and Analysis

4.1 Training Process

The data set of this experiment contains a total of 520 sets, 500 of which are applied for training and 20 for testing. The training process of the experiments is that the Pix2Pix model generates the Taihu stone profiles by learning the variation pattern of spatial elements between each group of samples. A total of 600 iterations are conducted during the experiment. From the training results (Fig. 6), it can be seen that the boundary of the image generated by the model is clear and the distribution pattern of the hole location is in line with the variation pattern between two adjacent slice samples. The “leakage” space is centered on the “permeability” space, and the generated profile boundary are consistent with the input samples.

Fig. 6.

The part process of the training.

From the generated results of the test experiments (Fig. 7), it can be seen that most of the generated profile variation distributions follow the variation pattern of the hole distribution in adjacent slices. The difference between the test sample generation results lies in the magnitude of the profile hole variation. The generation results of the test experiments can prove that the Pix2Pix model has mastered the change pattern of adjacent slices in each group.

Fig. 7.

The results of the testing training.

4.2 Generation of Continuous 2D Sections

In order to generate continuous architectural sections, the first section needs to be input and the trained Pix2Pix model can generate a second section with the spatial variation pattern of the Taihu stone. Then the second section is used as input to generate a third section. Through this method, the generation of all continuous architectural sections is completed (Fig. 8).

Fig. 8.

The process of generating architectural sections.

The continuous 2D sectional images generated by Pix2Pix cannot be applied directly to architectural further design. Therefore, it is necessary to convert the 2D image into a modeling object in architectural language. The Rhino and Grasshopper plug-ins are applied to extract the boundaries of the different elements of the profile based on different colors, and then the boundaries are converted from pixels to curves (Fig. 9). The transformed sections are arranged equidistantly in sequential order (Fig. 10).

Fig. 9.

Transformation of image to geometry.

Fig. 10.

Generation of multiple architectural sections.

4.3 Transformation of 2D Sections into 3D Model

By converting the building sectional curves into faces with Rhinoceros, and then extruding the faces in the same direction, the final 3D building volume is obtained. During the transformation process, the solid boundary curves are transformed into solid spaces, and the “permeable” and “leaky” spaces are transformed into the void spaces of the building. From the generated building (Fig. 11), it can be seen that the solid spaces in the building are interconnected and intricately changed, which is in accordance with the characteristic changes of “leakage” and “permeability” of Taihu Stone. At the same time, the solid space of the generated 3D architectural volume has similar qualities of flowing space and transparency in contemporary architectural design. Thus, the experimental results suggest that the transformation of complex spaces of Taihu stone in contemporary architectural design is achieved by this method.

Fig. 11.

Architecture with the spatial pattern of Taihu stone.

4.4 Result Analysis

In terms of the generated results of individual 2D profiles, it can be found that the generated section variation pattern conforms to the spatial variation pattern of the Taihu stone slices. The boundary contours of the generated 2D sections are consistent with the first input section. The “leaky” space is centered on the “transparent” space. As the sections are generated iteratively, the “leaky” spaces are sometimes connected and then separated from each other.

Regarding the results of continuous 2D section generations, the similarity of adjacent sections is an important indicator of the generated results. By analyzing the continuous 2D sections, it can be found that the overlapping area of adjacent sections can reach 64.25% to 69.94%. This suggests that the variation between sections maintains a well continuity during the generation of continuous sections. At the same time, about 30% of the generated section can vary along a certain trend, which avoids the overfitting of the generated results (Figs. 12 and 13).

Fig. 12.

Similarity analysis of adjacent sections

Fig. 13.

Spatial continuity analysis of adjacent sections

The results of the 3D model generation indicate that Pix2Pix is able to grasp the variation pattern of the “permeable” and “leaky” space of the 3D Taihu stone. The generated 3D void space model (Fig. 14) shows that the internal space of the building changes from the original separated state to the connected state, and then slowly separates. This is in line with Li Yu’s statement in “The idle feeling is sent occasionally” that “the beauty of a mountain or a rock is in the three words: transparent, leaky and thin. This leads to the other, the other leads to this, if there is a road feasible, the so-called transparent; there are eyes on the stone, exquisite on all sides, the so-called leaky.”¹ The effect of interconnection between building spaces maintains a high degree of consistency with his idea. Combined with the generated 3D building volumes, it can be illustrated that the Pix2Pix model can master the changing patterns of 3D complex spaces by learning 2D continuous profiles. In a word, the results of this experiment suggest that the transformation of the space of Taihu stone in contemporary architectural design has been realized.

Fig. 14.

Architectural solid space (left) and void space (right).

5 Conclusion and Discussion

This paper is based on machine learning to extract and grasp the spatial variation patterns of 3D Taihu stone. This paper improves the effect of machine learning to master the change pattern of the 3D model by improving the labeling method and converting the 3D model into continuous 2D training samples. PIx2Pix achieves the extraction of complex spatial change pattern of Taihu stone by training 500 sets of samples and testing 20 sets of samples in this study. By analyzing the spatial effects of the generated 2D sections and 3D models, it can be proved that this experiment has accomplished the transformation of the complex space of 3D Taihu stone in contemporary architectural design. At the same time, this research provides new ideas for machine learning to master the 3D space variation law. More importantly, this research provides a new method for the translation of traditional Chinese architectural space in contemporary architectural design.

Of course, this study still has some limitations. First, after the Pix2Pix model is trained, the process of generating 2D sections is tedious and complicated. By inputting one section, only the corresponding next section can be generated. It needs to be iterated continuously by consuming more time to generate a series of consecutive sections. In the future, this problem can be solved by improving the neural network architecture as well as the structure of the training samples.