1 Introduction

3D modelling of man-made objects is increasingly being used in the cultural heritage (CH) sector for its documentation, dissemination, and enhancement. Among others, technological tools and interactive solutions have the ability to create fascination far beyond other traditional solutions, such as a tourist brochure or any other printed material. Additionally, 3D models representing CH can be accessed through a variety of solutions, which include interactive 3D worlds, serious games, mixed reality, virtual reality or augmented reality, which can be reported as the main tools for cultural heritage dissemination in the near future due to their technological impact in viewers’ attention [3, 5, 18].

Historical fabrics are fragile heritage, and some historical weaving techniques that require artisanal work are in danger of disappearing, because of a lack of generational replacement. These objects form part of our tangible and intangible heritage, but few actions are taking place in order to properly document them.

Traditionally, weaves and weaving techniques have been documented making use of a combination of textual description of the techniques and 2D graphics, sometimes represented in matrix-like forms [12]. In such graphics, the representation of non-complex techniques and weaves is straightforward. For instance, a damask is composed of a single layer of weft and warp yarns, and the pictorial and the background information are made of the same weave but inverted. However, more complex techniques (e.g. espolín with damassé ground) are difficult to represent in 2D forms. Therefore, in order to understand how a fabric is formed, an interpretation is needed, requiring a minimum understanding of technical drawings and the weaving process. In this sense, we believe that the representation by means of 3D interactive graphics can bring added value to the documentation and understanding of such tangible and intangible heritage: a 3D model holds an extra dimension, so the fabric (also a 3D object) can be fully represented, avoiding the need to interpret representations in 2D and textual information.

Developed under the H2020 SILKNOW project [24], Virtual Loom (VL) is a tool that embeds historical weaves and weaving techniques in order to produce 3D models of fabrics, given an input image (of a fabric or a technical drawing). VL’s graphical user interface (GUI) is designed in such a way that is highly intuitive, guiding users through the process and making it possible for them to produce the models without any previous knowledge about textiles and/or old techniques. The definition and modelling of such techniques in VL has been possible thanks to close collaboration of experts in computer graphics, art history and old textiles.

A preliminary work describing the workflow of VL for subtracting the design of a textile was presented in Portalés et al. [25]. In Gaitán et al. [10], some basic 3D models of a weaving technique were also shown, and in Gaitán et al. [11], we explained how to apply the similarity functionality in the VL, to restore the design of traditional silk fabrics, also obtaining 3D representations. Therefore, these works either present a preliminary design of the VL or are focused in showing a very specific aspect. In this paper, we present the complete workflow and GUI of the VL, presenting them in detail, from design subtraction, to the modelling of yarns, description of weaving techniques, etc. We also provide results for all the considered weaving techniques by making use of different yarns. In the current version of Virtual Loom standalone version [33], we have considered 39 weaves (tabby, twill, satin and variations of them) and five traditional weaving techniques (damask/damassé, damassé with metallic weft, lampas, espolín and espolín with damassé ground). Additionally, in order to expand VL to a larger audience as well as the creative industries, we have introduced a weaving technique called “freestyle”, from which all kinds of implemented weaves and modelled yarns can be chosen in order to experiment with new forms of generating 3D models for a given design, without the restrictions of such techniques. For instance, damasks are not made of metallic yarns, but with the freestyle option in VL, one can build a damask-like 3D representation of a fabric with a gold yarn.

For all the techniques, users can also experiment alternative 3D representations by changing the color of the yarns, increasing/decreasing the number of yarns, increasing/decreasing the number of areas (of the design), mirroring an image (with a “symmetry” function) to complete a design, etc., always respecting the restrictions for old techniques. The generated 3D models can be downloaded, as well as the design after the posterization of the image. Additionally, VL allows users to upload any image. Therefore, they can upload their own design in order to virtually weave it and produce a 3D model that could be used in a 3D printer.

Other textile-related software that involve 3D representations at the yarn level, have the purpose to weave a real fabric based on a given design. Differently, in VL, the design is subtracted given an image of a real textile, and the final product is a 3D representation. Additionally, VL embeds the historical weaves and weaving techniques, which are not always available in other software, as some old weaving techniques cannot be reproduced in mechanical looms, because of their artisanal implications.

The novelty of the work presented in this paper is fourfold: Firstly, VL is able to automatically subtract the original design of a fabric, including cases where the fabric is damaged; secondly, it embeds old weaves and weaving techniques, allowing users to use them without previous knowledge; thirdly, thanks to interactive 3D graphics, it brings new forms of representing historical fabrics, at the yarn level; finally, it is very versatile: e.g. it allows weaving new designs with old techniques, weaving old designs with a freestyle, changing the weaves and/or weaving technique for a given old fabric, changing the type and color of yarns, etc.

This document is organized as follows: firstly, we give an overview of the state of the art, explaining the novelty of VL in relation to other existing textile-based software. Then, we explain the design and implementation of VL, which includes the strategy for image processing, the 3D modelling of yarns, weaves and weaving techniques and the GUI. In Section 4, we show examples and results after processing some images with VL. Finally, a discussion followed by conclusions are given.

2 State of the art

As explained above, in this paper we propose the representation of historical fabrics by means of 3D interactive graphics. In fact, 3D representations are widely used in other areas related to Cultural Heritage. For instance, 3D modelling and reconstruction has been used in archaeological sites to reconstruct the non-existing elements [23, 26]; to reconstruct the current shape of artifacts, including paintings [14], ceramics [6], statues [4, 6], etc., or to provide enhanced experiences at the museum level and cultural heritage sites by means of augmented reality technology [13, 21, 31].

Related to the textile area, some authors have already dealt with 3D representation fabrics and clothes [7, 9, 16, 17], although not focused on historical fabrics. For instance, Etzmuss et al. [9] presents an efficient model for physical simulation of clothes, obtaining realistic clothes movement at 25 frames per second on a P4 1700MHz. The works presented in Jiang and Chen [16] and Cirio et al. [7] propose a representation of knitted cloth at the yarn level. In Cirio et al. [7] yarn-yarn contacts are treated as persistent, avoiding expensive contact handling altogether. In VL, we are also representing textiles at yarn level, but we are not interested in dealing with their physical movement.

On the other hand, CAD authoring tools (Computed Aided Design) have also been applied to the field of textile design. Nawab et al. [22] makes a deep review of CAD tools, which are classified by the purpose of the design: weave new textile structures, aesthetic or artistic design, analyze the physical behavior of a fabric, etc. In this sense, our VL could be considered as a CAD tool but with a great difference in comparison to other existing solutions, as the objective is not to create new textiles but to better understand historical textiles and fabrics. In VL we have two main purposes: virtually weaving historical fabrics to get a better understanding of their structure and creating realistic views of the yarns.

Related to the first purpose (i.e., virtually weaving historical fabrics), several CAD tools can be mentioned. For instance, WiseTex software suite [19, 20] is an integrated tool for modelling textiles and textile composites, embedding a variety of software modules for the internal geometry, the virtual reality visualization, the resistance of textiles to tension, shear and compaction, etc. Another example is TexGen [30] which allows the modelling of the geometry of textile structures with a variety of properties, including textile mechanics, permeability and composite mechanical behavior. ScotWeave [27] and 35D Weave [1] are software suites, to design and fabric new textiles. In these CAD-CAM software packages, 3D representations at yarn level can be done, and also the final aspect of the fabric can be simulated. Also, some of them use matrix-based models to describe the structural hierarchy, as we do in VL.

VL differs from other textile-based CAD tools in two main aspects: the final users to which the tool is addressed (i.e. the purpose of the tool) and the realistic representation of yarns when generating the 3D models. All the above-mentioned CAD suites have in common that they are intended for textile experts, so new textile structures can be created using low-level parameters with the purpose of weaving a real textile at the end of the design phase. However, in VL, the workflow is reversed, because we allow both textile experts and non-experts to create the virtual fabrics, starting with a photograph of the real historical fabric, or any other photograph that embeds a design (to be subtracted). Low-level parameters can also be configured, but the whole process can be achieved using pre-defined settings describing historical yarns and textile structures. The other great difference is that VL offers a realistic view at the yarn level, focusing on silk yarns, while representing the structure of the fabric. To the best of our knowledge, all the available textile-based CAD suites use a schematic representation of the yarns, while showing a small portion of the fabric structure. In VL, users can zoom-in a fabric, to inspect the structure of yarns while keeping a realistic view of them.

In a nutshell, it can be said that both VL and other textile-related CAD tools involve similar objects (fabric, design, 3D model/yarns), but the final product, and mainly the workflow, are different. In VL, the input data is a photograph of a real fabric, from which the textile is subtracted, and the final product is a realistic 3D model, at the yarn level. In other CAD tools, the input data is a design, from which a basic 3D model can be reproduced, and the final product is a real fabric (or a file for the production of a fabric). This is schematized in Fig. 1.

Fig. 1
figure 1

Overall concept of Virtual Loom in comparison with other existing tools

Full size image

Related to the realistic aspect of the individual yarns, Zhao et al. [35] presents a method to procedurally create yarns, including fibers. In this model, the configuration parameters are automatically measured from real yarns images, so they can be reconstructed. This approach generates very realistic individual yarns and fibers, but the geometry is too complex to create textiles. Wu and Yuksel [34] present a GPU based technique to render full cloth pieces, at fiber level. Different levels of detail are proposed, in order to optimize the necessary geometry to render each fiber. In low-levels of detail, the geometry is simplified, so the fibers are simulated using textures in pixel shaders (height-maps, self-occlusion maps, normal maps, etc.). In VL a similar approach is implemented, so yarns’ geometry is simplified, and a custom pixel shader has been developed to keep the realistic aspect of the yarns, including very complex yarns like the ones with a metallic cover.

3 Design and implementation of Virtual Loom

In this section, we describe the design and implementation of VL, defining all the technologies involved in its development, and providing screenshots of the current state of the tool.

3.1 Overview

Figure 2 describes a simplified workflow of the tool, where the input parameters for Virtual Loom are an image and a related information (e.g. weaving technique) that describe a given fabric. This information can be provided by users, or obtained from ADASilk [2], an ontological system that is another important result of the SILKNOW project. ADASilk allows users to select different historical fabrics from a database and generate a 3D model in VL using the photography and technical information of the fabric. This information is sent to VL using a json file. Users can maintain or change these parameters in order to either reproduce in 3D the original fabric or represent other possible solutions.

Fig. 2
figure 2

Simplified representation of Virtual Loom workflow

Full size image

Once these parameters are loaded into the tool, users can automatically extract the design by means of image analysis. To that end, users can tweak some parameters and configurations in order to subtract the background of the image and the pictorial part(s). As part of the design, VL offers a variety of yarns, weaves and weaving techniques which can be applied (with some restrictions depending on the techniques, as explained in Section 3.6). The generated image, as well as the selected parameters (e.g. yarns, weave), are then used to produce a 3D interactive representation of the fabric, which can also be exported in STL form.

VL has been developed using the Unity3D engine. Unity3D provides a complete environment where we have been able to develop the user interface and the 3D representation of a given fabric. It also allows us to unify the development process for different platforms (standalone and WebGL). The Image processing is performed using the OpenCV plugin for Unity. In this way, everything is performed using the Unity3D engine. In the following sub-sections, we give more technical details of the relevant steps included in the VL.

3.2 Input data

Virtual Loom is both integrated as part of ADASilk [2] and available as a standalone application (2020e). ADASilk (Advanced Data Analysis for Silk heritage) is the exploratory search engine of the SILKNOW’s knowledge graph that contains nearly 40,000 fabric entries with images and other relevant information describing them (e.g. production place, production timespan, material, technique, etc). The documented objects can be reconstructed in 3D in VL. To that end, ADASilk sends an image of the textile and other relevant information in order to suggest to the user how to create the 3D model. The following attributes are sent in json format:

  • imgUri: The URI of the image to be processed. This information is mandatory.

  • dimension: Dimension in cm of the fabric. This information is optional. If not available, the user can manually change the aspect ratio for a given image.

  • weavingTechnique: Name of the weaving technique originally used when fabric was created. This is also optional, as the user can select a weaving technique.

  • weave: Name of the weave originally used when fabric was created (optional).

  • backgroundColor: Color used as background in the picture of the fabric (optional).

  • materials: Array of the different materials used originally when fabric was created (optional).

All the optional fields are sent when the knowledge graph has the information. The following json file is an example of the information sent to Virtual Loom.

figure a

From ADASilk, users can download json files to be used in the standalone version. Additionally, in the standalone version, users can use images uploaded from their computer, so non-historical fabrics or other objects can also be represented making use of VL.

3.3 Strategy to process images

In this section, we explain the strategy to process images in VL, which is schematized in the graph shown in Fig. 3. Below, the graph is explained in detail, following the numbered steps in this figure:

  1. 1.

    An image is loaded into VL. It might have some perspective distortion.

  2. 2.

    The user interactively selects four points, corresponding to the corners of the area of interest. While the user sets the points, the image is rectified, correcting the perspective distortion in real time.

  3. 3.

    The user has the possibility of changing the aspect ratio of the image and adding mirroring effects.

  4. 4.

    The image is analyze d in order to obtain different areas (image analysis module in Fig. 2). To this end, we use the kmeans method implementation from OpenCV: Firstly, the user selects the number of colors or pictorial zones. Then, as a result, VL produces a posterized image. In the example given in Fig. 3, the resulting posterized image would contain four plain colors (i.e. four areas).

  5. 5.

    Black and white images are generated, discerning between the background and the pictorial part. These images are not visible for the users, as they are used for internal analysis. Specifically, each one will be used to virtually weave each type of yarn, in other words, to create the mathematical description of the curves that the yarn’s geometry will follow. Depending on the weaving method, different images are derived:

  1. a.

    For damask: In this case, the background image is the inverse image of the pictorial part. Therefore, only two images are generated. It has to be taken into account that the pictorial part in this case is a plain color.

  2. b.

    For damassé with metallic weft or lampas: the background image is the whole image. The pictorial part is embedded in a single image, as it is formed by a single color.

  3. c.

    For espolín: the background image is the whole image. For each area (each color), one image is generated.

  4. d.

    For espolín with damassé ground: In this case, the background image is given the same treatment as for damask (5a). In the given example in Fig. 3, the orange area is considered as part of the pictorial part of the background. For the rest of the pictorial parts, one image is generated for each area.

Fig. 3
figure 3

Schema of the image processing in VL. It can be seen that, depending on the weaving technique, different images are derived

Full size image

3.4 3D modelling of yarns

We have integrated a variety of yarns which are needed in order to produce the 3D models with the different weaving techniques. We have focused our design on representing silk, cotton and metallic yarns, which were the most commonly used in the historical fabrics which we are considering for SILKNOW (15th to 19th centuries). In order to extract the main properties that can characterize a yarn in such historical silk fabrics, we have performed an optical study (Fig. 4a) on four fabrics, which are reproductions of historical fabrics, weaved in the same way and making use of similar yarns. This study makes use of an Optical Microscopy (LM), a Scanning Electron Microscope (SEM) and Inverted Optical Microscope (LMi) and is available at Vázquez de Ágredos Pascual and Rojo Iranzo [32]. We have determined that for our model, yarns can vary in their section, thickness, texture, composition, color, etc. For instance, we have observed that usually yarns belonging to the pictorial parts are thicker than those of the weft or the warp. Some other yarns are composed of threads with different compositions, even mixing silk with metals. The following properties are considered (Fig. 4b):

  • Section and thickness: Yarns are usually composed of a group of threads distributed longitudinally inside a cylinder. This cylinder can be characterized by its radius (thickness). Depending on the type of yarn, it can be more or less compressed in the fabric, and therefore the section has two radii.

  • Composition. Yarns are composed of a group of threads, while each thread is composed of two baves. In VL, composition is defined by the number of threads in a yarn.

  • Torsion: Each type of yarn can be twisted several times. Torsion can be clockwise (Z) or counterclockwise (S). VL supports both types, using a positive value for S torsion and a negative one for Z torsion.

  • Color: Color is one of the parameters defined by the user. Color can be chosen from a complete RGB palette. In metallic yarns, the color of the core is restricted to three possibilities, according to the required effect (e.g. brown to simulate aged gold).

  • Detail textures: In order to reduce and simplify geometry, we use textures to represent small details of the visual aspect of the yarn. The texture can modify the normal surface vector to simulate the baves of each thread. A procedural texture, generated from the rest of parameters, is combined with the detail texture to represent a yarn formed by multiple threads (silk or cotton).

  • Metallicity: Yarns have different responses to light according to their type. Also, they have different visual responses according to the direction of observation. This is why it is very important to implement an anisotropic reflection model, commonly used in brushed materials or human hair. Silk yarns have a higher metallicity value than cotton. Metallic yarns have the highest value, to represent surface reflections of the environment.

Fig. 4
figure 4

3D modelling of yarns in VL, where: a Real yarns from the optical study (top) and virtual yarns considered in VL (bottom); b Properties of yarns considered in VL, including those for the metallic yarns

Full size image

Metallic yarns have two extra parameters (Fig. 4b):

  • Metallic band: defines the width of the metallic band that rolls around the inner yarn (the core). It is defined as a normalized value. 0 represents no metallic band and 1 represents a metallic band that completely occludes the core.

  • Torsion of the band: In this case, the metallic torsion is the number of turns per unit length that the band wraps around the core.

These properties allow us to graphically represent different types of yarns in Unity3D (Table 1). For their 3D representation, simple geometries have been taken into consideration to allow VL to run on a wide variety of systems. Yarns are implemented as an interpolated 3D curve, making use of Hermite interpolation, which permits enough accurate control of the curve with a low number of control points. In VL, the generated geometry is extruded through the curve using 4 samples per control point. Each sample is the section of the yarn implemented as a 3D polygon with 6 vertices. The thickness of the section varies according to a parameter automatically calculated for each control point, so we can simulate the compressiveness of the yarn near the intersection between the warp and weft. The other properties are implemented in a fragment shader that allows to simulate graphically the number of threads, torsion, color, texture detail, metallicity and the metallic band and its own parameters.

Table 1 Characteristics of the yarns implemented in VL v.091b. Grid is 0.1 mm
Full size table

In the current version of VL, we have integrated a total of six yarns, which are is schematized in Table 1, according to the aforementioned properties. The field “name” refers to the name we have given them in Virtual Loom’s GUI. In the case of silk yarns, the number included in the name refers to the number of threads that compose a yarn, e.g. silk-3 is a yarn composed of three threads.

3.5 Weaves

In VL, the three basic weaves are considered, i.e., tabby, twill and satin, and also a number of variations. In total, we have considered 39 weaves. Variations of tabby include Gros de Tours, Lousine, Natté and extended tabby. Variations of twill included in VL are the Roman twill and the Batavia twill. In the case of satin, we consider a variety of regular and irregular satin, and the queen satin. For more information about these techniques, the reader is referred to the SILKNOW - Virtual Loom [28].

These weaves are stored in VL in the form of images, with pixel values being either black or white. An example is shown in Fig. 5. For instance, queen satin is an image of 6 × 6 pixels, 6 of them being black and the rest white. This is the usual way that weaves are represented in 2D graphics, in the textile domain. In VL, these textures work like tiles where black and white color identifies when the weft is over the warp (white) and vice versa. These tiles are combined in different ways (defined by the weaving technique) with the pictorial part of the fabric, which are added to the fabric as a new weft with the corresponding color. All this information is needed to generate the yarns in the 3D model.

3.6 Weaving techniques

The weaving techniques considered so far in VL are damask/damassé, damassé with metallic weft, lampas, espolín and espolín with damassé ground. The definitions of these techniques are included in the SILKNOW’s Thesaurus [29]. In the following lines, the restrictions considered in VL are explained, for each of the techniques.

Damask and damassé

This weaving technique consist of a satin weave, and the pictorial part is made by inverting the weave pattern. As with satin weaves, we have introduced the basic one (regular satin - see Fig. 5) as the default weave; the user can change it for any of the rest of the implemented regular satins, or the queen satin. There is only one layer of warp yarns and one layer of weft yarns. In a damask, the weft and warp yarns are of the same color, as shown in Fig. 6. There are other fabrics that, geometrically speaking, are equal to damask, but have different colors for the warp and the weft yarns. These fabrics are referred to as “damassé” instead of “damask”. In Virtual Loom, we have considered the option to change the colors of either the weft or warp, in order to weave a damassé. However, we have not included damassé as a different technique from damask, but as its variation, and therefore in VL they are included in a single weaving technique.

Fig. 5
figure 5

Examples of weaves available in VL

Full size image
Fig. 6
figure 6

Example of a damask (from Espolines de Garín 1820 [8]): a detail; b complete fabric

Full size image

Damassé with metallic weft

The damassé with metallic weft has a background formed by a satin weave (Fig. 7), which is composed of a layer of weft yarns and a layer of warp yarns, and this background goes through the entire fabric. In the fabric shown in Fig. 7, these layers correspond to grey vertical and horizontal yarns. Regular satin is considered as the default weave; the user can change this for any of the rest of the implemented regular satins or the queen satin. The pictorial part is made up of an additional layer of metallic (silver or gold) weft yarns. This layer is visible either on the face or in the reverse side of the fabric, creating the “damassé” effect, as the weft yarns go from one side to the other side of the fabric. The geometry of this weaving technique differs from a damask, although the fabric might look similar if not observed with a microscope.

Fig. 7
figure 7

Example of a damassé with metallic weft (from Espolines de Garín 1820 [8]): a detail; b complete fabric

Full size image

Lampas

For the 3D generation of a lampas in VL, we consider that the background is formed by the basic weave tabby, which is composed of a layer of weft yarns and a layer of warp yarns, and that this background goes through the entire fabric. In the fabric example shown in Fig. 8, these layers correspond to green vertical and horizontal yarns. The pictorial part is made of an additional layer of weft yarns, the yellow colored ones in Fig. 8. This additional layer goes up and down, being visible either on the face or on the reverse side of the fabric, and the weft yarns go from one side to the other side of the fabric (similar to damassé with metallic weft). Therefore, the geometry of a lampas is different from a damask or the damassé with metallic weft, although the fabric might look similar if not observed with a microscope.

Fig. 8
figure 8

Example of a lampas made of two colors (from [15]): a detail; b complete fabric

Full size image

Espolín

In an espolín, the background is uniform throughout the entire fabric and is formed by a satin weave (Fig. 9). As the default, we take the queen satin. The user can choose to use other types of regular satins. The pictorial parts are woven on top of the background, and the weft goes from one side to the other side of the pictorial parts. On the reverse side, the weft yarns are also concentrated in the pictorial part, therefore, they do not go from one side to the other side of the fabric. We have also implemented a binding warp that binds wefts, so the pictorial part is warped at a given step. The user can choose if the drawing has this binding warp or not.

Fig. 9
figure 9

Example of an espolín (from Espolines de Garín 1820 [8]): a detail; b complete fabric

Full size image

Espolín with damassé ground

This technique is similar to the espolín, with the addition that the background has a pictorial part formed with the damask technique (Fig. 10).

Fig. 10
figure 10

Example of an espolín with damassé ground (from Espolines de Garín 1820 [8]): a detail; b complete fabric

Full size image

Freestyle

As explained in the introduction section, in order to expand VL to larger audiences and the creative industries, we have implemented a weaving technique called “freestyle”, where all types of implemented weaves and modelled yarns can be chosen in order to experiment with new forms of generating 3D models for a given design embedded in an image.

3.7 Graphical user Interface

The GUI has been implemented in order to make the subtraction of designs and the generation of 3D models straightforward, so users without any previous knowledge of weaves and weaving techniques can produce 3D representations of fabrics, thus making it accessible for the general public, too. For instance, for the subtraction of designs the user only needs to indicate the number of different colors (or yarns) that are visible in the picture. The GUI is divided into a set of steps represented as tabs. This defines a sequential workflow, where users have to complete each step before advancing to the next one. These steps are shown in Fig. 11, and further detailed in the following subsections.

Fig. 11
figure 11

Interaction steps of Virtual Loom

Full size image

Load data

In the first step, an image is loaded from a disk (Fig. 12). Images can be in a variety of formats (e.g. jpg, png). We recommend using input images with a minimum resolution of ca. 200 × 200 pixels to produce a good representation of the design embedded in the fabric.

Fig. 12
figure 12

Virtual Loom - load data

Full size image

Image processing

Once an image is loaded, users are presented with a window where they can adjust some parameters in order to process the image from a geometrical point of view. In this step (Fig. 13), a homography can be performed in order to correct the perspective and to select the “best” part of the original picture, or just an area of interest for the user. Users also have different controls to rotate the image (90° left/right); here, it is important to highlight that VL considers that warp yarns follow the vertical direction of the image, while the weft yarns follow the horizontal direction similar to a real loom. Therefore, if the image of a fabric has been taken from another direction, the users need to rotate it. Users also have a control to extend the image with symmetry (horizontal or vertical). This control could be relevant for those cases where the original image of a fabric is not complete. By using symmetry in VL, the whole design can be reproduced, as shown in Fig. 19. In this step, the aspect ratio for the final representation can also be defined.

Fig. 13
figure 13

Virtual Loom - Image processing

Full size image

Technique definition and clusters

correct the image, the next step refers to the weaving technique: if this information is given within the textual input data, the given technique is automatically loaded, otherwise, the user can select it. On this same screen, the image processing by OpenCV to extract the design (background and pictorial parts) is performed. To start with, users must select a number of clusters to separate the incoming image into the different zones that make up the fabric design. Some of the implemented weaving techniques (e.g. damask) have restrictions concerning the number of clusters that can be chosen. With the selected technique and the number of clusters, the tool posterizes the image and separates the incoming image into a “image zone” for each of the clusters. This process also generates one black and white image for each of the generated zones, as schematized in Fig. 3. Therefore, if the user has selected “Espolín” as the technique and 4 clusters (Fig. 14), the tool will posterize the image with 4 colors.

Fig. 14
figure 14

Virtual Loom - Technique Selection and Image posterization

Full size image

Weaves and yarns

The next step in VL is to define the yarns to be used in the 3D representation of the fabric. In Fig. 15, the user interface for this step is shown.

Fig. 15
figure 15

Virtual Loom - Yarn and Weave configuration

Full size image

From the previous step, we get a list of clusters in the posterized image. Each of these clusters will be represented by a yarn. As shown in Fig. 15, we have defined 2 sections: Background zone and Pictorial zone. Users can select which of the generated yarns represent the background of the fabric.

The type of warp yarn is fixed as silk-3 (Table 1). For the rest of the yarns, depending on the weaving technique selected in the previous step (except for freestyle), restrictions in the type of yarns may be applied, as we have taken into account those commonly used in historical fabrics. To represent the colors of the yarns, we take the color produced in the previous step with the posterization. However, as this might not be close to the original color of the yarns (either because of the posterization procedure or because it is not well represented in the original image), we have allowed users to modify it by clicking on the icon representing the “Output Color” (colored squares, to the right of the input colors). These output colors will be the ones mapped to the yarns in the 3D representation.

Users can also select the weave to be used in the 3D fabric representation. Weave types are also restricted by the selected weaving technique, as previously given in Section 3.5. The interface for weave selection is represented in Fig. 5.

Fabric 3D visualization

Once the user has defined all the yarns and the weave configuration, he/she can advance to the next step and generate the final 3D representation, producing a 3D model such as the one shown in Fig. 16.

Fig. 16
figure 16

Virtual Loom - Generated 3D fabric representation

Full size image

Once all the configuration information is defined, VL starts the process of weaving the 3D model. The tool starts by defining the control points of the curves used to represent the yarns, after that 3D geometry is generated following the control points.

The first step is to create all the yarns representing the warp, which in a traditional loom are the ones in the vertical direction. Once all the warp yarns have been created, VL starts to create the weft yarns. Control points for these yarns are positioned based on the selected weaving technique and the background drawing. Once the warp and weft yarns are ready, the last step of the representation generation is to create the pictorial parts. VL needs to generate yarns for each of the pictorial parts extracted from the fabric image by the user. In this case, the control points are positioned based on the drawing and the selected weaving technique.

The user can interact with the 3D fabric by exploring it, moving and bringing the virtual camera closer to the fabric. There is also a control that allows users to increase or decrease the fabric resolution, changing the number of yarns used in the visualization. At this point, users can also export the resulting 3D model to be printed in STL format.

4 Results

4.1 Design subtraction from images

In this section, we show some examples for design subtraction from images, taking into consideration different conditions and the complexity of the designs themselves. The first example is shown in Fig. 17a. It consists of a technical drawing, with a grid background and a pictorial part made up of a single color. The image shown in Fig. 17b is the result of design subtraction with VL, with two clusters. As can be seen, the implemented algorithm is able to correctly eliminate the grid, and thus only the pictorial part is present in Fig. 17b.

Fig. 17
figure 17

Design subtraction of a technical drawing: a input image; b posterized image with two clusters

Full size image

The second example consists of extracting a complete design for a fabric that is incomplete. The original image is shown in Fig. 18a, together with the selection of the area of interest in VL. The image shown in Fig. 18b, is the completed fabric with the symmetry function. The design shown in Fig. 18c is derived by applying two clusters in 19-b.

Fig. 18
figure 18

Example of a damaged damask fabric and the extraction of the complete design using the symmetry function in VL: a the original image and selection of the area of interest; b the mirrored image; c the posterized image with two clusters

Full size image

The next example consists of an espolín with the background having an extended tabby weave. The original image is shown in Fig. 19a, and consists of five areas (red, yellow, green, blue and purple). The next images (Fig. 19b and c) are the processed ones, taking into consideration different numbers of clusters. As can be seen, with only two clusters (Fig. 19b) the design is well subtracted from the background. With five clusters (Fig. 19c), all the colors are separated, although the purple zone is not well defined. In this way, we can find out if the algorithm is able to properly discern the five areas.

Fig. 19
figure 19

Design subtraction of an espolín: a input image (acquired with a conventional scanner); b posterized image with two clusters; c posterized image with five clusters

Full size image

The next example consists of an espolín with the queen satin weave background. The original image is shown in Fig. 20a, and consists of 14 areas (pink, gold, silver, yellow, green, blue, dark blue, purple, magenta, garnet, light orange, orange, dark orange, white). In Fig. 20b-f the posterized images are depicted for different clusters. As can be seen, despite the complexity of the image, the algorithm is able to discern a lot of areas. The design is well subtracted from the background with four clusters (Fig. 20c) or more, as with fewer clusters there are some pictorial parts that are merged with the background. With ten clusters (Fig. 20e), the background is not well defined, as there are some shadows belonging to one area that are considered part of another area. Therefore, the limit of the algorithm for this image is nine different discernable areas (Fig. 20d).

Fig. 20
figure 20

Design subtraction of an espolín fabric: a input image (acquired with a conventional scanner); b posterized image with two clusters; c posterized image with four clusters; d posterized image with nine clusters; e posterized image with ten clusters; f posterized image with 14 clusters

Full size image

4.2 3D modelling applying different parameters and weaving techniques

In this section, we start from a single design in order to reproduce 3D models based on different techniques. The design is the one previously shown in Fig. 21a, and it is schematized in Fig. 21a. It corresponds to a fabric with the “espolín with damassé ground” technique. However, as VL can weave any technique given a design, we are using it for all cases in order to compare the results of the 3D models. We make an assumption that for all the techniques, except for the espolín with damassé ground, the background of the image is composed of a single color, i.e. without the damask effect. To this end, we consider areas 1 and 2 to be a single area. Because of this, and in order to avoid the current limitations of design subtraction, we have produced synthetic images for espolín (Fig. 21c) and espolín with a damassé ground (Fig. 21d), which are the input data for VL.

Fig. 21
figure 21

Design used to generate the 3D simulations for the different weaving techniques considered in the VL: a original image; the numbers correspond to different areas (types of yarns/colors); b posterized image with two clusters, which is used for the damask, damassé, damassé with metallic weft and lampas; c synthetically generated image with ten areas, used for espolín; d synthetically generated image with 11 areas, used for the espolín with a damassé ground technique

Full size image

Based on the image of two clusters (Fig. 21b), we have produced four different 3D models: a damask (Fig. 22), a damassé (Fig. 23), a damassé with metallic weft (Fig. 24) and a lampas (Fig. 25). The damask and damassé models have the same geometry, as they have a weft layer and a warp layer, but the yarns are colored differently.

Fig. 22
figure 22

3D model generated with VL for the damask weaving technique: a detail; b complete fabric

Full size image
Fig. 23
figure 23

3D model generated with VL for the damassé weaving technique: a detail; b complete fabric

Full size image
Fig. 24
figure 24

3D model with the damassé with metallic weft technique: a detail; b complete fabric

Full size image
Fig. 25
figure 25

3D model with the lampas weaving technique: a detail; b complete fabric

Full size image

For the damassé with metallic weft and lampas weaving techniques, we consider the pictorial part (merging areas 3 to 11) as a layer that goes above the background. In the case of lampas, we are simplifying the original design to produce a model with only two colors (one for the background and another for the pictorial part), as explained above. The damassé with metallic weft, is composed of two colors, one for the background and another for the pictorial part, this being woven with a metallic yarn (gold or silver). The generated 3D model for the damassé with metallic weft is shown in Fig. 24.

In Fig. 25, the generated 3D model with the lampas weaving technique is depicted. As can be seen, the weft and warp yarns which correspond to the background are thinner than the weft yarns that correspond to the pictorial part.

The espolín weaving technique is shown in Fig. 26. In this case, the input image for the 3D modeling in VL is the one depicted in Fig. 22c. The pictorial parts (areas 3 to 11) are considered as independent. In this way, different yarns (forms and colors) can be applied in VL.

Fig. 26
figure 26

3D model with the espolín weaving technique: a detail; b complete fabric

Full size image

The espolín with damassé background weaving technique is shown in Fig. 27. The input image for the 3D modeling in VL is the one depicted in Fig. 21d. The difference between this technique and the espolín one is that, in this case, instead of considering the background homogeneous, it consists of two zones (one for area 1 and another for area 2). For the pictorial part (areas 3 to 9), similar outputs are produced with both weaving techniques.

Fig. 27
figure 27

3D model with the espolín with damassé ground weaving technique: a detail; b complete fabric

Full size image

Finally, we have also derived a 3D model using the freestyle option, with the following presets: twill weave; gold yarn (for the warp); silver yarn (for the weft); weave it similar to espolín (brocaded pictorial part). The resulting 3D model is depicted in Fig. 28. Although this representation does not correspond to any particular historical weaving technique (e.g. gold yarns are never used for the warp, and an espolín does not have a twill ground), it is geometrically feasible and therefore it can be represented in 3D.

Fig. 28
figure 28

3D model with the freestyle technique: a detail; b complete fabric

Full size image

5 Discussion

Taking into consideration the number of weaves implemented in Virtual Loom (a total of 39) and the number of yarns with different geometries (five, as all the considered metallic yarns share the same geometry), for an image with five clusters (that means six yarns, as the ground is made of two yarns), like the one shown in Fig. 19c, a total of 39 × 5 × 6 = 1140 different 3D models can be achieved in the “free style”. Additionally, the user can choose between weaving the pictorial part as brocaded or not, so this doubles the number of possible 3D models. On the other hand, for reproducing the historical weaving techniques (e.g. espolín), less outputs are possible, because not all the weaves or yarns can be used for a certain technique. For instance, the ground of an espolín is usually composed of a limited number of satin weaves, and the most commonly used was the queen satin. Therefore, such restrictions are considered in Virtual Loom, resulting in 3D models that truly represent historical fabrics at the yarn level.

It is worth to mention that the final quality of the generated 3D models in Virtual Loom in part depends on the both the conservation state of the original fabric itself, and the quality of the photograph. This is due to the automated process for the design subtraction, that results in different areas that are automatically assigned to different colors/yarns, as reported in Section 3.3 and exemplified in Section 4.1. The better the conservation state of the fabric and the better quality of the photograph, the more reliable will be the design subtraction and thus the 3D reproduction for a given fabric. Additionally, the less colors that the fabric is made of and/or the more different the colors are, the easiest will be for the algorithm to discern between different areas. In this regard, other textile related software do not perform automatic subtraction of designs from photographs, and thus the quality of their 3D models only depends on the graphics applied.

Also different from other related solutions (recall Section 2), in Virtual Loom we are modelling specific kinds of yarns, that were used in historical silk fabrics dating from 15th to 19th centuries, and thus the number of yarns is limited and have specific characteristics. This makes it not possible to allow users to freely modify the parameters related to the yarns’ geometry (as other software do), although internally, we have implemented a yarn editor in the Unity solution, and thus we could make available a greater number of yarns if more historical weaving techniques were considered in Virtual Loom.

Finally, it is worth to highlight that Virtual Loom is able to reproduce virtual representations of fabrics that are weaved with traditional looms (e.g. Jacquard looms), such as fabrics made with the espolín technique, that require the work of an artisan. These techniques cannot be manufactured in current looms. Other textile related software whose aim is to produce a real fabric, cannot reproduce historical techniques such as espolín.

6 Conclusion

In this paper, we have described the details of the design and implementation of VL, an interactive tool aimed to document, preserve and represent historical weaves and weaving techniques in 3D forms. A section of results shows how to extract the designs from images, and how a given design can be represented in 3D, making use of different weaving techniques. With these experiments we show how versatile VL is in producing 3D representations of historical fabrics based on different weaves, yarns and weaving techniques. In “freestyle”, users are not restricted to a given weaving technique, as other options are possible.

In the results section, we also address the limitations of our technology in extracting the original design of historical fabrics. It can be said that the more variety of colors and the more complexity of the design (e.g. many small areas), the less reliable is the posterization of the image. On the other hand, it is also important to have good input images to avoid shadowed areas or other artifacts that hinders the good performance of the image analysis. In this regard, we recommend using scanned images rather than images taken with a camera.

As further work, we intend to improve the visual aspect of the yarns, so they are more similar to the real ones, while keeping non-complex geometry. We will focus on the different properties of yarns, making use of the optical study that we performed. For example, we want to add some Perlin noise in the twist in order to remove the uniformity that appears along the yarn. We are also working on defining the rear side of the fabrics. The rear side of fabrics are not input data because, in most cases, this information doesn’t exist, i.e. fabrics are not documented from their rear side. Currently, VL weaves a possible rear side according to the technique, but without taking into account possible human decisions. Some of these human decisions (e.g. using a single yarn or different yarns, to weave motifs of the same color) will be implemented in the following versions of the VL.