3D Real Time Virtual Fitting Room for Women

Abstract

The purpose of this experiment is to set up the women virtual fitting room where user can try virtual dress without taking off her dress. The first part is to simulate virtual parts of the body that were covered with dress and then incorporate them into the real body. After having the new partial virtual body, then user can try some virtual clothes. In order to mobilize the user wearing virtual dress, Kinect was used to detect the skeleton and joints of the real body, virtual part of the body and virtual dress, which were aligned into the same position.

1 Introduction

The women’s queues for the fitting room in the clothing department are usually longer than for the men. The reason may be due to the fact that women want to make sure that the dresses they have selected fit well and also look good on them. Waiting for the fitting room for a long time, some of the users may lose interest in buying the dresses. These problems exist not only for the department stores offering dresses in the physical form but also on e-commerce sites selling women dresses as well because users are not able to try on the dresses. The solution for this problem may rely on providing a good virtual fitting room. Virtual fitting room is a real-time interactive platform, where users can try virtual dresses on their bodies. The concepts of the virtual fitting room have been investigated for quite some times. With the new technology on body scanning, such as Kinect device by Microsoft, there is a chance that the perfect fitting room might be possible. Using Kinect to track the body of the user and then place a 2D or 3D partial image of the dress on top of the body seemed to be exciting for a while but it did not look realistic especially when the user moved but the dress did not move along. Many researchers have tried to modify Kinect or other devices with the same technology to form perfect fitting room, but the results were still far from perfect. Most researches on improving virtual fitting room concentrated on how to improve the use of Kinect or other body scanner to detect the body of the user, but not on the realistic of the dresses that the users will see when they are wearing the dresses.

Contrary to others, we will emphasize on the 3D dresses that the user will try on when they are in the virtual fitting room. So, instead of putting the dress on top of the body, the dresses will wrap around the body just like we are wearing the dress in real life.

2 Related Work

Various areas of researches on Virtual Fitting Room (VFR) have been done for quite some time, ranging from framework designing online virtual fitting room to VFR for mobile phone. Zhao et al. [1] has studied framework design for online virtual fitting room on a local cluster computing platform. Fast body modeling algorithm for cloth simulation is proposed and the key techniques for cluster computing based online Virtual Fitting Room (VFR) were discussed and a hierarchical architecture was proposed. The result from the implementation shown that the speed for the body modelling process was acceptable for online virtual fitting experience. Li et al. [2] proposed a framework of the virtual Try-on system which forms the basis of a realistic, three dimensional simulations and visualization of garment on virtual counterparts of real users. Users can view the clothing animation on the various angles, plus can change actor’s hairstyles and accessories, etc. Garcia C. et al. [3] proposed an algorithm for virtual fitting room application for mobile devices with the objective to obtain a real time, platform independent application. The three stages algorithm are: detection and sizing of the user’s body, detection of reference points based on face detection and augmented reality markers, and superimposition of the clothing over the user’s image. The application can run in real time on existing mobile devices. Nakamura et al. [4] presented an estimation method of the dress size for Virtual Fitting Room (VFR) using Kinect. Several Kinects were used for high accuracy of estimation and tacking.

3 Experimental Setup

In this research, three parts of making perfect fitting room for woman were proposed.

The first part was related to finding the real body of the users and then replaced some parts of it with virtual model (avatar) that replicated the body of the users or users. The second part was related to the making of 3D dresses that would fit on the users and the last part was related to making the virtual body of the users moved in real time in accordance to the movement of the users while having 3D clothes fitting on the virtual body. With this concept, the users can see how they look when moving while moving with the dresses on, and make the decisions on whether they look good with the dresses or not. For the first part, the virtual body of the user was done using “Kinect sensor” and Autodesk’s “Maya” 3D animation software.

For the second part, dresses were made from 2D pattern into 3D virtual dresses. The size of the 3D virtual dresses could be adjusted by changing the parameters.

In the third part, Kinect sensor was used for tracking the movement of the user. By using the joints (nodes) of the user’s body, detected by Kinect scanner, then assigned those joints to the virtual model and the dress worn by the virtual model at the same position, the synchronous movement of the user, the avatar virtual model and the dress could be done. In our research, we have tried to use more than one Kinect sensor but we found out that using either two Kinect sensors or one Kinect sensor made no differences in terms of the appearances, so we choose only one sensor for the experiment.

3.1 3D Body Modeling

The main objective of setting up the virtual fitting room is that users can try on any virtual dresses without having to take off their clothes. So, the purpose for this part is to take the clothes off virtually. If they have the new virtual dresses on while still having their original dresses on, it will not look perfect, for example, having the virtual dress on top of pants (see Fig. 1).

Fig. 1.

Virtual dress on the top of the body [5]

The solution for getting rid of the physical dress, is to replace all the body parts that are covered with physical dresses and replaced them with virtual undress model or avatar but keeps the rest of the body such as head, hand and feet unchanged. When user evaluates the dresses through some postures and movements, they will feel like they are trying on the real physical dress by themselves. The three main steps of our 3D body modeling process are described below.

Step 1: Getting the 3D model of the body

In this step, the user body is scanned using Kinect scanner. Kinect has two cameras but it does not accomplished 3D sensing through stereo optics. A technology called Light Coding makes the 3D sensing possible. The output from the scanner can be set in the form of 3D model. Unfortunately, the quality of rendered Depth Frame images is very poor. So, instead of getting 3D model, we will settle for 2D picture. Using 2D picture of the front and side views of the user (Fig. 2), we have developed the techniques to get 3D model of the user without dress easily.

Fig. 2.

2D image of the front and side view of the user

From the front view 2D picture of the user, Image processing technique and edge detection were used to get the boundary shape of the user as shown in Fig. 3.

Fig. 3.

Boundary of the front image of the user

Standard 3D models of a female user were created and compared to the boundary shape (2D) of the user (Fig. 4a). By scanning from top down, if some part of the boundary is smaller than the model (for example Fig. 4b), the program will adjust the model to make them smaller (Fig. 4c) and if it is larger than the model, then the program will make the model larger. After finishing front view, this process will be applied to the side view as well, using the output model of the first process to begin with.

Fig. 4.

Scanning process of the front view to get the 3D model of the user

For accurate results, scanning only main parts such as bust, waist, hips and etc., is enough (see Fig. 5).

Fig. 5.

Body measurement [6]

Step 2: Stripping out the physical dress

As mentioned earlier, the purpose of the virtual fitting room is to be able to remove the physical dress and put the virtual one on instead. In order to do that, we have to know the exact body shape. In case that the user wears clothes that disguised her real body shape, then more accurate data of her body have to be identified either by manual measuring or by automatic scan. For automatic scan, color ribbon may be used for wrapping around some parts of the body such as waist, hip, etc. (see Fig. 5), automatic scan can reveal her real body shape.

Step 3: Construction of 3D body with virtual body part incorporated

After getting the virtual body of the user, some parts of the body that were covered with dress will be replaced by the virtual body parts getting from step 2. Figure 6 shows 3D model of the user before and after the process.

Fig. 6.

3D model construction of the user

3.2 Virtual Dress

In order to make the virtual fitting room look more realistic, the dress must look realistic too. Usually, in the virtual fitting room today, the dress can be categorized into 2 types, i.e. 2D dress (Fig. 7) and 3D dress (Fig. 8).

Fig. 7.

Model with 2D dress [7]

Fig. 8.

Model with 3D dress [8]

Both the 2D dress and 3D dress did not guarantee that it will look realistic when users try it on. In our case, designing the dress for virtual fitting room will be on different approach. Dresses were made from 2D pattern into 3D virtual dress using the concept of “Digital Clothing Suite”, 3D graphics simulation software from Seoul National University. The size of the 3D virtual dress can be adjusted by changing the parameters. Since the dress will be wrapped around the body instead of lying in front of the user, it will look more realistic. Example of this designing concept can be seen in Fig. 9.

Fig. 9.

Designing concept of 3D virtual dress [9]

With this concept, we have made the dresses in 3 different sizes: M, L and XL which will be tested for trying on experiment. All the dresses were shown in Fig. 10.

Fig. 10.

3D Virtual dress in different sizes

3.3 Tracking Movement of Avatar with Dress on

After getting the virtual model incorporate with the real body as shown in Fig. 10, next step is putting the dress on. In order to make the real body, the virtual parts of the body and the virtual dress are incorporated into one object, so that when the user moves, she can see the virtual parts of the body and the dress move along with her. Since we are using Kinect scanner for tracking the movement of the body, we have to assign the skeleton and the joints of these three parties aligned in the same position as designed by Kinect. Figure 11 shows the joint and the skeleton of the body assigned by Kinect.

Fig. 11.

(a) Skeleton and joint assign by Kinect [10] (b) Skeleton and joint assigned to real people [11] and (c) Movement of the skeleton and joint shown in graphics [12].

After skeleton and joints were assigned, tracking the movements can be done easily. Figure 12 shows the user in T-post position with dress size M, L and XL respectively.

Fig. 12.

User with dress in sizes M, L and XL respectively

With different dress size, we can see whether the dress is too big or too small. Details of this observation can be seen in Fig. 13.

Fig. 13.

Details of the dress whether it is perfect or not.

For the movement of the body with virtual dress (Fig. 14), we have found that the real body, the virtual parts of the body and the virtual dress move along quite well, even though the color of the virtual parts of the body is a little bit different from the real body.

Fig. 14.

Tracking the movement of the user wearing virtual dress

4 Conclusion and Discussion

With the prototype of the real time fitting room set up, we have tested for user satisfaction by 20 volunteer women. The result showed that they all agreed on how easy and time saving using this prototype. They were also satisfied with the realistic experience the real time virtual fitting room provided. Though there are a lot of things needs to be improved before it will be available for commercial use, but this experiment proved that we are in to the right direction to have for the perfect virtual fitting room.