Keywords

1 Introduction

In manufacturing processes, appearance inspections are conducted to ensure that industrial products are free of scratches, stains, and other visible defects. During an appearance inspection, it is difficult to reliably detect a wide variety of defects that occur in the production process and correctly distinguish between good and defective products simultaneously based on appearance inspection standards. Therefore, human visual (sensory) inspection is used commonly in the field. In addition, as production patterns shift to high-mix low-volume production schemes, automated inspections have not progressed because they are not cost-effective in terms of development, introduction, and operation costs [1].

In response to these problems, recent studies focusing on appearance inspection have taken an ergonomic approach to human visual inspection [2], as well as an approach using image processing and machine learning [3] for machine-based visual inspection, aiming to improve inspection accuracy and efficiency. Recently, a work support system has been proposed that combines the findings of the two approaches, in which a machine searches for defects and a human inspector categorizes the results. The effectiveness this approach has been experimentally demonstrated on model images that imitate industrial products [4]. Specifically, the work support concept entails using a camera to capture images of the product before it is sent to the human inspector and presenting the results of the computer analysis to the human inspector on a display, so that only potentially defective items are inspected.

The work support system can be configured using commercially available inexpensive cameras and computers that conduct machine learning (Deep Learning) with product images as input; there is no need to set individual features for each product. In other words, this approach directly addresses the cost-effectiveness issue discussed above and could be widely applicable to many industrial products.

However, the usual defects in actual industrial products have certain characteristics that affect the visibility of defects greatly depending on the lighting conditions [5]. Therefore, product images taken under appropriate lighting conditions should be used as input images. The usefulness of the work support system has not yet been demonstrated while considering such defect characteristics.

Therefore, in this study, we experimentally evaluate the relationship between lighting condition and visibility of defects as a case study on actual automobile parts. Our objective is to study the practical applicability of the work support system based on an understanding of the defect characteristics.

2 Experimental Design

2.1 Target Products and Image Capture Environment

The target products collected and used were 80 defective automobile parts measuring 87.5 mm in length and 440.0 mm in width produced by Company A. The breakdown of defects in the 80 parts was, 20 bumps, 14 scratches, 11 dust spots, 7 paint over-sprays, 8 thread debris, 7 dents, 6 irregularities, 4 white spots, and 3 stains. In the image capture environment, shown in Fig. 1, the camera (Nikon, D5500) and target product were fixed in position. Moreover, two planar illuminators (SUNTECH, LG-E268C) were used at different positions and angles to capture images under various lighting conditions.

Fig. 1.
figure 1

Image capture environment.

Full size image

2.2 Experimental Factors

Nine types of defects (bumps, scratches, dust spots, paint over-sprays, thread debris, dents, irregularities, white spots, and stains) were used as experimental factors. As shown in Fig. 2, the angle θ between the illuminator and target product was set at five settings (-60°, −30°, 0°, 30°, and 60°). Note that 0° is the position perpendicular to the target product, −60° and −30° denote position after counterclockwise rotation, and 30° and 60° denote position after clockwise rotation. Based on these experimental factors, a total of 400 images were captured and used in the experiment for a combination of 80 defective products (9 types) with 5 lighting angle settings.

Fig. 2.
figure 2

Experimental factor

Full size image

2.3 Methods for Evaluating Visibility of Defects

Three university students (including graduate students) aged 21 to 24 years participated as evaluators to conduct subjective evaluation of a total of 400 images on a 5-point scale of 1 (invisible), 2 (somewhat visible), 3 (visible), 4 (somewhat easily visible), and 5 (easily visible). The mean and standard deviation of the scores given by the three evaluators for each image were then used as the values for evaluating the visibility of defects. In this study, university students with no experience in appearance inspection were recruited as evaluators but we believe that it is necessary to recruit experts in the appearance inspection processes as evaluators in the future.

3 Experimental Results

The mean and standard deviation of the evaluated defect values are shown in Fig. 3, which depicts that the highest visibility rating was 4.0 for white spots and lowest was 1.5 for irregularities, indicating that visibility varies greatly depending on the type of defect. It was also observed that the standard deviation was large for some types of defects, such as bumps and paint over-sprays, and visibility of defects differed even for the same type of defect.

Fig. 3.
figure 3

The mean and standard deviation of the evaluated defect values

Full size image

Next, the evaluation values with respect to lighting angle for each defect are shown in Fig. 4. The figure depicts that the effect of lighting angle differs depending on the defect type. For example, some defects, such as white spots and scratches, have higher visibility ratings at −60° and 60°, while others, such as stains, have higher ratings at 0°, and some defects, such as paint over-sprays and bumps, have the same level at all lighting angles.

Fig. 4.
figure 4

The evaluation values with respect to lighting angle for each defect

Full size image

4 Discussion

The experimental results demonstrated that visibility differs for each defect type, as well as the same defect type under different lighting conditions. Furthermore, the effect of lighting angle differs depending on the type of defect. Therefore, a plot of the mean and standard deviation values of the evaluation results for each individual defect is shown in Fig. 5. The figure clearly demonstrates that it is important to evaluate and understand the characteristics of individual defects as well as their types and consider their applicability to work support systems because some defects of the same type have different characteristics.

Fig. 5.
figure 5

a plot of the mean and standard deviation values of the evaluation results for each individual defects

Full size image

If Fig. 4 is divided into four quadrants, defects in the second quadrant are visible regardless of the lighting angle and are categorized as a group of defects that are easily detected by the work support system. The defects belonging to the first quadrant may be visible depending on the lighting angle and are considered to be detectable by the work support system by appropriately considering the image capture conditions of the input image. However, we can also confirm that only approximately 24% of all the defects belong to the first and second quadrants. In contrast, the number of defects belonging to the third and fourth quadrants was approximately 76% of the total. Subsequently, we determined that measures need to be considered for practical use because the product images as they are cannot be effective inputs to the work support system for the target products considered in this study.

The abovementioned findings elucidated that to put the work support system into practical use, it is important to construct a methodology to further pursue lighting conditions that increase the visibility of defects and take images considering the visibility of defects as input images for deep learning.

5 Conclusion

In this study, the relationship between lighting condition and defect visibility was experimentally evaluated using 80 actual automobile parts to realize a novel appearance inspection method based on human-machine integration. The results confirmed that the visibility and its characteristics differ for each type of defect, as well as the same type of defect under different lighting conditions. We also found that for about 24% of the total of 80 defects the visibility was sufficiently high depending on the lighting condition, while for the remaining 76%, the visibility was not sufficiently ensured. These findings elucidated that to put the work support system into practical use, it is important to construct a methodology to further pursue lighting conditions that increase the visibility of defects and take images that consider the visibility of defects in input images for deep learning.

In future studies, we would like to use inspectors who are field experts as evaluators to assess more detailed defect characteristics, propose appropriate lighting conditions based on a better understanding of the defects, and apply the subsequent result to the work support system to confirm its usefulness.