1 Introduction

Electronics manufacturing is one of the most recurrent industrial fields with expressive exponential growth. In 2021, electronics manufacturing and design contracts were estimated at USD481.3 billion, with an annual growth rate of 10.5%, expected to hit around USD1158.6 billion by 2030 [1].

Despite the larger share of China’s electronics manufacturing market, the global trend shows an increasing distribution of these products manufacturing worldwide, implying fiercer competitive pressure for a larger market share, which raises a flag for developing ever-better quality control protocols [2]. Currently, the overall industrial research and development (R&D) investments show greater attention to implementing high-standard technology not just to the product manufacturing itself but also for more versatile and automatic quality inspection systems. Regarding the technology optimization processes behind the Industry 4.0 concept, these improved structures are usually called Smart Manufacturing Systems (SMS) [3].

Current electronic manufacturers’ efforts for better production quality are directed at developing high-precision inspection SMS, following the recent “Zero Defects Manufacturing” (ZDM) trend, which aims to include in the company’s roadmap a strategy to mitigate its manufacturing faults as far as possible [4]. Concerning the electronics industry, the main piece in every electronic device is the printed circuit board (PCB). Therefore, the PCB can be the product’s most vulnerable part, making it the primarily focused segment regarding manufacturing defects [5,6,7].

As a well-consolidated product concept, PCB manufacturing can be divided into two stages, the bare PCB (BPCB) and the PCB assembly (PCBA). The BPCB relates to a plain insulated material board containing barely conductive tracks, pinholes, vias, and soldering pads, while the PCBA refers to the PCB components assembly stage, where most pieces are soldered to the board (e.g. resistors, capacitors, inductors, integrated circuits, etc.) [6].

In the early manufacturing stages, the PCB inspection was supported only by human specialists applying electrical tests and visual inspection. However, this approach runs shortly after the increasing production rate, higher precision standards, and smaller components, besides the humanly stressful and wearing environment [6, 7]. Hence, multiple PCB inspection strategies have been developed since then, such as automatic optical inspection (AOI), X-ray imaging, Laminography, ultrasonic imaging, and thermal imaging [8]. Although all of these methods provided ever-better inspection quality, most of them have rapidly become limited by factors as expensiveness or time consumption. Thus, leaving the AOI as the most helpful method as it can detect almost any type of defects in a faster and completely indestructible approach [6,7,8]. Nevertheless, even with AOI’s ubiquitous preference for PCB inspection systems over the past three decades, it is still so expensive and provided by a limited amount of private companies that mid-range PCB manufacturers still cannot dispossess human visual inspection [5, 9]. Most of the state-of-the-art (SOTA) work has recently taken advantage of the new developments in AI for computer vision applications to improve the performance of AOI. Therefore, the last few years of R&D efforts have been addressed to develop more affordable AOI PCB inspection systems [5,6,7, 10].

With the advent of machine learning, more focused on deep learning, inspection systems based on computer vision are now a high-standard available technology. However, they tend to be expensive in terms of data acquisition. Deep learning vision systems usually depend on datasets composed of thousands of images to prepare the model for a robust and practical application, which is not always economically viable. Further, they are usually not proof of concept drift, a critical issue where the AI model suffers degradation with natural changes in the data domain. This makes human specialists still necessary, even if he has limited inspection speed and accuracy. Nonetheless, recent AI approaches have been demonstrating promissory results for high-speed applications with satisfactory efficacy.

The present survey introduces the most relevant applications with AOI systems for PCB quality control, highlighting the significant technological changes over time and pointing to the SOTA’s most approached strategy trends and challenges. Moreover, this work intends better to comprehend the latest AI applications for PCB inspection, showing their usefulness and ongoing limitations.

The paper is organized as follows: sect. 2 contextualizes what comprises a PCB and how its inspection strategies are usually divided. Section 3 presents multiple works related to inspecting BPCBs, while the 4 section focuses on papers on PCBA inspection methods. Then, the sect. 5 explains the general composition of AOIs, as per their current advantages and limitations. Finally, sect. 6, the main open-sourced PCB datasets are exemplified, describing their main composition and features.

2 Background

The PCB inspection process can be divided by the defect domain, the BPCB, or the PCBA. Following the literature about PCB defect analysis, it is possible to observe a familiar universe of defect classes shared by practically any type of PCB. In both domains, there are common classes of defects that completely disable the PCB functionality: unetched copper, open circuit, short circuit, missing hole, and misaligned hole in BPCBs and missing, burned, or inverted component, and open soldering in PCBA. Even when some defects do not completely disable the PCB function, they still may cause electronic disturbances during the PCB operation and possibly lead to early failure due to faster degradation: over-etching or under-etching copper, mouse bite, partial open, spurious copper, incipient short, dirt contamination, and air bubble in BPCB, and misaligned component, tomb-stone, cold soldering, partial component burning, and pre-defective component in PCBA [5,6,7,8]. Figure 1 illustrates defective samples of one BPCB and one PCBA.

Fig. 1
figure 1

Defective samples of a BPCB and a PCBA. a A BPCB taken from a public PCB dataset by Ding et al. [11]. b A PCBA taken from dataset FICS-PCB by Lu et al. [12]

Full size image

It is possible to address the PCB inspection methods with traditional approaches utilizing computer-facilitating tools, multiple extra apparatuses, or just manual human skills. On the other hand, combining all of these previous methods with AI approaches equipped with some training datasets, may increase the potential for faster and more accurate inspection systems. Hence, addressing the defect domains and possible approaches for PCB inspection methods, this survey is structured as illustrated in Fig. 2.

Fig. 2
figure 2

Survey structure of PCB inspection methods, domains, and approaches

Full size image

The following two sections, 3 and 4, summarize the main BPCB and PCBA inspection SOTA methods, respectively. Each section is divided into two inner parts, one for methods without AI approaches and another for it. Further, the section 5 details the mostly desired industrial PCB inspection method, the AOI, bringing its main positive features along with its limitations. Finally, since AI methods are the most recently explored approaches, and their training datasets comprise an essential role, this survey also presents a resume of publicly available sources of PCB datasets in the section 6.

3 Bare PCB inspection

The following two sections, 3.1 and 3.2, describe the papers about Bare PCB inspection covered by the present survey. Their summaries are structured in a chronological line, from the recent past until the current SOTA, following the timeline illustrated in Fig. 3.

Fig. 3
figure 3

Timeline of Bare PCB inspection methods covered in the present survey

Full size image

3.1 Traditional methods

Putera and Ibrahim [13] combine two previous techniques to analyze BPCB tracks by segmentation morphology extraction based on the template-matching approach. It needs test and template samples to be thoroughly registered in relation to each other, i.e. both images must have the same relative positional orientation. Then, using the MATLAB computation tool, each image is segmented into four morphological groups: square segment, hole segment, thick-line segment, and thin-line segment. The resulting images are binarized into black and white so each respective template and test segment can be paired in an XOR logical operation, where the pair differences are the unique residue left, which may be accused as defects, illustrated in Fig. 4. The introduced technique could identify 7 BPCB defects out of 14 described in the study. Although the authors report a high detection confidence score, they highlight some occasions where the segmentation and binarization steps can produce undesired noise, increasing the miss detection risk.

Fig. 4
figure 4

Example of XOR operation between template and test sample, by Putera and Ibrahim [13]

Full size image

Following the template-matching approach, Chauhan and Bhardwaj [14] also present an image comparison method with XOR logical operation. However, they do not segment the image by morphological regions as in the previous method, but they provide a geometric statistical analysis of the resulting XOR operations. These analyzes show repetitive patterns from the possible BPCB errors that serve as important features for the following deterministic image classification algorithms. The authors also emphasize that the XOR operation method can be limited to image registration precision to acquire significant results.

Guan and Guo [15] introduce a functional mathematical approach to enhance a BPCB image as a defect inspection preprocessing technique. The authors take advantage of the high-contrast profile of BPCB photographs between conductive tracks and the insulated board to deduce what they call a double sigmoid operation capable of enhancing the image sharpness and further filtering noises. Summarily, the double sigmoid operation consists of a sigmoid derivative operation of grey-scaled images to reduce the common smoothness photography effect in edges and a post-integrative operation to recover the original image. The authors claim their primary method’s advantage against standard noise filtering algorithms that usually add undesired edge smoothness, while theirs can simultaneously filter noise and sharpen the image’s edges. Figure 5 shows one of their results examples.

Fig. 5
figure 5

Comparison between original and enhanced images, by Guan and Guo [15]

Full size image

Looking to improve the XOR template-matching approach, Ibrahim et al. [16] present a pipeline algorithm with multiple image processing steps, including the XOR operation, and a final step divided into multiple algorithms for each type of defect to be detected, comprising seven types: missing hole, wrong size, open, short, pinhole, under etching, and mouse bite defects. An important innovative aspect proposed by the authors was the registration algorithm, composed of a combination of geometric transformations that can precisely match the test image orientation in relation to the template image. The authors report an average processing time of approximately 20 s for 1580x917 resolution images using MATLAB software, which can be an industrial limitation, depending on the production rate. However, it is still faster than Human inspection.

Chaudhary [17] introduces a novel automatic BPCB inspection system that claims to be fast enough for real-time manufacturing scenarios. Their proposal utilizes FAST as an image feature extractor algorithm [18] and MSAC for an image registration algorithm [19], applied in gray-scaled image pairs of a template and a test image. Further, similar to Putera and Ibrahim et al. [13, 16], this method applies image segmentation to isolate what the authors consider the three main parts of the BPCB, wire tracks, soldering pads, and holes. The segmentation is made from normalized histogram thresholding from the binarized image, shown in Fig. 6. From the histogram, it is possible to extract the binarization thresholds to segment the background, the wire tracking, and the soldering pads, while the holes can be segmented apart from the latter. After segmenting, the template-test image pair is subtracted as in previous works [13, 16], creating a pair of positive and negative images. Depending on the defect type, each resulting image serves as the input for defect detection and classification algorithms. The authors introduce one image processing algorithm based on geometric analysis for each of the 14 most common BPCB defect patterns and report that they can accurately detect and classify them. Moreover, they claim a processing time of about 2.53 s, which could lead to a real-time AOI method.

Fig. 6
figure 6

Normalized histogram for image segmentation, by Chaudhary et al. [17]

Full size image

Zhu et al. [20] approach five BPCB defects analysis using template matching. The main difference from previous methods was the utilization of Discrete Wavelet Denoising for image quality enhancement [21], and Hough Transform as feature extracting technique for image registration [22]. Their techniques to detect and classify the defects is similar to previous works, with image subtraction for detection and geometric analysis algorithms for classification. From a 30 defective PCB images dataset, the authors claim a 100% detection rate and 90% classification accuracy. Figure 7 illustrates their work as an example of defect detection.

Fig. 7
figure 7

Example of 5 defects detection in BPCB, by Zhu et al. [20]

Full size image

3.2 AI methods

Probably one of the first applications of machine learning techniques to improve PCB AOI systems was provided by Londe and Chavan [23]. It implements MATLAB’s Image Processing Toolbox to perform a template matching difference extraction between a pair of images and further processes the results to extract discriminative features to classify all 14 known BPCB defects. The ML algorithm utilized is the K Nearest Neighbours (KNN), a non-parametric model to cluster the feature dataset; this is, it extracts the feature distances of all samples in the dataset and classifies them in K clusters, predefined by the designer. For example, in a 14-defect class system, the KNN algorithm may be configured with \(K = 15\), where one class represents “no defect”. The authors report an average accuracy for all 14 defects of 75,37%, but the number of tested images and defect counting is not declared.

Using a supervised machine learning (ML) classification algorithm, Lu et al. [24] propose a combination of deterministic image processing techniques and support vector machines (SVM) for the ML strategy. The image preprocessing starts with binarization and then denoising with median and mean filters. Further, two feature extraction steps provided with LBP and HOG algorithms are responsible for dimensionality reduction and information filtering [25, 26]. Both resulting features are merged with Bayes Fusion and an SVM training step [27]. With 184 training images and 108 test images, the dataset comprises positive and negative pairs, and the authors claim a defect detection accuracy of 89.22%.

Ding et al. [11] describe that BPCB defects are formed mainly by a tiny number of image pixels, which has a significantly small size in relation to the whole high-resolution images taken from PCBs. These relatively small sizes represent a challenge for traditional convolutional neural network (CNN) architectures, where tiny image pieces tend to vanish in deeper layers of the network. To address this issue, they propose a pyramidal concatenation technique, illustrated in Fig. 8, inspired by ResNet from He et al. [28], that takes the information from previous layers of the CNN and injects it into deeper layers of the model, which should preserve the small-size information. The study is applied in the public dataset of Huang et al.[10] and reports a 98.9% mAP score.

Fig. 8
figure 8

Pyramidal residual connections model architecture, by Ding et al. [11]

Full size image

Ihar et al. [29] present the conversion of an unsupervised to a self-supervised deep learning approach for BPCB inspection. The authors highlight the usual lack of defective PCB samples in industrial environments, which becomes a considerable challenge for supervised machine learning applications. To deal with this, they explain how a self-supervised model could learn from a representative set of non-defective samples and a few defective ones. In this study, they used a pre-trained VGG16 model from Simonyan and Zisserman [30] and fine-tuned it with PCB data in a self-supervised way, so it learned how to cluster the data into two groups that should be considered regular or defective samples. They reported a mean accuracy of 80% at defect detection, but some defect classes reached more than 95% while capable of being trained with a significantly small amount of defective samples, which they claim to be an essential industrial feature.

As in the previous study, He et al. [31] utilize the VGG16 network for a self-supervised approach to deal with the lack of a representative defective sample number. The main difference between them is that the present study uses the usual template-matching subtraction technique. However, instead of deterministically extracting the image-pair features, they use the VGG16-lite backbone to extract them in a self-supervised way. Hence, the model learns a better representation map of the data. After the feature extraction, they are subtracted. The resulting feature passes through another deep learning custom network that detects and classifies six common BPCB defects, divided into three outputs in accordance with the defect bounding box size. Figure 9 illustrates their model architecture with inputs and outputs. From 2000 training and 1000 testing samples, the authors report 98.6% classification accuracy.

Fig. 9
figure 9

Model architecture, inputs and outputs, by He et al. [31]

Full size image

In Adibhatla et al. [32], the SOTA deep learning model called YOLOv5 is used [33], applied to an object detection and classification task in a supervised approach. The authors describe a dataset composed of 23,000 PCB samples cut in small patches to represent better the tiny defects in relation to the whole board, split into nine parts for training and one for testing, along with a 10-fold cross-validation strategy. The YOLO model is trained to detect defect regions inside a rectangular bounding box with a class confidence score. In this study, the model is not trained to discriminate between multiple defect classes but all as a single defect class. They claim this option increases the model’s detection effectivity at the expense of not judging the defect class, which could be a posterior task. The authors report the best mean accuracy score of 99.74% with a 0.23% false positive rate. Figure 10 illustrates the paper’s example of true positive findings of small BPCB defects.

Fig. 10
figure 10

Sample images of BPCB defect detection, from Adibhatla et al. [32]

Full size image

Bhattacharya and Cloutier [5] introduce a similar method to the previous work, using the YOLOv5 model to detect defects. Differently, they modify the original YOLOv5’s architecture with a custom structure that they claim to be more data-efficient and still capable of classifying each desired defect class. In their experiment, they utilized a dataset composed of only 1,386 images with six BPCB defects from Huang et al. [10]. Further, they introduce a data synthesizing method to provide higher data representativeness under small-dataset scenarios. They report a mean accuracy of 98.3%, a false positive rate under 5%, and compare previous SOTA PCB inspection methods in the same data conditions, where their method achieved the best score.

4 PCB assembly inspection

The next two sections, 4.1 and 4.2, summarize the works about PCB assembly inspection covered by this survey. Their chronological line is illustrated in the Fig. 11.

Fig. 11
figure 11

Timeline of PCB assembly inspection methods covered in the present survey

Full size image

4.1 Traditional methods

Back in the 1990s, Honda et al. [34] and Loh [35] made part of the beginning of modern PCBA AOI inspection systems, but the constrained hardware capabilities seem to limit the development of efficient systems substantially. Both works rely on human-operated visual inspection machines, where they must take PCB pictures from small critical regions of interest and further apply digital image processing techniques to detect solder joint defects. Their primary focus is to evaluate the component solder quality with deterministic methods and computer-assisted human visual inspection.

In 2010, Ma et al. [36] claimed the solder joint visual inspection still lacked refinement, both in inspection method and standard metrics. They introduce a machine vision methodology to increase the image acquisition quality of small solder joints in PCBA pieces, including a custom illumination setup for better image quality. Further, they provide a mathematical computer-processing technique to analyze the solder joint images that could serve as feature extraction tools for future inspection works.

In Borthakur et al.’s work [37], they present a PCBA inspection method inspired by previous BPCB methods. Similar to Guan and Guo [15], this study uses wavelet transform to enhance the image processing quality. After multiple preprocessing steps of orientation adjustments and feature extraction, they apply image subtraction. When subtracted, it is expected to have a higher contrast where a component difference between the template and test images is found. The authors claim an 82.5% mean accuracy classification score on five different PCB types.

As does Ibrahim et al. [16], Nahar and Phadke [38] use MATLAB image processing toolbox to extract PCBA features and subtract the template image from the test one. They use the positive and negative feature results and apply deterministic geometric algorithms to find the object centroids and determine if they represent a defect or not. Before subtraction, both features must be under the same orientation, or registered. With 103 PBCA samples comprising a sum of 134 individual defects, the authors report 91.1% detection accuracy with no defect class discrimination.

Tsai and Huang [39] introduce a significative strategy modification for PCB inspection. While still providing a template matching strategy, they previously apply a Fourier transform in both images, leveraging their information to the frequency domain instead of spatial. The authors claim this strategy is insensitive to image noise and translation while keeping the frequency information of the image. Hence, the image subtraction in the frequency domain can return a more robust operation to contrast the present defects. Figure 12 illustrates some application examples of both BPCB and PCBA defect detection. One crucial limitation described in this study is the image resolution and information density. This method seems to lose effectiveness when used in whole PCB images. To address this issue, the authors apply a patch division over the images to extract smaller pieces, apparently solving the limitation with representative results.

Fig. 12
figure 12

Example of a solder joint defect detection using smaller image patches. a1 and b1 template and test partitioned images, respectively; a2 and b2 respective frequency domain images; c subtracted and reconstruced images in spacial domain, and d thresholded image

Full size image

Jessurun et al. [40] try to approach PCBA component detection with an unusual technique, introducing what they call shadow detection enhancement or SHADE. Their algorithm comprises multiple PCB photographs under different illumination angles to cast different shadows over the components. They combine these images using MATLAB software by subtracting the repetitive components under multiple light shadows, providing a template shadow. A distinct shadow is expected when applying the same technique to a defective test sample. The authors conclude their SHADE technique can enhance the image features but apparently does not surpass current SOTA inspection methods. Still, they suggest it may serve as an extra feature extraction for more robust methods.

4.2 AI methods

Acciani et al. [41] highlight the importance of PCB solder joint quality inspection and introduces an attempt to combine computer visual inspection with artificial neural networks. The first process consists of segmenting the PCB’s regions of interest containing solder joints on a relatively large scale. The AI they apply is a supervised learning multi-layer perceptron (MLP), trained with positive and negative solder joint image samples. Furthermore, the authors apply wavelet and geometric feature extraction methods to serve as input data to the MLP, which provides better discriminability potential. The experiments do not specify the size of the training dataset, but they report a 10-fold cross-validation test with a 98.8% accuracy score to classify five solder joint conditions.

Xie et al. [42] introduce one of the first genetic learning program (GP) applications in the PCB inspection field. While still using image registration and template subtraction techniques, this work introduces a sliding window strategy to provide more data samples with less mutual information from the whole PCB. Each sliding window must be previously annotated as positive or negative by its defect status to serve as training data. The windows are extracted from the image subtraction operation and applied to the genetic algorithm so it tries to learn the features that may represent a defect and the ones that do not. The authors defend this method as more robust than previous template-matching ones because the GP can learn the minor natural deviations from industrial mass production despite their negative defect statuses, which usually triggers high false positives in traditional methods. They report a false positive rate of 84,75%, which does not seem to be a high score compared to SOTA methods. However, the authors defend this by claiming that the GP method can be insensitive to PCB domain changes. This critical problem was later introduced in the general field of machine learning as “concept drift” [43], which may be a substantial innovative feature in relation to previous works.

Similar to the previous work, Wu et al. [44] use a genetic algorithm but, this time, focuses on PCBA solder joint inspection. The author’s intention is to mitigate the human-wearing operation of traditional AOI machines for solder inspection by taking their knowledge of solder joint quality and creating a set of 14 deterministic classification features to discriminate solder quality. They apply an MLP network to discriminate six main industrial solder joint classes, but they claim the MLP alone suffers from a high overfitting rate. They apply a GP technique to filter the input features to mitigate it, increasing the MLP’s generalization capacity. From 195 solder joint samples, they report a 98.46% true positive rate, which is higher than previous machine learning methods applied in the same set.

Li et al. [45] present a series of massive PCB component detection experiments. They assume the YOLOv3 architecture [46] as their baseline deep learning object detector, shown in Fig. 13. Instead of developing a defect inspection system, the authors focus on a previous analysis stage, the raw component detection task. Its main objective is to train the deep learning model to extract the precise position of each PCBA component. Figure 14 shows an output example of this work. The model can correctly classify an extensive table of 29 PCB component categories with an mAP score of 93.07%, where 14 components had 100% accuracy. To accomplish such substantial results, the authors introduce multiple essential techniques, since a significative architecture change in the original YOLOv3 until a thorough data augmentation stage, considering even virtual PCBAs generated with the Altium Designer PCB software, ending with a hybrid dataset composed of 50 different PCBAs summing 182,900 components. Despite the relatively high development complexity, the authors claim their system is a robust tool for integrating AOI machines and providing automatic PCBA components extraction, making possible ever-better defect inspections.

Fig. 13
figure 13

YOLOv3 model architecture, used by Li et al. [45]

Full size image
Fig. 14
figure 14

Improved YOLOv3 detection output example, by Li et al. [45]

Full size image

With the continuous SOTA success implementation of YOLO models, Caliskan and Gurkan [47] propose a method with the next YOLO generation, YOLOv4 object detector model [48] to detect and classify solder joint defects in PCBA. They train the model on 40,000 samples obtained at a real-production line and thoroughly annotate more than 70,000 solder joint defects for the model training. The authors highlight the model’s invariance to image orientation, ending the need for image registration techniques. Also, they claim the model’s robustness when applied to different types of PCBAs outside of the training dataset. The study reports 97% recall and 82% precision scores.

The recent study of Zhao et al. [49] introduces an innovative view of high-precision PCB inspection systems. The paper describes how recent SOTA works present high-accuracy inspection systems at the expense of massive datasets or complex deep learning development, which may be significantly challenging to adapt to industrial scenarios where the data domain is constantly changing due to new technologies and designs, i.e. the concept drift phenomenon. To surpass such issue, the authors propose a hybrid approach combining robust deterministic image processing techniques to extract essential features and simpler machine learning classification algorithms that may have lesser performance in comparison with deep learning models, such as YOLOs, but have higher data adaptation and efficiency, not requiring tens of thousands of training samples, which may be a more realistic manufacturing scenario under constant product adaptations.

5 AOI machines

Despite China’s majority market share in PCB production worldwide, their predominant method for inspection remains the traditional visual approach, relying on human operators aided by magnifying tools. This method’s widespread usage is not simply coincidental; rather, it can be attributed to its affordability and adaptability. Visual inspection incurs no preparative expenses and does not require acquiring specialized equipment. However, these advantages come at the expense of subjectivity in the inspection process, extended inference time, and elevated long-term labour costs. Additionally, manual visual inspection introduces the inherent risk of PCB damage due to the necessity of physically handling the boards [50].

As mentioned earlier in the sect. 1, automatic optical inspection (AOI) machines are widely regarded as the most desired approach for PCB inspection. Once correctly configured, these machines enable automated inspections without the need for physical contact, resulting in a substantially higher defect detection rate and no risk of damaging the board.

As denoted by many recent publications, what limits a massive AOI industrial implementation is its prohibitive costs [5, 7, 50, 51]. An AOI machine can comprise multiple different setups. Still, its main structure is composed of a pipelining of various processing stages, starting with the image acquisition booth, followed by a series of parameterized image processing algorithms, usually applying a template reference for image subtraction strategy after the feature extraction steps. Figure 15 illustrates a classical pipeline of an AOI application.

Fig. 15
figure 15

Overview of AOI system steps, by Sanguino [52]

Full size image

Once parameterized, modern AOIs can extract microscopical (e.g., 15 μm) image features from the template and test images in an automatic fashion, leading to high-standard accuracy results. Some AOI machines can achieve more than 99% of BPCB defect detection. However, their application is commonly bounded to a massively parameterized setup for each specific type of PCB, which can limit the total number of defect classes to be inspected [52]. Further, their inspection time is still a relevant issue, radically varying on the image processing pipeline, sometimes reaching more than ten seconds for one PCB inspection.

Current SOTA publications have been focusing on improving the AOIs limitations by implementing AI strategies on multiple stages of its pipeline [5, 51, 52], aiming for faster inference timings and a more self-adapting parameter setup, which could dramatically reduce its application costs.

6 Public PCB datasets

One of the first publicly available PCBA datasets was provided by Pramerdorfer and Kampel [53] in 2015, comprising 748 PCBs from a recycling facility with a high-standard photography protocol, which adds substantial domain variation. The annotations include segmentation of all PCBs, 9,310 integrated circuits bounding boxes along with their respective metadata.

Huang et al. [10] highlight the importance of public datasets to provide benchmark references to further research. However, PCB datasets are commonly composed of private data, and each respective inspection system development is applied to them. They describe how this negatively affects the broad development field once there is a lack of reference data for benchmark purposes. Hence, they provide an open dataset composed of 1,386 images taken from 10 different template BPCBs, comprising six defect types: missing hole, mouse bite, open circuit, short, spur, and spurious copper. All defects have bounding boxes annotation, and the rotated images have their orientation transformed for registration tasks.

Following the same idea of the previous work, He [31] proposes another BPCB benchmark dataset composed of the same six most common BPCB defects. Still, the images are standardized at 640 x 640 resolution and binarized this time. Moreover, they provide a collection of digitally synthesized defects to increase the positive class representativeness, comprising a dataset of 1,500 positive and negative BPCB image pairs, which allows for the implementation of supervised template-matching approaches.

Lu et al. [12] proposes a densely annotated PCBA dataset consisting of 9912 images of 31 PCBA samples, comprising 77,347 annotated components, including extensive metadata about every PCB. The FICS-PCB data set includes annotations of bounding boxes and other types of shapes, even from conductive tracks, being probably the biggest open PCBA dataset until 2020.

In 2022, Jessurun et al. [54] proposes a new PCBA dataset that is bigger than the FICS-PCB and probably the present biggest SOTA open dataset. Their FPIC dataset contains the most extensive annotation types. The authors describe a periodic data addition strategy to make it a general-purpose PCBA open dataset. They provide over 58,000 annotations of 73 double-layer PCB samples comprising bounding boxes, component classes, polygon traces, and multiple metadata. At least, FPIC is the first open PCB dataset with component segmentation annotations.

7 Discussion

The past 25 years of R&D concerning PCB quality inspection have undergone substantial improvements in hardware capability and methodology innovations. When taking a far-sight observation of all the analyzed papers in the present survey, it is possible to notice some robust patterns in PCB inspection methods. Figure 16 exemplifies these patterns, showing traditional and AI approaches’ most critical logical operations flowcharts. While some works focused on strictly traditional image processing techniques, others introduced AI modelling for the entire inspection process. Tables 1 and 2 summarize the primary proposed method of each paper and their respective significant claims.

Table 1 Papers summary—part 1
Full size table
Table 2 Papers summary—part 2
Full size table
Fig. 16
figure 16

Summary of most common AOI logical operations flowcharts

Full size image

Despite each peculiarity proposed in all the observed works, a few crucial steps have continued to serve as canonical modelling steps for AOI methods. Therefore, even the most modern AI approaches use, sometimes traditional techniques, such as the template-matching strategy. Hence, they are called hybrid approaches, so they do not rely on systematic repetitions of traditional or AI approaches. For example, [23, 42] utilized self-supervised AI models with a template-matching strategy, while [5, 11, 24, 32, 41, 44, 45, 47, 49] used similar AI techniques without template images, but direct test samples. On the other hand, [23, 31, 41, 42, 44, 47] used AI with traditional image registration, and [5, 29, 32, 45] shows that it is possible to run AI models that are robust to unregistered images.

Most traditional approaches use the template-matching technique with image registration, where a defect-free PCB image serves as a template for the test image. Multiple possible operations exist between both images, such as image subtraction and XOR logical operation. Thus, the core template-matching concern is detecting the differences between the samples and applying some specific geometric transformations to filter and classify them as defective. As per the full-AI approaches, a training dataset composed of multiple PCB samples must be provided so the model can learn how to distinguish which and where the defects are.

The hybrid approaches try to combine the most valuable features of the previous techniques to compensate for their disadvantages. For example, it is possible to train a template-matching AI model that learns to automatically compare the image-pair, ending the necessity of complex image pre-processing algorithms and even image registration, becoming a more generalizable solution for a wider variety of PCBs [31]. However, [49] demonstrates the usefulness of some generally old image preprocessing techniques combined with simpler machine learning algorithms to better adapt under concept drift scenarios, substantially improving the inspection potential.

When concerning the achieved accuracy and detection performances, it is hard to discern which method is reliably better than the other because there seems to be a considerable lack of representative public PCB datasets for benchmark purposes. Since private companies mainly provide these types of research, each of them provides a particular and private dataset, which makes a direct comparison between different methods’ performance an unfair metric. Some authors have been trying to address this issue by systematically collecting PCB samples, annotating them, and making them available to the public [10, 31, 54]. However, most of the analyzed papers still use private data. Furthermore, these private datasets often lack a substantial number of image samples to provide reliable statistical relevance, and therefore, the present research is not suited for direct performance comparisons.

This limitation on representative PCB datasets is one of the most significant restrictions of AI approaches in the field. Current SOTA’s successful deep learning image applications are strongly related to datasets often comprising over 100,000 samples and not rarely reaching over 10,000,000 samples. In contrast, the two most extensive open-PCB datasets have about 10,000 images, which are far from insignificant but still leave room for development.

Besides the direct performance comparison issue, it is still possible to observe in the big picture a chronological improvement in both accuracy rates, public dataset availability, and the number of defects detected by a single approach. Such enhancements seem to be correlated with an increasing demand for AI-ready approaches. Regarding the analyzed papers, Fig. 17 illustrates the relative proportion of traditional vs AI approach works accumulated over the last 25 years, where until 2004, all studies relied on conventional approaches with systematic algorithms that required many preprocessing steps and were restricted to a small range of possible defects inspection. Following the “2010 The rise of deep learning” [55], when DL became considerably notorious with optimized learning algorithms and more capable hardware, 33% of the PCB inspection works were led by AI techniques. By 2022, with an exponential growth rate, more than half of them had applied more complex and robust AI models capable of more generalized inferences and fewer preprocessing steps.

Fig. 17
figure 17

Accumulated relative proportion over years of papers concerning their main PCB inspection approach; if included only traditional techniques or also with AI

Full size image

Finally, the increasing AI approach rate not only brings substantial performance improvements but also seems to unveil a complex challenge in most PCB manufacturing scenarios: the concept drift problem. Every time a new PCB design starts to be produced, it faces a high probability of comprising new components, new architectures, and new organizations, and hence, new visual features, even more with PCBA once they have more varied components than BPCB. This problem represents the weak spot of AI models previously trained with specifically older PCBs that did not contain the new features in the following pieces’ generation, creating an information bias that relentlessly impacts the model’s accuracy.

Trying to address these concept drift scenarios, some modern works introduce a combination of hybrid techniques that aims to develop a model that can be trained in older PCB samples but somehow become less sensitive to new board designs and, therefore, robust to concept drift [42, 49]. However, such approaches still struggle with raw accuracy scores in relation to other solutions that are ready for specific PCBs and share a tiny proportion of the whole PCB inspection research spectrum aware of such problem, as observed in FIG. 18.

Fig. 18
figure 18

Relative proportion of the analyzed papers that were concerned with the concept drift problem

Full size image

8 Conclusion

The present survey introduces a chronological and categorical analysis of PCB quality inspection papers over the past 25 years. It describes how PCB inspection environments are usually divided into two categories, BPCB and PCBA, observing the core traditional techniques used until the advent of modern real-time AI solutions.

Regarding the BPCB inspection, recent results using DL models seem to almost completely solve the tracks and paddings inspection with nearly 100% accuracy most of the time. Nevertheless, the PCBA inspection did not follow the same results. This represents a more complex challenge due to a higher component variety and updating rate, which evoke concept drift problems for AI models. Still, modern PCBA inspection methods are capable of detecting thousands of different electronic components in fractions of a second, which increases the efficiency of previous AOI machines’ inspection quality and affordability.

One of the most critical recent R&D limitations is the lack of representative public PCB datasets to serve as benchmark references to multiple works and provide a general performance rank to compare different solutions, which is why the present survey’s tables lack one objective performance metric column since each paper’s result are strictly related to its specific testing data. As an example, some modern AI publications reviewed in the present survey report high-standard results in the field but still applied to private datasets, which restricts the reproducibility of such achievements. Nonetheless, some valuable PCB datasets have been publicly disposed of in the last five years, which could lead to significant field improvements for the following publications.

Regarding the solutions’ data efficiency, multiple studies rely on AI models trained on thousands of samples composed of high-cost private datasets. This is not a well-desired engineering solution since industrial environments are often used to changing their mainline products and, thus, are vulnerable to concept drift. Hence, further development in data-efficient systems is expected to address this issue regarding deep learning applications.