Introduction

Solar power is the fastest-growing source of new energy according to the International Energy Agency. Today, Photovoltaics (PV) is the third renewable energy source in terms of global capacity (Masson and Brunisholz 2016). PV employs solar panels composed of solar cells made of semiconducting materials to convert light into electricity. The main component of conventional solar cells is crystalline silicon (c-Si), appearing with a monocrystalline or polycrystalline structure.

Monocrystalline silicon (mono-Si) cells present an octagonal shape cut from cylindrical ingots and an uniform look that indicates high-purity. Mono-Si cells are expensive and efficient. On the other hand, polycrystalline or multicrystalline silicon (multi-Si) cells have a square shape and present a “frost” texture caused by different crystals molten and solidified together. Making polycrystalline silicon is simpler and costs less, but multi-Si cells are less efficient than mono-Si cells. Figure 1 shows the typical structure of Mono-Si and multi-Si solar cells, appearing as a grid with strips made of copper plated with silver, called busbars. Busbars act as conductors for the current produced from the incoming photons that are connected by perpendicular thinner strips called fingers that collect the current and deliver it to the busbars.

Fig. 1
figure 1

(Source: Ersol Solar Energy AG, Germany, under GNU Free Documentation License)

A conventional crystalline silicon solar cell. Electrical contacts made from busbars (the larger silver-colored strips) and fingers (the smaller ones) are printed on the wafer.

Full size image

PV manufacturing often produces defects that appear in the cells surface due to manipulation errors, excessive mechanical pressure, defects on the material…, etc. (Qian et al. 2017). PV defect taxonomy is a topic of discussion. Researchers usually don’t agree on the categories they use, or they employ the same term with different meanings. The term Shunt is one of the most used and most ambiguous (Breitenstein et al. 2004). A Shunt is defined as a local increase in the dark forward current of a cell, that affects its efficiency by reducing the Fill Factor (FF) and the Open Circuit Voltage (VoC). Shunts can be caused by material defects or they can be process induced (Correia et al. 2006). Up to nine different types of Shunts are described in literature, caused by cracks, holes, scratches, aluminum particles, Schottky-type contacts, faulty edge insulations, defects on the material and others (Breitenstein et al. 2004). Most studies avoid the taxonomy problem by focusing on specific defects such as Finger Interruptions (Tseng et al. 2015) or Cracks (Anwar and Abdullah 2014). We consider three types of defects using their visual appearance in Electroluminescence (EL) images as criterion: Area Defects, Finger Interruptions and Cracks, Fig. 2 shows examples of defined categories.

Fig. 2
figure 2

Categories of defects. Area Defects (a, d), Finger Interruptions (b, e) and Cracks (c, f)

Full size image

Cell defects are usually invisible in ordinary visual inspections, and nowadays researchers use techniques based on Infrared (IR) and Electroluminescence (EL) imaging. IR imaging assumes that in damaged regions, solar energy is not properly converted into electricity, heating the solar cell up as a result so defects can be detected with an IR camera. However, IR cameras are expensive and have relatively low resolution, making small defects difficult to detect (Deitsch et al. 2018). In EL imaging, current is applied to the cell, inducing EL emission in the MIR spectrum (1150 nm). The light emitted by the connected cell can be captured by a cooled CCD camera. In EL images, defects appear as dark areas in the image. The EL technique can be used at high resolutions and EL imaging does not suffer from blurring due to lateral heat propagation. However, the analysis of EL images is typically a manual process that is expensive, time-consuming, and requires expert knowledge (Deitsch et al. 2018). Automatic image analysis in industrial environments with is a rapidly evolving field (Gonzalez-Val et al. 2019). And several researchers are focusing on developing automatic techniques to detect defects in EL images. Though as far as we know, there are no published results regarding location of defects on a pixel level, many contributions are worth mentioning:

Chiou et al. (2011) used a near infrared (NIR) imaging system to detect Cracks in multi-crystalline silicon cells. Claiming an accuracy of 0.99 in detected defects.

Li and Tsai (2012) used wavelet coefficients to distinguish local defects in multicrystalline solar wafers, obtaining an accuracy of 0.98 in a set of 46 defective samples and 50 defect free samples.

Tsai et al. (2012) used Fourier image reconstruction to detect small Cracks, breaks, and Finger Interruptions in polycrystalline cells from EL images. They detected the defects by comparing the differences between the original image and an its spectral representation. This work was criticised for not being able to detect defects with complex shapes (Deitsch et al. 2018). We also find that the small sample used, 15 defective samples and 308 defect-free samples, is not enough to support the claimed 1.00 identification rate.

Tsai et al. (2013) introduced a supervised learning method for identification of defects using Independent Component Analysis (ICA) in EL images. They used a training stage with 300 solar cell and an inspection stage, where each solar cell subimage is reconstructed as a linear combination of the learned basis images. They achieved a mean recognition rate of 0.93 for a set of 80 test samples.

Anwar and Abdullah (2014) developed an algorithm for the detection of Cracks in EL images of polycrystalline solar cells. They used anisotropic diffusion filtering followed by shape analysis. The authors used 600 randomly selected solar cells: 240 for training and 360 for testing. They reported a sensitivity of 0.97, a specificity 0.80 and an accuracy of 0.88 with a Support Vector Machine (SVM). This technique is limited to Cracks detection.

Rodriguez and Garcia (2014) proposed a SVM to detect defects in a dataset of 47 EL images of cells with Shunts and Cracks. They obtained a precision of 0.91 and 0.86 respectively. The authors counted classified defects. Additionally, this technique relies on the capacity of the SVM to separate connected busbars and other features from the useful areas of the cell. In our experiments, we found this approach only usable in very specific types of designs.

Tseng et al. (2015) proposed a method to detect Finger Interruptions in multicrystalline solar cells using EL imaging. They applied a binary clustering technique to features from candidate regions, achieving and accuracy of 0.99 in 60 multicrystalline solar cells. The main disadvantage of this technique is that it can only be applied to Finger Interruptions.

Xiaoliang et al. (2017) used an EL segmentation technique combined with a morphological post-processing. The original research is not available in English, but an analysis by Qian et al. (2017) reports a precision of 0.87, a recall of 0.70; and an f-measure of 0.78 using an unspecified region unit. A separate experiment reports 1 misclassification in 60 defect-free images and 40 defective images.

Deitsch et al. (2018) used a SVM and an end-to-end deep Convolutional Neural Network (CNN) to achieve an average accuracy of 0.82 and 0.88 respectively when detecting cells with defects in EL images. With this technique, they predicted a defect likelihood that may lead to efficiency loss in a cell. The authors did not provide quantitative metrics of the results.

Chen et al. (2018) used a Convolutional Neural Network (CNN), to analyse multispectral EL images and detect visual defects in polycrystalline solar cells. They obtained an average precision of 0.88 for 6 types of defects defined in visual categories such as thick lines, color differences or dirty cells.

In summary, previous works showcase how useful computer vision techniques can be to detect defects in solar cells, achieving high accuracy and precision. However, an analysis of these researches also exposes problems. Most of these works employ few samples, are limited to classification of solar cells into defective or non-defective or are specific for one type of defect. More importantly, they did not report result metrics in pixels, so it is unclear how reliable computer vison techniques are when detecting the position of a defect. This is a relevant problem cause most faults due to Shunts and Cracks may be repaired by cutting or isolating a piece of the cell using laser technology (Schmauder et al. 2012), and knowing the type and location of each defect is crucial for repairing.

Recently, some authors (Rodriguez-Araujo and Garcia-Diaz 2014; Schmauder et al. 2012) theorised about the feasibility of cell repair automation based on EL imaging. However, the ability of a Computer Vision system to classify multiple defects and find their accurate location and extension remains unknown and such system was never constructed or tested. We prove that state of the art computer vision techniques can detect the type and location of common defects in EL monocrystalline cell images. We also show the design and results obtained with Cell Doctor, to our knowledge, the first built system used to diagnose and repair solar cells in manufacturing conditions.

Materials and methods

Cell Doctor is a fully automatic system that can be integrated in a manufacturing line. It detects manufacturing problems and repairs them with no human intervention. Figure 3 shows Cell Doctor in operation.

Fig. 3
figure 3

Cell Doctor in operation

Full size image

The main components of Cell Doctor are: an EL Station, a Laser Station and a I–V Station. In these stations, stages of analysis and treatment take place. An industrial OMRON adept Viper 650 robot equipped with a vacuum gripper moves the solar cell from stage to stage and the whole process is controlled by a Programmable Logic Controller (PLC) and an industrial PC computer that exchange information with each other and with the other elements of the system. Figure 4 shows the design of Cell Doctor. It works as follows:

  1. 1.

    Conveyor Belt The conveyor belt brings packs of solar cells, candidates for inspection, into Cell Doctor. This component is the input of the system.

  2. 2.

    Vacuum Gripper Belt The vacuum gripper belt takes solar cells from the conveyor belt and moves them to the centering support, one at a time, to be processed.

  3. 3.

    Centering Support The function of the centering support is to mechanically align the incoming cells into a predefined position so the robot can operate with them with sub-millimetre accuracy.

  4. 4.

    EL station This is an Electroluminescence inspection chamber. The robot places the solar cell in a precise position into the station and the door closes. Isolating the chamber from the outside. Then, a pin connector is placed automatically on the busbars. The PLC triggers all these operations and communicates the resulting status to the PC. Then, the PC triggers simultaneously the power source of the Electroluminescence process and a Cooled Deep Depletion CCD camera. The camera is placed in a zenithal position inside the chamber and cooled to –10 °C. The camera shutter is open for 8s while the cell is connected to 7.2 A. In the last step, the PC analyses the obtained image and performs the adequate diagnosis, planning the repair decisions to be made.

  5. 5.

    Laser Station This is where solar cell treatment happens. The process initiates when the PC calculates the treatment plan and sends the corresponding instructions and parameters to the laser station and the PLC. Then, triggered by the PLC, the robot places the cell in a predefined position of the chamber and the door closes. The chamber is equipped with a Q-switched Nd:YAG laser with a fundamental wavelength of 1064 nm, nanosecond pulses and an average power of 100 W. Finally, the laser performs the isolation or cutting process using a galvanometer scanner to direct the laser radiation over the cells.

  6. 6.

    I–V Station After the laser process (isolation/cutting), solar cells are placed in the last station. This is a testing station, where a Solar Simulator provides the I–V curve of the solar cells and the main I–V parameters; such as fill factor or percent efficiency. This stage allows us to determine the cell’s output performance and solar efficiency.

  7. 7.

    Output Tray The output tray is the output of the Cell Doctor.

Fig. 4
figure 4

Architecture of Cell Doctor

Full size image

Treatment plan

Cell Doctor examines the EL images to locate the position of Area Defects, Finger Interruptions and Cracks (see Fig. 2). These defects affect the cell efficiency and durability in different ways and we used the criterion of experts to define a set of treatment rules, that Cell Doctor uses to create a treatment plan for each specific case.

  1. a.

    The laser engraves an isolation ellipse around any Area Defect not located in the vicinity of a busbar.

  2. b.

    The laser cuts the cell in half, to discard a part containing Area Defects close to a busbar (~ 2 mm).

  3. c.

    When the cell has a Crack, we cut and discard the half of the cell containing the crack if possible.

  4. d.

    Cells that contain defects described in b. or c. affecting both halves of the cell, are discarded.

  5. e.

    We do not treat Finger Interruptions.

Pose estimation and calibration

The first step in cell analysis process is to calibrate the camera system estimating the transformation from image coordinates to space coordinates understood by the laser station (laser coordinates). Calibration is a key factor in the repair process, since treatment depends not only on the spatial position of the defect but also on its relative position in the cell (i.e. we need to consider the proximity of cracks to busbars since when a Crack affects a busbar, experts consider the cell unrecoverable).

Mechanical components such as the robot arm and the Centering Support (Fig. 4.3) introduce a source of error, adding variability to the positioning of the cell that cannot be eliminated since there is no camera to reacquire the position in the Laser Station (Fig. 4.5). We control this error with a careful configuration of each component, achieving a variability in position < 0.1 mm and a variability in rotation < 0.1°. This magnitude is negligible and we assume that the robot will always place the cell in the same position.

The system detects the position of each cell in the EL image and then finds the homography that projects its position into the laser coordinates (Rodríguez 2014). Knowing the orientation (pose) of the cell in the EL Station, allows the detection of undesired features in the image. For example, solar cells have different shapes or marks depending on the manufacturer; and the EL station can use busbars connectors of different types that interfere in the results. This technique also makes the process robust to the position and orientation of the camera.

We detect the position and pose of the cell in the images using the following steps. Figure 4 details this algorithm:

  1. 1.

    A threshold transformation separates in the image the area occupied with the cell from the background.

  2. 2.

    The Douglas-Peucker algorithm (Prasad et al. 2012) finds the polygon that describes best the resulting area.

  3. 3.

    The orientation from the polygon is the initial estimation of the cell position.

  4. 4.

    Canny edge detector and a Hough transform techniques detect with accuracy the edges of the cell in the image.

  5. 5.

    Edge information refines the initial pose estimation. We use a least-squares scheme to match resulting refined corners with reference laser coordinates with an homography transformation.

Feature detection

An important step in our process is to detect the connected busbars and possible branding marks before analysing the effective cell surface. This procedure solves the problem in Rodriguez-Araujo and Garcia-Diaz (2014) where these features can be usually diagnosed as defects when tested with some experimental designs (Fig. 5).

Fig. 5
figure 5

Pose estimation algorithm. Solar cell image, with artificially added underexposed and overexposed areas (0). Threshold mask (1). Polygon that describes the threshold mask using the Douglas-Peucker algorithm (2). Polygon Orientation (3). Canny edge detector (4). Initial pose estimation (outer rectangle) and refined pose (inner rectangle) (5)

Full size image

Figure 6 details the busbar detection algorithm. The algorithm works as follows:

  1. 1.

    Using the cell pose, we align the image so that the connected busbars appear horizontally.

  2. 2.

    A median filter normalizes and smooths the images. This standardizes the intensity range and removes most of the texture detail from the image.

  3. 3.

    A threshold segmentation separates the dark regions with no EL activity that may belong to the busbars.

  4. 4.

    A closing morphological operation fills the holes in the segmented image.

  5. 5.

    We sample the segmented image in predefined vertical bands and transform each band in a vector adding up rows horizontally. That is, a vector represents a sampled band in the image and each vector number represents the number of pixels with no EL activity in a row of the band.

  6. 6.

    A threshold operation binarizes the vectors, setting to 0 the pixels with significant EL activity in the corresponding band row.

  7. 7.

    We find the 0-to-1 or black-to-white (BW), and the 1-to-0 or white-to-back (WB), transitions in each vector. Those transitions correspond to the edges of the busbars in the current band. Finally, we adjust the result by looking for additional edge values, filter invalid points and add missing values by interpolation.

  8. 8.

    A linear regression model obtains the lines that define the edges of the busbars.

Fig. 6
figure 6

Busbar detection algorithm. Solar cell aligned so busbars appear horizontally (1). Normalization and median filtering (2). Threshold segmentation (3). Closing operation (4). Band sampling (5). Threshold binarization (6). Locating edge transitions (7). Linear regression detects busbar edges (8)

Full size image

After detecting the busbars in the image, we use an algorithm to find special branding marks in the images. Our algorithm uses the Circle Hough Transform (Illingworth and Kittler 1987) to detect the branding marks in the solar cells, but other types of marks may require a different pattern matching strategy. Figure 7 shows an example of the branding mark detection process.

Fig. 7
figure 7

Branding mark detection. Solar cell image after pose estimation (a light blue polygon marks edges), and after busbar location (horizontal green lines created from blue sampled points show the location of the busbars); the branding mark appears as a circular pattern in near the corner of the cell surface (a). Detected branding mark (A bright blue circle marks the detected mark) (b)

Full size image

Solar cell diagnosis

To diagnose defects in a solar cell, a machine learning algorithm analyses the effective surface of the cell from EL images. After excluding the area outside the cell, the edge pixels, the busbars, and the branding marks from the analysis; Cell Doctor classifies then each pixel into a correct category and 3 different categories of defects: Area Defects, Finger Interruptions and Cracks (see Fig. 2). The classification strategy follows this pipeline:

Step 1 Images are first normalized, then downsampled to a size of 512 × 512 pixels and finally converted to a black and white colour space with 256 possible levels of grey.

Step 2 A set of Log-Gabor Filters (Field 1987) represents the spatial and frequency information around each pixel. Log-Gabor filters are a logarithmic transformation of the original Gabor filters that, according to some authors, can model the visual cortex of mammalian brains (Marĉelja 1980). Gabor filters allows the simultaneous analysis of space and frequency in a EL image by capturing specific frequencies in specific orientations around each pixel. We used 20 filters with central frequencies f0 = (1/3, 1/6, 1/12, 1/24, 1/48) and with orientations θ0 = (0°, 45°, 90°, 135°). Our filters have a bandwidth in frequency of B = 0.8145 and an angular bandwidth of Bθ = 1.2188. The output of each Log-Gabor filter is a complex number so, for each filter; we store its real and imaginary part, its magnitude and its phase. As a result, each pixel is described with 80 features and the original pixel value, adding up to a total of 81 features.

Step 3 A Principal Component Analysis (PCA) obtains a new set of features, called principal components, constructed as a linear combination of the original set of Log-Gabor Filters. Principal components are built iteratively, and each new linear combination has maximum variance for the data, being uncorrelated with the previous linear combinations. We chose to preserve all the variance of the original set in this step, obtaining a new set of 81 linearly uncorrelated features.

Step 4 Our feature set is standardized by removing the mean and scaling the data to a variance of 1.

Step 5 A random forest model (Breiman 2001) classifies each pixel into one of the 4 different categories. A random forest is a technique that aggregates several decision trees to create a composite model. Our algorithm operates by repeatedly sampling with replacement the original training set to obtain 35 new training sets that train 35 decision trees with a maximum depth of 15. With each tree split in the training stage, we randomly select a subset of 25 features, reducing significantly the complexity of the final model. The result of the algorithm is the average of the results of the 35 individual trees.

Results and discussion

We trained, validated and tested our model using 67 EL images of solar cells with different types of defects. These images were labelled by a set of experts, and regions marked as too unclear or ambiguous were ruled out of the set. The resulting dataset consists in approximately 10 million samples where each sample represents a set of features from a pixel.

We then divided our data into a Training/Validation set that contains 80% of the samples, and a Test dataset, containing 20% of the samples. To select and tune our model, we performed an exhaustive parameter search with a fivefold cross validation strategy. This strategy randomly divides the Training/Validation set into 5 subsets, each one with 20% of its samples. The training process uses 4 of the sets for training and 1 for validation. The training is repeated 5 times, using a different set for validation each time.

Once the final architecture was defined, we trained a new model with the entire Training/Validation dataset and used the test dataset to obtain an unbiased evaluation. Table 1 summarizes the structure of the datasets and details the number of samples in each category. Table 2 details the obtained results.

Table 1 Dataset structure
Full size table
Table 2 Performance metrics for each defect in the first experiment at pixel level
Full size table

Results from Table 2 show an excellent performance in all metrics at pixel level (above 90%), with the exception of precision values for Finger Interruptions and Cracks that show values around 60%. These results show that a significant part of the pixels detected as Finger Interruptions or Cracks, are False Positives. This is an expected result, due to ambiguity in defect boundaries and the arbitrary width of Finger Interruption and Crack labels. To remove the variability in the performance of the experts from the results, we conducted a second experiment.

In the second experiment, the images of all datasets were classified again, this time frame by frame. This is how Cell Doctor would function in real conditions and this operation mode allows us to preserve the spatial reference of each label. Thus, we performed a dilation morphological operation in the defect labels when computing false positives. With this simple operation, variability of manual labelling is accounted for. Table 3 shows the obtained results.

Table 3 Performance metrics for each defect in the second experiment
Full size table

Results from Table 3 show an increase in precision from 0.57 to 0.77 for Finger Interruptions and from 0.62 to 0.99 for Cracks. These results prove the overall reliability of Cell Doctor.

Cell Doctor cannot be directly compared with recent works, since they not provide pixel metrics, usually reduce the problem to binary classification, use different types of solar cells and in most cases provide only partial results, being Accuracy and Precision the most common metrics. Table 4 shows a comprehensive summary of results from recent works.

Table 4 Summary of previous reported results
Full size table

Analysing results from Table 4, Cell Doctor obtains an accuracy similar to Tseng et al. (2015), a technique specific for Finger Interruptions; and improves significantly all other reported metrics.

To test Cell Doctor in real operation conditions, we performed a final experiment with 50 new images recorded with different experimental conditions. We used two different busbars connectors, different camera shutter speeds (ranging from 6 to 8 s) and small amperage changes in the EL station (± 1 A). Cell Doctor performance in this experiment was obtained by comparing its results with those detected by a set of experts. Table 5 shows the number of correctly and incorrectly identified defects of each type and Table 6 shows the resulting performance metrics.

Table 5 Classification results for the test with factory conditions
Full size table
Table 6 Performance metrics for the test with factory conditions
Full size table

Results from Table 6 prove the generalization ability of Cell Doctor obtaining similar results to our previous experiments and show that our technique can be used successfully in industrial environments with real manufacturing conditions.

Conclusions

Solar cells represent nowadays one of the most important sources of clean energy in modern societies, and to meet the production requirements of the market, it is important to be able to detect and repair defects introduced in the manufacturing process.

We introduce Cell Doctor, a system based in computer vision that when integrated in a manufacturing line can accurately detect and diagnose defects in real time in the manufactured cells. Cell Doctor performs then a treatment plan that allows to recover or discard defective cells, reducing wastes and improving the efficiency on the manufacturing process.

In this paper, we present the computer vision approach used for the automatic detection and diagnosis, and evaluate it against a novel dataset of Electroluminescence Infrared imaging of PV cells.

Obtained results show a significant improvement in accuracy, precision and recall compared to previously reported results. Achieving an accuracy and recall above 90% for Cracks, Area Defects and Finger interruptions; and a Precision of 77% for Finger Interruptions and above 90% for Cracks and Area Defects.

Our experimental methodology shows for the first time that the position of these defects can be obtained with accuracy using image inspection, and the automatic diagnosis system presented here, was integrated in Cell Doctor with a treatment plan allowing for the automatic recovery or discardment of defective cells in industrial manufacturing plants, reducing wastes and improving the efficiency on the manufacturing process.