Skip to main content
SpringerLink
Log in

Meta-FSDet: a meta-learning based detector for few-shot defects of photovoltaic modules

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In the initial stage of the establishment of photovoltaic (PV) module production lines or the upgrading of production processes, the available data for some defects are limited. The detection for these data-scarce defects of photovoltaic (PV) modules is a challenging task, because most detectors hardly extract meaningful high-level features from limited information of several samples. To address this challenge, this paper proposes a novel end-to-end meta-learning based few-shot detector (Meta-FSDet). Firstly, a prototype vector extractor (PVE) is proposed, which utilizes locational prior knowledge of data-scarce defects to extract corresponding prototype vectors. Then, a novel saliency highlighting network (SHN), which measures the similarity between extracted prototype vectors from PVE and each spatial-position vector of the query feature map, is proposed to infer a class-agnostic saliency map, in which the defective areas are well highlighted. Next, a remodel region proposal network (RRPN), which further leverages the saliency map from SHN to pixel-wisely reweight the inputted feature map of region proposal network (RPN), is presented to better infer meaningful high-level region-of-interest (RoI) features of data-scarce defects. Finally, these RoI features from RRPN are further processed by downstream classification and regression module to generate final prediction results. Extensive experiments on PV module dataset demonstrate our Meta-FSDet owns superior performance compared to other existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., & Ahmad, A. (2020). Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning. Solar Energy, 198(1), 175–186. https://doi.org/10.1016/j.solener.2020.01.055

    Article  Google Scholar 

  • Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., Zhao, X., & Ahmad, A. (2019). CNN based automatic detection of photovoltaic cell defects in electroluminescence images. Energy, 189(15), 116–319.

    Google Scholar 

  • Balzategui, J., Eciolaza, L., & Maestro-Watson, D. (2021). Anomaly detection and automatic labeling for solar cell quality inspection based on Generative Adversarial Network. Sensors, 21(13), 4361. https://doi.org/10.3390/s21134361

    Article  Google Scholar 

  • Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar cell surface defect inspection based on multispectral convolutional neural network. Journal of Intelligent Manufacturing, 31(2), 453–468. https://doi.org/10.1007/s10845-018-1458-z

    Article  Google Scholar 

  • Chen, H., Wang, Y., Wang, G., & Qiao, Y. (2018). Lstd: A low-shot transfer detector for object detection. In Proceedings of the AAAIConference on Artificial Intelligence (Vol. 32, Issue 1). https://doi.org/10.1609/aaai.v32i1.11716.

  • Dunderdale, C., Brettenny, W., Clohessy, C., & van Dyk, E. E. (2020). Photovoltaic defect classification through thermal infrared imaging using a machine learning approach. Progress in Photovoltaics: Research and Applications, 28(3), 177–188. https://doi.org/10.1002/pip.3191

    Article  Google Scholar 

  • Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303–338. https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  • Fan, Q., Zhuo, W., Tang, C. K., & Tai, Y. W. (2020). Few-shot object detection with attention-RPN and multi-relation detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4013–4022). https://doi.org/10.1109/CVPR42600.2020.00407.

  • Han, G., Huang, S., Ma, J., He, Y., & Chang, S. F. (2022). Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 780–789. https://doi.org/10.1609/aaai.v36i1.19959

    Article  Google Scholar 

  • Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021). Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (Early Access). https://doi.org/10.1109/TPAMI.2021.3079209

    Article  Google Scholar 

  • Hsu, C. Y., & Chien, J. C. (2022). Ensemble convolutional neural networks with weighted majority for wafer bin map pattern classification. Journal of Intelligent Manufacturing, 33(3), 831–844. https://doi.org/10.1007/s10845-020-01687-7

    Article  Google Scholar 

  • Huang, F., Wang, B. W., Li, Q. P., & Zou, J. (2021). Texture surface defect detection of plastic relays with an enhanced feature pyramid network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01864-2

    Article  Google Scholar 

  • Kang, B., Liu, Z., Wang, X., Yu, F., Feng, J., & Darrell, T. (2019). Few-shot object detection via feature reweighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 8420–8429). https://doi.org/10.1109/ICCV.2019.00851.

  • Kim, Y., Kim, T., Youn, B. D., & Ahn, S. H. (2022a). Machining quality monitoring (MQM) in laser-assisted micro-milling of glass using cutting force signals: An image-based deep transfer learning. Journal of Intelligent Manufacturing, 33(6), 1813–1828. https://doi.org/10.1007/s10845-021-01764-5

    Article  Google Scholar 

  • Kim, T., Lee, J. W., Lee, W. K., & Sohn, S. Y. (2022b). Novel method for detection of mixed-type defect patterns in wafer maps based on a single shot detector algorithm. Journal of Intelligent Manufacturing, 33(6), 1715–1724. https://doi.org/10.1007/s10845-021-01755-6

    Article  Google Scholar 

  • Li, A., Huang, W., Lan, X., Feng, J., Li, Z., & Wang, L. (2020). Boosting few-shot learning with adaptive margin loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12576–12584). https://doi.org/10.1109/CVPR42600.2020.01259.

  • Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 2980–2988). https://doi.org/10.1109/TPAMI.2018.2858826.

  • Liu, B., Kang, H., Li, H., Hua, G., & Vasconcelos, N. (2020a). Few-shot open-set recognition using meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 8798–8807). https://doi.org/10.1109/CVPR42600.2020a.00882.

  • Liu, Q., Majumder, O., Achille, A., Ravichandran, A., Bhotika, R., & Soatto, S. (2020b, August). Incremental few-shot meta-learning via indirect discriminant alignment. In European Conference on Computer Vision (ECCV) (pp. 685–701). https://doi.org/10.1007/978-3-030-58571-6_40.

  • Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  • Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 618–626). https://doi.org/10.1007/s11263-019-01228-7.

  • Su, B., Chen, H., Chen, P., Bian, G., Liu, K., & Liu, W. (2020). Deep learning-based solar-cell manufacturing defect detection with complementary attention network. IEEE Transactions on Industrial Informatics, 17(6), 4084–4095. https://doi.org/10.1109/TII.2020.3008021

    Article  Google Scholar 

  • Su, B., Chen, H., & Zhou, Z. (2021b). BAF-detector: An efficient CNN-based detector for photovoltaic cell defect detection. IEEE Transactions on Industrial Electronics, 69(3), 3161–3171. https://doi.org/10.1109/TIE.2021.3070507

    Article  Google Scholar 

  • Su, B., Chen, H., Zhu, Y., Liu, W., & Liu, K. (2019). Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor. IEEE Transactions on Instrumentation and Measurement, 68(12), 4675–4688. https://doi.org/10.1016/j.energy.2019.116319

    Article  Google Scholar 

  • Su, Y., Tao, F., Jin, J., & Zhang, C. (2021a). Automated overheated region object detection of photovoltaic module with thermography image. IEEE Journal of Photovoltaics, 11(2), 535–544. https://doi.org/10.1109/JPHOTOV.2020.3045680

    Article  Google Scholar 

  • Sun, B., Li, B., Cai, S., Yuan, Y., & Zhang, C. (2021). FSCE: Few-shot object detection via contrastive proposal encoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 7352–7362).

  • Tang, W., Yang, Q., Xiong, K., & Yan, W. (2020). Deep learning based automatic defect identification of photovoltaic module using electroluminescence images. Solar Energy, 201(1), 453–460. https://doi.org/10.1016/j.solener.2020.03.049

    Article  Google Scholar 

  • Wang, Y. X., Ramanan, D., & Hebert, M. (2019). Meta-learning to detect rare objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9925–9934).

  • Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3–19). https://doi.org/10.1007/978-3-030-01234-2_1.

  • Wu, X., Sahoo, D., & Hoi, S. (2020). Meta-RCNN: Meta learning for few-shot object detection. In Proceedings of the 28th ACM International Conference on Multimedia (pp. 1679–1687). https://doi.org/10.1145/3394171.3413832.

  • Yan, X., Chen, Z., Xu, A., Wang, X., Liang, X., & Lin, L. (2019). Meta R-CNN: Towards general solver for instance-level low-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9576–9585). https://doi.org/10.1109/ICCV.2019.00967.

  • Zhang, G., Cui, K., Wu, R., Lu, S., & Tian, Y. (2021a). PNPDet: Efficient few-shot detection without forgetting via plug-and-play sub-networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 3823–3832). https://doi.org/10.1109/WACV48630.2021a.00387.

  • Zhang, G., Luo, Z., Cui, K., & Lu, S. (2021b). Meta-detr: Few-shot object detection via unified image-level meta-learning. arXiv:2103.11731, 2(6).

  • Zhao, Y., Zhan, K., Wang, Z., & Shen, W. (2021). Deep learning-based automatic detection of multitype defects in photovoltaic modules and application in real production line. Progress in Photovoltaics: Research and Applications, 29(4), 471–484. https://doi.org/10.1002/pip.3395

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by National Natural Science Foundation of China under Grant 62073117, Grant U21A20482, Grant 62173124 and Natural Science Foundation of Hebei Province under Grant F2019202305.

Author information

Authors and Affiliations

  1. College of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, 300130, China

    Shijie Wang, Haiyong Chen, Kun Liu, Ying Zhou & Huichuan Feng

Authors
  1. Shijie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huichuan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiyong Chen.

Ethics declarations

Conflict of interest

The authors declare that none of the authors have any financial or scientific conflicts of interest with regard to the research described in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Chen, H., Liu, K. et al. Meta-FSDet: a meta-learning based detector for few-shot defects of photovoltaic modules. J Intell Manuf 34, 3413–3427 (2023). https://doi.org/10.1007/s10845-022-02001-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-02001-3

Keywords

Search

Navigation