Abstract
Efficient management and control of wafer defects are paramount in enhancing yield in IC chip manufacturing. Scanning Electron Microscope imagery of wafer surfaces, however, presents a challenge due to complex backgrounds and a minimal presence of actual defects. This complexity often hampers traditional convolutional neural networks tasked with defect classification and segmentation, making them prone to disturbances from background elements. To address this issue, we introduce a novel interwoven network architecture that synergizes convolution and Transformer models. This integrated approach is specifically designed to surmount the dual challenges of classification and joint segmentation in wafer defects, achieving a balance between computational efficiency and prediction accuracy. Our research, grounded in real-world production line data from IC chip manufacturing, demonstrates that our network attains a segmentation accuracy of 83.15% and a classification accuracy of 96.88%. The proposed method for automatic defect information extraction is shown to be viable for industrial application. The merger of convolutional neural networks with Transformer models in this innovative architecture shows considerable promise for enhancing wafer defect analysis, thereby improving the precision of defect classification and segmentation in semiconductor manufacturing processes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets analyzed during the current study are available from the corresponding author on reasonable request.
References
Alawieh, M. B., Boning, D., & Pan, D. Z. (2020). Wafer map defect patterns classification using deep selective learning. In 2020 57th ACM/IEEE Design Automation Conference (DAC) (pp. 1-6). IEEE. https://doi.org/10.1109/DAC18072.2020.9218580
Cheon, S., Lee, H., Kim, C. O., & Lee, S. H. (2019). Convolutional neural network for wafer surface defect classification and the detection of unknown defect class. IEEE Transactions on Semiconductor Manufacturing, 32(2), 163–170. https://doi.org/10.1109/TSM.2019.2902657
Chin, R. T., & Harlow, C. A. (1982). Automated visual inspection: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.1982.4767309
e Oliveira, E., Miguéis, V. L., & Borges, J. L. (2023). Automatic root cause analysis in manufacturing: An overview and conceptualization. Journal of Intelligent Manufacturing, 34, 2061–2078.
Kanarik, K. J., Osowiecki, W. T., Lu, Y., Talukder, D., Roschewsky, N., Park, S. N., Mattan, K., David, M. F., & Gottscho, R. A. (2023). Human–machine collaboration for improving semiconductor process development. Nature, 616(7958), 707–711.
Khakifirooz, M., Fathi, M., & Wu, K. (2019). Development of smart semiconductor manufacturing: Operations research and data science perspectives. IEEE Access, 7, 108419–108430. https://doi.org/10.1109/ACCESS.2019.2933167
Kim, M., Lee, M., An, M., & Lee, H. (2020). Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. Journal of Intelligent Manufacturing, 31, 1165–1174. https://doi.org/10.1007/s10845-019-01502-y
Lin, B. S., Cheng, J. S., Liao, H. C., Yang, L. W., Yang, T., & Chen, K. C. (2021). Improvement of multi-lines bridge defect classification by hierarchical architecture in artificial intelligence automatic defect classification. IEEE Transactions on Semiconductor Manufacturing, 34(3), 346–351. https://doi.org/10.1109/TSM.2021.3076808
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 10012–10022).
Nakazawa, T., & Kulkarni, D. V. (2018). Wafer map defect pattern classification and image retrieval using convolutional neural network. IEEE Transactions on Semiconductor Manufacturing, 31(2), 309–314. https://doi.org/10.1109/TSM.2018.2795466
Nti, I. K., Adekoya, A. F., Weyori, B. A., et al. (2022). Applications of artificial intelligence in engineering and manufacturing: A systematic review. Journal of Intelligent Manufacturing, 33, 1581–1601. https://doi.org/10.1007/s10845-021-01771-6
Phua, C., & Theng, L. B. (2020). Semiconductor wafer surface: Automatic defect classification with deep CNN. In 2020 IEEE region 10 conference (TENCON) (pp. 714–719). IEEE.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241). Springer International Publishing. https://doi.org/10.1007/978-3-319-24574-4_28
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. Preprint retrieved from https://arxiv.org/abs/1312.6034
Song, S., & Back, J. G. (2023). Representation Learning for Wafer Pattern Recognition in Semiconductor Manufacturing Process. In 2023 international conference on artificial intelligence in information and communication (ICAIIC) (pp. 264–269). IEEE. https://doi.org/10.1109/ICAIIC57133.2023.10067020.
Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., & Dosovitskiy, A. (2021). MLP-mixer: An all-MLP architecture for vision. Advances in Neural Information Processing Systems, 34, 24261–24272.
Van Molle, P., De Strooper, M., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2018). Visualizing convolutional neural networks to improve decision support for skin lesion classification. In D. Stoyanov (Ed.), Understanding and interpreting machine learning in medical image computing applications: MLCN 2018, DLF 2018, IMIMIC 2018. (Vol. 11038). Springer.
Wei, Y., & Wang, H. (2022). Mixed-type wafer defect pattern recognition framework based on multifaceted dynamic convolution. IEEE Transactions on Instrumentation and Measurement, 71, 1–11. https://doi.org/10.1109/TIM.2022.3178498
Wu, Z., Cai, N., Chen, K., Xia, H., Zhou, S., & Wang, H. (2023). GAN-based statistical modeling with adaptive schemes for surface defect inspection of IC metal packages. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-023-02146-9
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. Preprint retrieved from https://arxiv.org/abs/2105.15203
Yu, J., & Liu, J. (2022). Multiple granularities generative adversarial network for recognition of wafer map defects. IEEE Transactions on Industrial Informatics, 18(3), 1674–1683. https://doi.org/10.1109/TII.2021.3092372
Yu, N., Chen, H., Xu, Q., & Sie, O. (2022). Wafer map defect patterns classification based on a lightweight network and data augmentation. CAAI Transactions on Intelligence Technology. https://doi.org/10.1049/cit2.12126
Yuan, T., Kuo, W., & Bae, S. J. (2011). Detection of spatial defect patterns generated in semiconductor fabrication processes. IEEE Transactions on Semiconductor Manufacturing, 24(3), 392–403. https://doi.org/10.1109/TSM.2011.2154870
Zeiler, M. D., & Fergus, R. (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision (pp. 818–833). Springer
Zhang, Y., Liu, H., & Hu, Q. (2021). Transfuse: Fusing transformers and CNNs for medical image segmentation. Preprint retrieved from https://arxiv.org/abs/2102.08005
Zhi, Z., Jiang, H., Yang, D., Gao, J., Wang, Q., Wang, X., Wang, J., & Wu, Y. (2023). An end-to-end welding defect detection approach based on titanium alloy time-of-flight diffraction images. Journal of Intelligent Manufacturing, 34(4), 1895–1909. https://doi.org/10.1007/s10845-021-01905-w
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. ZM and YL completed the data analysis and code writing. ZM and YQ designed and drafted the manuscript, and YC revised the paper. All authors read and approved the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Mei, Z., Luo, Y., Qiao, Y. et al. A novel joint segmentation approach for wafer surface defect classification based on blended network structure. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02324-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10845-024-02324-3