Skip to main content
SpringerLink
Log in

Industrial few-shot fractal object detection

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In practical industrial visual inspection tasks, foreign object data are difficult to collect and accumulate, hence few-shot object detection has gradually become the focus of research. It has been observed that industrial foreign objects are often different from natural data and are always fractal objects. Its form is a rough or fragmented geometric shape, and its features are relatively monotonous and difficult to distinguish. Optimization-based meta-learning is a powerful approach to few-shot learning. It updates model weights through a parameter optimization strategy enabling more efficient learning when faced with new tasks with few samples. Therefore, we proposed a gradient scout strategy, which used the intelligent optimization idea to optimize the meta-training outer-loop parallel gradient optimization method to improve the training effect of few-shot fractal object detection. Meanwhile, we proposed a fractal information amplified learning module, which could improve the detection ability of few-shot fractal objects more quickly under the same training period. They formed FLGS (fractal information amplified learning with gradient scout), which was deployed at zero cost. YOLOv7 was advanced to a new industrial fractal object detection model under FLGS. The experimental results on the IGBT surface foreign object dataset showed that our gradient scout strategy was superior to the other eight few-shot meta-learning algorithms. FLGS significantly accelerated the improvement of fractal object detection ability and maintained a high-level mean average precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The datasets generated during the current study are available from the corresponding author on reasonable request.

References

  1. Bochkovskiy A, Wang C, Liao H (2020) Yolov4: optimal speed and accuracy of object detection. Preprint at arXiv:2004.10934

  2. Chen H, Wang Y, Wang G et al (2018) LSTD: a low-shot transfer detector for object detection. In: AAAI conference on artificial intelligence. AAAI, New Orleans, LA, pp 2836–2843

  3. Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. In: IEEE conference on computer vision and pattern recognition, vol 1. IEEE, San Diego, pp 539–546

  4. Fan Q, Zhuo W, Tang C et al (2020) Few-shot object detection with attention-RPN and multi-relation detector. In: IEEE conference on computer vision and pattern recognition. IEEE, ELECTR NETWORK, pp 4012–4021s

  5. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. Preprint at arXiv:1703.03400

  6. Fu G, Sun P, Zhu W et al (2019) A deep-learning-based approach for fast and robust steel surface defects classification. Opt Lasers Eng 121:397–405

    Article  Google Scholar 

  7. Girshick R, Donahue J, Darrell T et al (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE conference on computer vision and pattern recognition. IEEE, Columbus, OH, pp 580–587

  8. Gu K, Zhang Y, Qiao J (2021) Ensemble meta-learning for few-shot soot density recognition. IEEE Trans Ind Inf 13(3):2261–2270

    Article  Google Scholar 

  9. He K, Gkioxari G, Dollár P et al (2017) Mask R-CNN. IEEE Trans Pattern Anal Mach Intell 42:2961–2969

    Google Scholar 

  10. Hoffer E, Ailon N (2014) Deep metric learning using triplet network. Preprint at arXiv:1412.6622

  11. Hospedales T, Antoniou A, Micaelli P et al (2022) Meta-learning in neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(9):5149–5169

    Google Scholar 

  12. Hu H, Gu J, Zhang Z et al (2018) Relation networks for object detection. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 3588–3597

  13. Jocher G (2020) Yolov5. Code at https://github.com/ultralytics/yolov5

  14. Kang B, Liu Z, Wang X et al (2019) Few-shot object detection via feature reweighting. In: IEEE international conference on computer vision. IEEE, Seoul, South Korea, pp 8419–8428

  15. Khosla P, Teterwak P, Wang C, et al (2020) Supervised contrastive learning. Preprint at arXiv:2004.11362

  16. Klejch O, Fainberg J, Bell P et al (2019) Speaker adaptive training using model agnostic meta-learning. In: IEEE automatic speech recognition and understanding workshop. IEEE, Singapore, pp 881–888

  17. Li C, Jiang H, Weng K, et al (2022) Yolov6: a single-stage object detection framework for industrial applications. Preprint at arXiv:2209.02976

  18. Liu H, Socher R, Xiong C (2019) Taming MAML: efficient unbiased meta-reinforcement learning. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of machine learning research, vol 97. PMLR, Long Beach, CA

  19. Liu W, Anguelov D, Erhan D et al (2016) SSD: Single shot multibox detector. In: Leibe B, Matas J, Sebe N et al (eds) Lecture notes in computer science, vol 9905. Springer, Amsterdam, pp 21–37

    Google Scholar 

  20. Munkhdalai T, Yu H (2017) Meta networks. Preprint at arXiv:1703.00837

  21. Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. Preprint at arXiv:1803.02999

  22. Redmon J, Farhadi A (2017) Yolo9000: Better, faster, stronger. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 6517–6525

  23. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. Preprint at arXiv:1804.02767

  24. Redmon J, Divvala S, Girshick R et al (2016) You only look once: unified, real-time object detection. In: IEEE conference on computer vision and pattern recognition. IEEE, Seattle, WA, pp 779–788

  25. Ren S, He K, Girshick R et al (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39:1137–1149

    Article  Google Scholar 

  26. Ruwurm M, Wang S, Krner M et al (2020) Meta-learning for few-shot land cover classification. In: IEEE computer society conference on computer vision and pattern recognition workshops. IEEE, Seattle, WA, pp 788–796

  27. Santoro A, Bartunov S, Botvinick M, et al (2016) One-shot learning with memory-augmented neural networks. Preprint at arXiv:1605.06065

  28. Schweiker K (1993) Fractal detection algorithm for a ladar sensor. In: Proceedings of SPIE—The International Society for Optical Engineering 1960

  29. Shell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. In: Guyon I, Luxburg U, Bengio S et al (eds) Advances in neural information processing systems, vol 30. Curran Associates Inc, Long Beach

    Google Scholar 

  30. Shi B, Liang J, Di L et al (2019) Fabric defect detection via low-rank decomposition with gradient information. IEEE Access 7:130,423-130,437

    Article  Google Scholar 

  31. Sun B, Li B, Cai S et al (2021) FSCE: few-shot object detection via contrastive proposal encoding. In: IEEE conference on computer vision and pattern recognition. IEEE, ELECTR NETWORK, pp 7348–7358

  32. Vanschoren J (2018) Meta-learning: a survey. Preprint at arXiv:1810.03548

  33. Vinyals O, Blundell C, Lillicrap T, et al (2016) Matching networks for one shot learning. Preprint at arXiv:1606.04080

  34. Wang C, Bochkovskiy A, Liao H (2022) Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Preprint at arXiv:2207.02696

  35. Wang X, Huang T, Darrell T, et al (2020) Frustratingly simple few-shot object detection. Preprint at arXiv:2003.06957v1

  36. Wang Y, Ramanan D, Hebert M (2019) Meta-learning to detect rare objects. In: IEEE international conference on computer vision. IEEE, Seoul, South Korea, pp 9924–9933

  37. Wu J, Liu S, Huang D, et al (2020) Multi-scale positive sample refinement for few-shot object detection. Preprint at arXiv:2007.09384

  38. Xiao Y, Marlet R (2020) Few-shot object detection and viewpoint estimation for objects in the wild. Preprint at arXiv:2007.12107v1

  39. Yan X, Chen Z, Xu A et al (2019) Meta R-CNN: towards general solver for instance-level low-shot learning. In: IEEE international conference on computer vision. IEEE, Seoul, South Korea, pp 9576–9585

  40. Zhao Y, Yao YSD, et al (2020) Survey on deep learning object detection. J Image Graph 25:5–30

Download references

Funding

This work was supported by National Key R &D Program of China (2019YFB1705002), National Natural Science Foundation of China (51634002), LiaoNing Revitalization Talents Program (XLYC2002041).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Northeastern University, NO. 3-11, Wenhua Road, Heping District, Shenyang, 110819, Liaoning, China

    Haoran Huang, Xiaochuan Luo & Chen Yang

Authors
  1. Haoran Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaochuan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HH, XL contributed to Conceptualization, Formal analysis, investigation, Writing—review and editing; HH, XL, CY contributed to Methodology; HH contributed to Writing—original draft preparation; XL contributed to Funding acquisition, Resources and Supervision.

Corresponding author

Correspondence to Xiaochuan Luo.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A The loss curve of GSS and FLGS

Appendix A The loss curve of GSS and FLGS

In meta-learning, meta-training is part of the initial parameter gradient optimization, and meta-testing is part of regular training. Therefore, the epoch in Figs. 12, 13 and 14 contains data from these two stages of data at 0 to 10. It should be noted that there is a period in the curve that looks like loss (epoch from 0 to 10) is rising. It is not an error but a jump. This jump was caused by converting meta-training to meta-testing, which is normal. Observing Figs. 12, 13 and 14, it can be seen that during the meta-training and meta-testing stages, the change trends of the train and val loss curves are consistent, all showing a downward trend. This shows that the training data is effective. The network model training process is normal. There are no problems such as overfitting, underfitting, gradient explosion, and gradient disappearance. The network model is constantly adjusting weights and optimization.

Fig. 12
figure 12

The loss curve of GSS on the IGBT-232 dataset for train and val

Full size image
Fig. 13
figure 13

The loss curve of FLGS on the IGBT-232 dataset for train and val

Full size image
Fig. 14
figure 14

The loss curve of GSS and FLGS on the IGBT-101 dataset for train and val when the Line is 3

Full size image

Observing Figs. 12 and 13, it can be found that as the Line increases, the downward trend of the val loss curve becomes more obvious and more convergent. At the same time, GSS plays a full role when Line is greater than 1, and FLGS plays a full role when Line is greater than 2, which indirectly indicates that the multi-line gradient detection strategy is effective. The loss change in the val part of Fig. 14 has some oscillations. This is due to the small amount of data, which is an acceptable normal phenomenon. Their overall downward trend is not problematic.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, H., Luo, X. & Yang, C. Industrial few-shot fractal object detection. Neural Comput & Applic 35, 21055–21069 (2023). https://doi.org/10.1007/s00521-023-08889-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08889-z

Keywords

Search

Navigation