Skip to main content
SpringerLink
Log in

A maximum-entropy-attention-based convolutional neural network for image perception

  • S.I.: AI based Techniques and Applications for Intelligent IoT Systems
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, image perception such as enhancement, classification and object detection with deep learning has achieved significant successes. However, in real world under extreme conditions, the training of a deep learning model often yields low accuracy, low efficiency in feature extraction and generalizability, due to the inner uncourteous and uninterpretable characteristics. In this paper, a maximal-entropy-attention-based convolutional neural network (MEA-CNN) framework is proposed. A maximum entropy algorithm is first used for image feature pre-extraction. An attention mechanism is then proposed by combining the extracted features on original images. By applying the mechanism, the key areas of an image are enhanced, and noised area can be ignored. Afterward, the processed images are transferred into region convolutional neural network, which is a well-known pre-trained CNN model, for further feature learning and extraction. Finally, two real-world experiments on traffic sign recognition and road surface condition monitoring are designed. The results show that the proposed framework has high testing accuracy, with improvements of 17% and 2.9%, compared with some other existing methods. In addition, the features extracted by the model are more easily interpretable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Data available on request from the authors.

References

  1. Gu K, Zhang Y, Qiao J (2021) Ensemble meta-learning for few-shot soot density recognition. IEEE Trans Industr Inf 17(3):2261–2270

    Article  Google Scholar 

  2. Zhu M, Ge D (2020) Image quality assessment based on deep learning with FPGA implementation. Signal Process: Image Commun 1(83):115780

    Google Scholar 

  3. Han G, Cheng Q, Sun X, Li L, Di W (2019) A biological mechanism based structure self-adaptive algorithm for feedforward neural network and its engineering applications. IEEE Access 7:25111–25122

    Article  Google Scholar 

  4. Han H, Liu H, Li J, Qiao J (2021) Cooperative fuzzy-neural control for wastewater treatment process. IEEE Trans Industr Inf 17(9):5971–5981

    Article  Google Scholar 

  5. Han G, Li L, Di W, Sun X, Bu T, Lin T (2020) Multiscale convolutional generative adversarial network for anchorage grout defect detection. IEEE Trans Instrum Meas 70:1–10

    Google Scholar 

  6. Yang L, Wang L, Su Y, Gao Y (2021) Bag of shape descriptor using unsupervised deep learning for non-rigid shape recognition. Signal Process: Image Commun 1(96):116297

    Google Scholar 

  7. Yin P, Yuan R, Cheng Y, Wu Q (2020) Deep guidance network for biomedical image segmentation. IEEE Access 8:116106–116116

    Article  Google Scholar 

  8. Wong A, Famuori M, Shafiee MJ, Li F, Chwyl B, Chung J (2019) YOLO Nano: a highly compact you only look once convolutional neural network for object detection. Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition, 22–25.

  9. Oktay O, Ferrante E, Kamnitsas K (2018) Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans Med Imaging 37(2):384–395

    Article  Google Scholar 

  10. Pan G, Fu L, Thakali L (2017) Development of a global road safety performance function using deep neural networks. Int J Transp Sci Technol 6(3):159–173

    Article  Google Scholar 

  11. Samek W, Binder A, Montavon G, Lapuschkin S, Müller KR (2017) Evaluating the visualization of what a deep neural network has learned. IEEE Trans Neural Netw Learn Syst 28(11):2660–2673

    Article  MathSciNet  Google Scholar 

  12. Pan G, Fu L, Chen Q, Yu M, Muresan M (2020) Road safety performance function analysis with visual feature importance of deep neural nets. IEEE/CAA J Automatica Sinica 7(3):735–744

    Article  Google Scholar 

  13. Chen Q, Pan G, Chen W, Wu P (2021) A novel explainable deep belief network framework and its application for feature importance analysis. IEEE Sens J 21:25001–25009

    Article  Google Scholar 

  14. Gu K, Tao D, Qiao J, Lin W (2018) Learning a no-reference quality assessment model of enhanced images with big data. IEEE Trans Neural Netw Learn Syst 29(4):1301–1313

    Article  Google Scholar 

  15. Gu K, Zhang Y, Qiao J (2020) Vision-based monitoring of flare soot. IEEE Trans Instrum Meas 69(9):7136–7145

    Article  Google Scholar 

  16. Liu H, Chu W, Wang H (2020) Automatic segmentation algorithm of ultrasound heart image based on convolutional neural network and image saliency. IEEE Access 8:104445–104457

    Article  Google Scholar 

  17. Chen W, Gu K, Zhao T, Jiang G, Callet PL (2021) Semi-reference sonar image quality assessment based on task and visual perception. IEEE Trans Multimedia 23:1008–1020

    Article  Google Scholar 

  18. Zhu X, Zhang X, Zhang T, Zhu P, Tang X, Li C (2020) Discriminative feature pyramid network for object detection in remote sensing images. International Joint Conference on Neural Networks (IJCNN), 1–7.

  19. Shi X, Qiu G, Yin C, Huang X, Chen K, Cheng Y, Zhong S (2021) An improved bearing fault diagnosis scheme based on hierarchical fuzzy entropy and Alexnet network. IEEE Access 9:61710–61720

    Article  Google Scholar 

  20. Avula SB, Badri SJ, Reddy G (2020) A novel forest fire detection system using fuzzy entropy optimized thresholding and STN-based CNN. IEEE International Conference on Communication Systems & Networks, 750–755.

  21. Tian Y, Pan G (2020) An unsupervised regularization and dropout based deep neural network and its application for thermal error prediction. Appl Sci 10(8):2870

    Article  Google Scholar 

  22. Chen Q, Pan G (2021) A structure-self-organizing DBN for image recognition. Neural Comput Appl 33(7553):877–886

    Article  Google Scholar 

  23. Pan G, Fu L, Yu R, Muresan M, Evaluation of alternative pre-trained convolutional neural networks for winter road surface condition monitoring. IEEE International Conference on Transportation Information and Safety, (2019), 614–620.

  24. Gaus YFA, Bhowmik N, Akçay S, Guillén-Garcia PM, Barker JW, Breckon TP (2019) Evaluation of a dual convolutional neural network architecture for object-wise anomaly detection in cluttered X-ray security imagery. International Joint Conference on Neural Networks (IJCNN), 1–8.

  25. Nie D, Wang L, Adeli E, Lao C, Lin W, Shen D (2018) 3-D fully convolutional networks for multimodal isointense infant brain image segmentation. IEEE Trans Cybernetics 49(3):1123–1136

    Article  Google Scholar 

  26. Zhang M, Jiao L, Shang R, Zhang X, Li L (2020) Unsupervised EA-based fuzzy clustering for image segmentation. IEEE Access 8:8627–8647. https://doi.org/10.1109/ACCESS.2019.2963363

    Article  Google Scholar 

  27. Lei T, Liu P, Jia X, Zhang X, Meng H, Nandi AK (2020) Automatic fuzzy clustering framework for image segmentation. IEEE Trans Fuzzy Syst 28(9):2078–2092

    Article  Google Scholar 

  28. Bazaluk O, Kotenko S, Nitsenko V (2021) Entropy as an objective function of optimization multimodal transportations. Entropy 23:946

    Article  MathSciNet  Google Scholar 

  29. Li L, He H, Li J (2020) Entropy-based sampling approaches for multi-class imbalanced problems. IEEE Trans Knowl Data Eng 32(11):2159–2170

    Article  Google Scholar 

  30. Hussain L, Aziz W, Alshdadi AA, Ahmed Nadeem MS, Khan IR, Chaudhry Q (2019) Analyzing the dynamics of lung cancer imaging data using refined fuzzy entropy methods by extracting different features. IEEE Access 7:64704–64721

    Article  Google Scholar 

  31. Chakraborty DB, Pal SK (2018) Neighborhood rough filter and intuitionistic entropy in unsupervised tracking. IEEE Trans Fuzzy Syst 26(4):2188–2200

    Article  Google Scholar 

  32. Alwerfali HN, Al-qaness MAA, Elaziz MAbd, Ewees A, Oliva D, Songfeng L (2020) Multi-level image thresholding based on modified spherical search optimizer and fuzzy entropy. Entropy 22(3):328. https://doi.org/10.3390/e22030328

    Article  MathSciNet  Google Scholar 

  33. Gu K, Zhang Y, Qiao J (2020) Random forest ensemble for river turbidity measurement from space remote sensing data. IEEE Trans Instrum Meas 69(11):9028–9036

    Article  Google Scholar 

  34. Ye M, Yan X, Jia M (2021) Rolling bearing fault diagnosis based on VMD-MPE and PSO-SVM. Entropy 23:762

    Article  Google Scholar 

  35. Jalal A, Ahmed A, Rafique AA, Kim K (2021) Scene semantic recognition based on modified fuzzy c-mean and maximum entropy using object-to-object relations. IEEE Access 9:27758–27772

    Article  Google Scholar 

  36. Bera A, Wharton Z, Liu Y, Bessis N, Behera A (2021) Attend and guide (AG-Net): A keypoints-driven attention-based deep network for image recognition. IEEE Trans Image Process 30:3691–3704. https://doi.org/10.1109/TIP.2021.3064256

    Article  Google Scholar 

  37. Gu R et al (2021) CA-Net: comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans Med Imaging 40(2):699–711. https://doi.org/10.1109/TMI.2020.3035253

    Article  MathSciNet  Google Scholar 

  38. World Health Organization (WHO), Global status report on road safety, WHO Press, World Health Organization, Geneva, Switzerland, 2018.

Download references

Acknowledgements

This research was funded by the National Natural Science Foundation of China under Grant no. 62103056 and 62103177, the Development Plan of Youth Innovation Team of the University in Shandong Province under Grant no. 2019KJN007, and the Natural Science Foundation Program of Shandong Province under Grant no. ZR2019YQ28.

Author information

Authors and Affiliations

  1. School of Automation, Beijing Information Science and Technology University, Beijing, 100192, China

    Qili Chen

  2. School of Automation and Electrical Engineering, Linyi University, Linyi, 276000, China

    Ancai Zhang & Guangyuan Pan

Authors
  1. Qili Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ancai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangyuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangyuan Pan.

Ethics declarations

Conflicts of interest

All the authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Q., Zhang, A. & Pan, G. A maximum-entropy-attention-based convolutional neural network for image perception. Neural Comput & Applic 35, 8647–8655 (2023). https://doi.org/10.1007/s00521-022-07564-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07564-z

Keywords

Search

Navigation