Skip to main content

A deep neural network and classical features based scheme for objects recognition: an application for machine inspection

Abstract

Computer Vision (CV) domain is widely used in the current era of automation and visual surveillance for the detection and classification of different objects in a diverse environment. The automatic machine inspection of different objects in the scenes is based on internal and external parameters like features that provide a huge amount of information related to the nature of an object in the scene. In this work, we propose a new automated method based on classical and deep learning feature selection. The proposed object classification method follows three steps. The data augmentation is performed in the first step to make the balance database. Later, Pyramid HOG (PHOG) and Central Symmetric LBP (CS-LBP) features are serially fused along with deep learning-based extracted features. The deep learning features are extracted from the pre-trained CNN model name Inception V3. In the third step, a new technique name Joint Entropy along with KNN (JEKNN) is employed to select the best features. The best-selected features are finally classified by well-known supervised learning methods and choose the best one based on higher accuracy. The proposed method is evaluated on Caltech101 balanced dataset and achieved maximum accuracy of 90.4% on Ensemble classifier which outperforms as compare to existing techniques.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. Adeel A, Khan MA, Sharif M, Azam F, Umer T, and Wan S (2019) Diagnosis and recognition of grape leaf diseases: an automated system based on a novel saliency approach and canonical correlation analysis based multiple features fusion, Sustainable Computing: Informatics and Systems

  2. Arshad H, Khan MA, Sharif M, Yasmin M, Javed MYJIJOML, and Cybernetics (2019) Multi-level features fusion and selection for human gait recognition: an optimized framework of Bayesian model and binomial distribution," pp. 1–18

  3. Arshad H, Khan MA, Sharif MI, Yasmin M, Tavares JMRS, Zhang Y-D and Satapathy SC (2020) "A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition." Expert Systems: e12541

  4. Bilal M and Hanif MSJJoSPS (2019) High performance real-time pedestrian detection using light weight features and fast cascaded kernel SVM classification, vol. 91, pp. 117–129

  5. Cao X, Wu C, Yan P, and Li X (2011) Linear SVM classification using boosting HOG features for vehicle detection in low-altitude airborne videos, in 2011 18th IEEE International Conference on Image Processing, pp. 2421–2424.

  6. Cao H, Du H, Zhang S, and Cai S (2020) InSphereNet: a concise representation and classification method for 3D object, in International Conference on Multimedia Modeling, pp. 327–339.

  7. Chaudhuri DR, Chandra D, and Mittal A (2020) Indoor object classification using higher dimensional MPEG features," in Soft Computing for Problem Solving, ed: Springer, pp. 573–583.

  8. Cubuk ED, Zoph B, Shlens J, and Le QV (2019) RandAugment: Practical data augmentation with no separate search," arXiv preprint arXiv:1909.13719

  9. Cui Y, Xu H, Wu J, Sun Y, and Zhao JJIIS (2019) Automatic vehicle tracking with roadside LiDAR data for the connected-vehicles system

  10. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection

    Book  Google Scholar 

  11. Deng J, Dong W, Socher R, Li L-J, Li K, and Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database, in 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255.

  12. Fei-Fei L, Perona P (2005) A bayesian hierarchical model for learning natural scene categories, in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), pp. 524–531.

  13. Ghodrati H, Luciano L, Hamza ABJNPL (2019) Convolutional shape-aware representation for 3D object classification, vol. 49, pp. 797–817

  14. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

    Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.

  16. Kaur B, Bhattacharya JJESwA (2019) A convolutional feature map-based deep network targeted towards traffic detection and classification, vol. 124, pp. 119–129

  17. Khan MA, Akram T, Sharif M, Awais M, Javed K, Ali H et al (2018) CCDF: automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Comput Electron Agric 155:220–236

    Article  Google Scholar 

  18. Khan MA, Khan MA, Ahmed F, Mittal M, Goyal LM, Hemanth DJ et al (2020) Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recognition Letters 131:193–204

  19. Khan MA, Javed K, Khan SA, Saba T, Habib U, Khan JA et al (2020) "Human action recognition using fusion of multiview and deep features: an application to video surveillance." Multimedia Tools and Applications 1–27

  20. Khan MA, Rubab S, Kashif A, Sharif MI, Muhammad N, Shah JH et al (2020) Lungs cancer classification from CT images: An integrated design of contrast based classical features fusion and selection. Pattern Recognition Letters 129:77–85

  21. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks, in Advances in neural information processing systems, pp. 1097–1105.

  22. Kumar B, Pandey G, Lohani B, Misra SCJIjop, and R. Sensing (2019) A multi-faceted CNN architecture for automatic classification of mobile LiDAR data and an algorithm to reproduce point cloud samples for enhanced training, vol. 147, pp. 80–89

  23. Liu X, Zhang R, Meng Z, Hong R, and Liu GJWWW (2019) On fusing the latent deep CNN feature for image classification, vol. 22, pp. 423–436

  24. Mafarja M, Qasem A, Heidari AA, Aljarah I, Faris H, and Mirjalili SJCC (2019) Efficient hybrid nature-inspired binary optimizers for feature selection, pp. 1–26

  25. Majid A, Khan MA, Yasmin M, Rehman A, Yousafzai A and Tariq U (2020) Classification of stomach infections: A paradigm of convolutional neural network along with classical features fusion and selection. Microscopy Research and Technique

  26. Mirjalili S (2019) "Genetic algorithm," in Evolutionary Algorithms and Neural Networks, ed: Springer, pp. 43–55.

  27. Na B, Fox G (2019) Object classification by a super-resolution method and a convolutional neural networks. International Journal of Data Mining Science 1:16–23

    Google Scholar 

  28. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. Journal of Big Data 2:1

    Article  Google Scholar 

  29. Neumann J, Schnörr C, Steidl G (2005) Combined SVM-based feature selection and classification. Mach Learn 61:129–150

    Article  Google Scholar 

  30. Pan Y, Xia Y, Song Y, Cai WJMT, and Applications (2018) Locality constrained encoding of frequency and spatial information for image classification, vol. 77, pp. 24891–24907

  31. Quattoni A, Torralba A (2009) Recognizing indoor scenes," in 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–420.

  32. R. Rani, A. P. Singh, R. J. M. T. Kumar, and Applications (2019) Impact of reduction in descriptor size on object detection and classification," vol. 78, pp. 8965–8979

  33. Rashid M, Khan MA, Sharif M, Raza M, Sarfraz MM, Afza FJMT, et al. (2019) Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features, vol. 78, pp. 15751–15777

  34. Ravikumar S, Ramachandran K, Sugumaran V (2011) Machine learning approach for automated visual inspection of machine components. Expert Syst Appl 38:3260–3266

    Article  Google Scholar 

  35. Rehman A, Khan MA, Mehmood Z, Saba T, Sardaraz M and Rashid M (2020) Microscopic melanoma detection and classification: A framework of pixel‐based fusion and multilevel features reduction. Microscopy Research and Technique 83 (4):410–423

  36. Rish I (2001) An empirical study of the naive Bayes classifier, in IJCAI 2001 workshop on empirical methods in artificial intelligence, pp. 41–46.

  37. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. (2015) Imagenet large scale visual recognition challenge, vol. 115, pp. 211–252

  38. Saba T, Khan MA, Rehman A, Marie-Sainte SL (2019) Region Extraction and Classification of Skin Cancer: A Heterogeneous framework of Deep CNN Features Fusion and Reduction. Journal of Medical Systems 43(9)

  39. Sayed GI, Hassanien AE, Azar ATJNC, and Applications (2019) Feature selection via a novel chaotic crow search algorithm, vol. 31, pp. 171–188

  40. Shaheen M, Zafar T, and Ali Khan S (2019) Decision tree classification: Ranking journals using IGIDI," Journal of Information Science, p. 0165551519837176

  41. Sharif M, Khan MA, Tahir MZ, Yasmim M, Saba T and Tanik UJ (2020) "A Machine Learning Method with Threshold Based Parallel Feature Fusion and Feature Selection for Automated Gait Recognition." Journal of Organizational and End User Computing (JOEUC) 32(2):67–92

  42. Song J , Yoon G, Cho H, Yoon SMJMT, and Applications (2018) Structure preserving dimensionality reduction for visual object recognition, vol. 77, pp. 23529–23545

  43. Soucy P, Mineau GW (2001) A simple KNN algorithm for text categorization," in Proceedings 2001 IEEE International Conference on Data Mining, pp. 647–648.

  44. Srivastava S, Priyadarshini J, Gopal S, Gupta S, and Dayal HS (2019) Optical character recognition on bank cheques using 2D convolution neural network, in Applications of Artificial Intelligence Techniques in Engineering, ed: Springer, pp. 589–596.

  45. Sun H, Wang C, Wang B, El-Sheimy N (2011) Pyramid binary pattern features for real-time pedestrian detection from infrared videos. Neurocomputing 74:797–804

    Article  Google Scholar 

  46. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision, in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826.

  47. Tallón-Ballesteros AJ, Cavique L, and Fong S (2019) Addressing low dimensionality feature subset selection: reliefF (−k) or extended correlation-based feature selection (eCFS)?, in International Workshop on Soft Computing Models in Industrial and Environmental Applications, pp. 251–260.

  48. Tan B, Salakhutdinov R, Mitchell T, and Xing E (2019) Learning data manipulation for augmentation and weighting

    Google Scholar 

  49. Tilahun SL, Ngnotchouye JMT, Hamadneh NNJAIR (2019) Continuous versions of firefly algorithm: A review, vol. 51, pp. 445–492

  50. Wang Y, Chen Y, Yang N, Zheng L, Dey N, Ashour AS et al (2019) Classification of mice hepatic granuloma microscopic images based on a deep convolutional neural network. Appl Soft Comput 74:40–50

    Article  Google Scholar 

  51. Wang X, Zhang W, Wu X, Xiao L, Qian Y, and Fang ZJJoR-TIP (2019) Real-time vehicle type classification with deep convolutional neural networks, vol. 16, pp. 5–14

  52. Wei G, Cao H, Ma H, Qi S, Qian W, Ma ZJJoms (2018) Content-based image retrieval for lung nodule classification using texture features and learned distance metric, vol. 42, p. 13

  53. Weibel J-B, Patten T, Vincze M (2019) Robust 3D object classification by combining point pair features and graph convolution, in 2019 International Conference on Robotics and Automation (ICRA), pp. 7262–7268.

  54. Wu K, Zhang D, Lu G, Guo ZJPR (2019) Joint learning for voice based disease detection vol. 87, pp. 130–139

  55. Wu J, Shang Z, Wang K, Zhai J, Wang Y, Xia F, et al. (2019) Partially occluded head posture estimation for 2D images using pyramid HoG features, in 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 507–512

  56. Xiong F, Xiao Y, Cao Z, Gong K, Fang Z, and Zhou JT (2019) Good practices on building effective CNN baseline model for person re-identification, in Tenth International Conference on Graphics and Image Processing (ICGIP 2018), p. 110690I.

  57. Zhi S, Liu Y, Li X, Guo YJC, and Graphics (2018) Toward real-time 3D object recognition: A lightweight volumetric CNN framework using multitask learning," vol. 71, pp. 199–207

  58. Zhu Y, Shi J, Wu X, Liu X, Zeng G, Sun J et al (2020) Photon-limited non-imaging object detection and classification based on single-pixel imaging system. Applied Physics B 126:21

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sajid Ali Khan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 8 Summary of existing techniques

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hussain, N., Khan, M.A., Sharif, M. et al. A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-08852-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-020-08852-3

Keywords

  • Object classification
  • Augmentation
  • Classical features
  • CNN features
  • Feature selection