Skip to main content
SpringerLink
Log in

A Statistical Approach to Hyperparameter Tuning of Deep Learning for Construction Machine Classification

  • Research Article-Civil Engineering
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Deep learning methods have shown significant potential in computer vision applications within the field of civil engineering. One important area of research is the detection and classification of construction machines using convolutional neural networks (CNNs). However, a major challenge in adopting machine learning models for vision systems lies in hyperparameter optimization. Therefore, the objective of this paper is to propose a meticulous statistical approach to hyperparameter tuning for deep learning in the visual classification of construction machines. It is noteworthy that the method for selecting hyperparameters utilizes statistical concepts, including analysis of variance and the Tukey test. Moreover, three research questions were formulated to address statistical differences in performance when varying the CNN architecture, optimizer and configuration (architecture and optimizer). Two additional research questions were posed to determine the recommended hyperparameters for the task and assess performance differences for non-selected hyperparameters. For this purpose, 18 hyperparameter combinations were analyzed, encompassing three CNN architectures (DenseNet, Mobilenet and VGG16) and six optimizers (adadelta, adagrad, rmsprop, adam, adamax and sgd). The Alberta Construction Image Dataset, consisting of 2850 images of three construction machines (excavator, dump truck and concrete mixer truck), was used in the experiments. The results demonstrate that the hyperparameter configurations exhibit statistically significant differences in image classification. Furthermore, the recommended combination (DenseNet \(\times \) adagrad) yielded the highest average accuracy results in both binary tests (90.0%) and multiclass classification (77.8%).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The images analyzed during the current study are available in the Alberta Construction Image Dataset, https://www.acidb.ca/dataset.

Notes

  1. https://www.acidb.ca/dataset.

References

  1. Cha, Y.-J.; Choi, W.; Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)

    Article  Google Scholar 

  2. Finotti, R.P.; Barbosa, F.D.S.; Cury, A.A.; Pimentel, R.L.: Novelty detection using sparse auto-encoders to characterize structural vibration responses. Arab. J. Sci. Eng. 47, 13049–13062 (2022)

    Article  Google Scholar 

  3. Qi, Z.L.; Liu, D.; Zhang, J.; Chen, J.: Micro-concrete crack detection of underwater structures based on convolutional neural network. Mach. Vis. Appl. 33(5), 1–19 (2022)

    Article  Google Scholar 

  4. Wang, W.; Su, C.: Automatic classification of reinforced concrete bridge defects using the hybrid network. Arab. J. Sci. Eng. 47(4), 5187–5197 (2022)

    Article  Google Scholar 

  5. Vinod, S.; Shakor, P.; Sartipi, F.; Karakouzian, M.: Object detection using esp32 cameras for quality control of steel components in manufacturing structures. Arab. J. Sci. Eng. 48, 12741–12758 (2022)

    Article  Google Scholar 

  6. Li, Y.; Wei, H.; Han, Z.; Huang, J.; Wang, W.: Deep learning-based safety helmet detection in engineering management based on convolutional neural networks. Adv. Civ. Eng. 2020, 1–10 (2020)

    Article  Google Scholar 

  7. Ottoni, A.L.C.; Novo, M.S.; Costa, D.B.: Deep learning for vision systems in construction 4.0: a systematic review. SIViP 17, 1821–1829 (2022)

    Article  Google Scholar 

  8. Kim, H.; Kim, H.; Hong, Y.W.; Byun, H.: Detecting construction equipment using a region-based fully convolutional network and transfer learning. J. Comput. Civ. Eng. 32(2), 04017082 (2018)

    Article  Google Scholar 

  9. Xiao, B.; Kang, S.-C.: Development of an image data set of construction machines for deep learning object detection. J. Comput. Civ. Eng. 35(2), 05020005 (2021)

    Article  Google Scholar 

  10. Xiao, B.; Kang, S.-C.: Vision-based method integrating deep learning detection for tracking multiple construction machines. J. Comput. Civ. Eng. 35(2), 04020071 (2021)

    Article  Google Scholar 

  11. Xiao, B.; Lin, Q.; Chen, Y.: A vision-based method for automatic tracking of construction machines at nighttime based on deep learning illumination enhancement. Autom. Constr. 127, 103721 (2021)

    Article  Google Scholar 

  12. Hou, L.; Chen, C.; Wang, S.; Wu, Y.; Chen, X.: Multi-object detection method in construction machinery swarm operations based on the improved yolov4 model. Sensors 22(19), 7294 (2022)

    Article  Google Scholar 

  13. Zhang, A.; Wang, K.C.; Li, B.; Yang, E.; Dai, X.; Peng, Y.; Fei, Y.; Liu, Y.; Li, J.Q.; Chen, C.: Automated pixel-level pavement crack detection on 3d asphalt surfaces using a deep-learning network. Comput. Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)

    Article  Google Scholar 

  14. Li, S.; Zhao, X.: Image-based concrete crack detection using convolutional neural network and exhaustive search technique. Adv. Civ. Eng. 2019, 1–12 (2019)

  15. Gopalakrishnan, K.; Khaitan, S.K.; Choudhary, A.; Agrawal, A.: Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr. Build. Mater. 157, 322–330 (2017)

  16. Hoang, N.-D.; Nguyen, Q.-L.: A novel approach for automatic detection of concrete surface voids using image texture analysis and history-based adaptive differential evolution optimized support vector machine. Adv. Civ. Eng. 2020, 1–15 (2020)

    Article  Google Scholar 

  17. Cheng, J.C.P.; Wang, M.: Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom. Constr. 95, 155–171 (2018)

    Article  Google Scholar 

  18. Gulgec, N.S.; Takáč, M.; Pakzad, S.N.: Convolutional neural network approach for robust structural damage detection and localization. J. Comput. Civ. Eng. 33(3), 04019005 (2019)

    Article  Google Scholar 

  19. Kumar, S.S.; Abraham, D.M.; Jahanshahi, M.R.; Iseley, T.; Starr, J.: Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks. Autom. Constr. 91, 273–283 (2018)

    Article  Google Scholar 

  20. Kolar, Z.; Chen, H.; Luo, X.: Transfer learning and deep convolutional neural networks for safety guardrail detection in 2d images. Autom. Constr. 89, 58–70 (2018)

    Article  Google Scholar 

  21. Hutter, F.; Hoos, H.; Leyton-Brown, K.: An efficient approach for assessing hyperparameter importance. In: Proceedings of International Conference on Machine Learning 2014 (ICML 2014), pp. 754–762 (2014)

  22. Hutter, F.; Kotthoff, L.; Vanschoren, J. (eds.): Automated Machine Learning: Methods, Systems, Challenges. Springer, Berlin (2019)

    Google Scholar 

  23. Mantovani, R.G.; Rossi, A.L.D.; Alcobaça, E.; Vanschoren, J.; de Carvalho, A.C.P.L.F.: A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves svm classifiers. Inf. Sci. 501, 193–221 (2019)

    Article  Google Scholar 

  24. Kaur, S.; Aggarwal, H.; Rani, R.: Hyper-parameter optimization of deep learning model for prediction of Parkinson’s disease. Mach. Vis. Appl. 31(5), 1–15 (2020)

    Article  Google Scholar 

  25. Cheng, S.; Shen, H.; Shan, G.; Niu, B.; Bai, W.: Visual analysis of meteorological satellite data via model-agnostic meta-learning. J. Vis. 24(2), 301–315 (2021)

    Article  Google Scholar 

  26. Poulose, A.; Kim, J.H.; Han, D.S.; et al.: HIT HAR: human image threshing machine for human activity recognition using deep learning models. Comput. Intell. Neurosci. 2022, 1–21 (2022)

    Article  Google Scholar 

  27. Ottoni, A.L.C.; Nepomuceno, E.G.; de Oliveira, M.S.; de Oliveira, D.C.R.: Tuning of reinforcement learning parameters applied to sop using the Scott–Knott method. Soft. Comput. 24, 4441–4453 (2020)

    Article  Google Scholar 

  28. Ottoni, A.L.C.; Nepomuceno, E.G.; de Oliveira, M.S.; de Oliveira, D.C.R.: Reinforcement learning for the traveling salesman problem with refueling. Complex Intell. Syst. 8, 2001–2015 (2021)

    Article  Google Scholar 

  29. Lahmar, C.; Idri, A.: On the value of deep learning for diagnosing diabetic retinopathy. Health Technol. 12, 89–105 (2022)

    Article  Google Scholar 

  30. Ottoni, A.L.C.; Novo, M.S.; Costa, D.B.: Hyperparameter tuning of convolutional neural networks for building construction image classification. Vis. Comput. 39, 847–861 (2023)

    Google Scholar 

  31. Ottoni, A.L.C.; Amorim, R.M.; Novo, M.S.; Costa, D.B.: Tuning of data augmentation hyperparameters in deep learning to building construction image classification with small datasets. Int. J. Mach. Learn. Cybern. 14, 171–186 (2023)

    Article  Google Scholar 

  32. Tukey, J.W.: The Problem of Multiple Comparisons. Princeton University, Princeton (1953)

    Google Scholar 

  33. Montgomery, D.C.: Design and Analysis of Experiments, 9th edn Wiley, New York (2017)

    Google Scholar 

  34. Ravichandran, C.; Padmanaban, G.: A numerical simulation-based method to predict floor wise distribution of cooling loads in Indian residences using Tukey honest significant difference test. Adv. Build. Energy Res. 17(1), 1–29 (2023)

  35. Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017). arXiv:1704.04861

  36. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)

  37. Simonyan, K.; Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015—Conference Track Proceedings (2015)

  38. Zeiler, M.D.: Adadelta: An Adaptive Learning Rate Method (2012). arXiv:1212.5701

  39. Duchi, J.; Hazan, E.; Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(7), 2121–2159 (2011)

    MathSciNet  Google Scholar 

  40. Chollet, F.; Allaire, J.J.: Deep Learning With R. Manning Publications, Shelter Island (2018)

    Google Scholar 

  41. Kingma, D.P.; Ba, J.: Adam: A Method for Stochastic Optimization (2014). arXiv:1412.6980

  42. Shorten, C.; Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–14 (2019)

    Article  Google Scholar 

  43. Elgendy, M.: Deep Learning for Vision Systems. Manning Publications, Shelter Island (2020)

    Google Scholar 

  44. Jia, S.; Lin, P.; Li, Z.; Zhang, J.; Liu, S.: Visualizing surrogate decision trees of convolutional neural networks. J. Vis. 23(1), 141–156 (2020)

    Article  Google Scholar 

  45. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2020)

  46. Razali, N.M.; Wah, Y.B.; et al.: Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J. Stat. Model. Anal. 2(1), 21–33 (2011)

    Google Scholar 

  47. Bartlett, M.S.: Properties of sufficiency and statistical tests. Proc. R. Soc. Lond. Ser. A Math. Phys. Sci. 160(901), 268–282 (1937)

    Google Scholar 

  48. Miller, R.G.: Simultaneous Statistical Inference. Springer, Berlin (1981)

    Book  Google Scholar 

  49. Yandell, B.S.: Practical Data Analysis for Designed Experiments. Chapman & Hall, Boca Raton (1997)

    Book  Google Scholar 

  50. Shenoy, M.; Raju, P.V.S.; Prasad, J.: Optimization of physical schemes in WRF model on cyclone simulations over Bay of Bengal using one-way ANOVA and Tukey’s test. Sci. Rep. 11(1), 24412 (2021)

    Article  Google Scholar 

  51. Mishra, S.S.; Mohapatra, A.K.D.: Weavers’ perception towards sustainability of sambalpuri handloom: a Tukey’s HSD test analysis. Mater. Today Proc. 51, 217–227 (2022)

    Article  Google Scholar 

  52. Kumar, S.; Maity, S.R.; Patnaik, L.: Morphology and wear behavior of monolayer tialn and composite alcrn/tialn-coated plasma-nitrided dac-10 tool steel. Arab. J. Sci. Eng. 47(12), 15519–15538 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Professor Shih-Chung Kang from the University of Alberta for providing us the ACID dataset for experiments. Moreover, the authors are grateful to UFBA, UFRB and UFSJ.

Author information

Authors and Affiliations

  1. Technologic and Exact Center, Federal University of Recôncavo da Bahia, Cruz das Almas, Brazil

    André Luiz C. Ottoni

  2. Department of Electrical and Computer Engineering, Federal University of Bahia, Salvador, Brazil

    Marcela S. Novo

  3. Department of Mathematics and Statistics, Federal University of São João del-Rei, São João del-Rei, Brazil

    Marcos S. Oliveira

Authors
  1. André Luiz C. Ottoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcela S. Novo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcos S. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Luiz C. Ottoni.

Ethics declarations

Conflict of interest

All the authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ottoni, A.L.C., Novo, M.S. & Oliveira, M.S. A Statistical Approach to Hyperparameter Tuning of Deep Learning for Construction Machine Classification. Arab J Sci Eng 49, 5117–5128 (2024). https://doi.org/10.1007/s13369-023-08330-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-023-08330-6

Keywords

Search

Navigation