Skip to main content
SpringerLink
Log in

Bridge Load Classifier Based on Deep Learning for Structural Displacement Correlation

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Advanced computing brings opportunities for innovation in a broad gamma of applications. Traditional practices based on visual and manual methods tend to be replaced by cyber-physical systems to automate processes. The present work introduces an example of this, a machine vision system research based on deep learning to classify bridge load, to give support to an optical scanning system for structural health monitoring tasks. The optical scanning system monitors the health of structures, such as buildings, warehouses, water dams, etc. by the measurement of their coordinates to identify if a coordinate displacement befalls that could indicate an anomaly in the structure that can be related to structural damage. The use of this optical scanning system to monitor the structural health of bridges is a little more complicated due to the vehicle’s transit over the bridge that causes a vehicle-bridge interaction which manifests as a bridge oscillation. Under this scheme, the bridge oscillation corresponds to their coordinate’s displacement due to the vehicle-bridge interaction, but not necessarily due to bridge damage. So, a bridge load classifier is required to correlate the bridge coordinates measurements behavior with the bridge oscillation due to vehicle-bridge interaction to discriminate the normal behavior of the structure to abnormal behavior or identify tendencies that could indicate bridge deformation or discover if the bridge behavior due to loads is changing through the time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Similar content being viewed by others

REFERENCES

  1. Massobrio, R., Nesmachnow, S., Tchernykh, A., Avetisyan, A., and Radchenko, G., Towards a cloud computing paradigm for big data analysis insmart cities, Program. Comput. Software, 2018, vol. 44, no. 3, pp. 181–189.

    Article  Google Scholar 

  2. Massobrio, R., Nesmachnow Cánovas, S.E., Tchernykh, A.N., Avetisyan, A. ., and Radchenko, G.I., Towards a cloud computing paradigm for big data analysis in smart cities, Proc. Inst. Syst. Program. Russ. Acad. Sci., 2016, vol. 28, no. 6, pp. 121–140.

    Google Scholar 

  3. Mufti, A. and Helmi, K., A case for structural health monitoring (SHM) and civionics enhances the evaluation of the load carrying capacity of aging ridges, Innovative Infrastruct. Solutions, 2019, vol. 4, no. 1, p. 3.

    Google Scholar 

  4. Barousse Moreno, M. and Galindo Solozano, A., Bridge Management System (SIAP), Publicación técnica, 1994, vol. 49.

  5. Lydon, M., Taylor, S.E., Robinson, D., Callender, P., Doherty, C., Grattan, S.K., and O’Brien, E.J., Development of a bridge weigh-in-motion sensor: performance comparison using fiber optic and electric resistance strain sensor systems, IEEE Sens. J., 2014, vol. 14, no. 12, pp. 4284–4296.

    Article  Google Scholar 

  6. Wong, K.Y., Instrumentation and health monitoring of cable-supported bridges, Struct. Control Health Monitoring, 2004, vol. 11, no. 2, pp. 91–124.

    Article  Google Scholar 

  7. Aono, K., Hasni, H., Pochettino, O., Lajnef, N., and Chakrabartty, S., Quasi-self-powered piezo-floating-gate sensing technology for continuous monitoring of large-scale bridges, Front. Built Environ., 2019, vol. 5, no. 29.

  8. Ban, W.H., Hu, J.W., and Kaloop, M. R., Prestressed continuous bridge evaluation using structural health monitoring system, in IOP Conf. Ser.: Mater. Sci. Eng., 2019, vol. 473, no. 1, p. 012048.

  9. Çaktı, E. and Şafak, E., Structural health monitoring: lessons learned, in Seismic Isolation, Structural Health Monitoring, and Performance Based Seismic Design in Earthquake Engineering, Cham: Springer, 2019, pp. 145–164.

    Google Scholar 

  10. Lobo-Aguilar, S., Zhang, Z., Jiang, Z., and Christenson, R., Infrasound-based noncontact sensing for bridge structural health monitoring, J. Bridge Eng., 2019, vol. 24, no. 5, pp. 04019033.

    Article  Google Scholar 

  11. Megid, W.A., Chainey, M.A., Lebrun, P., and Hay, D.R., Monitoring fatigue cracks on eyebars of steel bridges using acoust ic emission: a case study, Eng. Fract. Mech., 2019, vol. 211, pp. 198–208.

    Article  Google Scholar 

  12. Sony, S., Laventure, S., and Sadhu, A., A literature review of next generation smart sensing technology in structural health monitoring, Struct. Control Health Monit., 2019, vol. 26, no. 3, pp. e2321.

    Article  Google Scholar 

  13. Rodríguez, A.S., Rodríguez, B.R., Rodríguez, M.S., and Sánchez, P.A., Laser scanning and its applications to damage detection and monitoring in masonry structures, in Long-Term Performance and Durability of Masonry Structures, Woodhead Publ., 2019, pp. 265–285.

    Google Scholar 

  14. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko, J. C. Rodrнguez-Quiconez, D. Hernбndez-Balbuena, & J. Rivera-Castillo. Energy center detection in light scanning sensors for structural health monitoring accuracy enhancement. IEEE Sensors Journal, vol. 14, no. 7, pp. 2355–2361, 2014.

    Article  Google Scholar 

  15. J. Rivera-Castillo, W. Flores-Fuentes, M. Rivas- ypez, O. Sergiyenko, F.F. Gonzalez-Navarro, J. C. Rodrнguez-Quiconez, & L. C. Bбsaca-Preciado. Experimental image and range scanner datasets fusion in shm for displacement detection. Structural Control and Health Monitoring, vol. 24, no. 10, pp. e1967, 2017.

    Article  Google Scholar 

  16. L. Ma, W. Zhang, W. S. Han, & J. X. Liu. Determining the dynamic amplification factor of multi-span continuous box girder bridges in highways using vehicle-bridge interaction analyses. Engineering Structures, vol. 181, pp. 47–59, 2019.

    Article  Google Scholar 

  17. C. Emmanouilidis, P. Pistofidis, L. Bertoncelj, V. Katsouros, A, Fournaris, C. Koulamas, & C. Ruiz-Carcel. Enabling the human in the loop: Linked data and knowledge in industrial cyber-physical systems. ISSN 0361-7688, Programming and Computer Software, 2020, Vol. .., No. .., pp. … Annual Reviews in Control, vol. 47, pp. 249–265, 2019.

  18. P. Ramachandran, B. Zoph, & Q. V. Le. Swish: a self-gated activation function. arXiv preprint arXiv:1710.05941, 7, 2017.

  19. K. B. Park, M. Kim, S. H. Choi, & J. Y. Lee. Deep learning-based smart task assistance in wearable augmented reality. Robotics and Computer-Integrated Manufacturing, vol. 63, pp. 101887, 2020.

    Article  Google Scholar 

  20. Y. Pan, F. He, & H. Yu. A correlative denoising autoencoder to model social influence for top-n recommender system. Frontiers of Computer Science, vol. 14, no. 3, pp. 143301, 2020.

    Article  Google Scholar 

  21. R. Wang, H. K. Cheng, Y. Jiang, & J. Lou. TDCF: A Two-stage Deep Learning based Recommendation Model. Expert Systems with Applications, vol. 145, pp. 113116, 2019.

    Article  Google Scholar 

  22. S. Wang, H. Chen, L. Wu, & J. Wang. A novel smart meter data compression method via stacked convolutional sparse auto-encoder. International Journal of Electrical Power & Energy Systems, vol. 118, pp. 105761, 2020.

    Article  Google Scholar 

  23. A. Joukhadar, D. K. Hanna, & E. A. Al-Izam. UKFBased Image Filtering and 3D Reconstruction. In Machine Vision and Navigation, pp. 267–289. Springer, Cham, 2020.

    Google Scholar 

  24. J. W. Bae, A. Rykhlevskii, G. Chee, & K. D. Huff. Deep learning approach to nuclear fuel transmutation in a fuel cycle simulator. Annals of Nuclear Energy, vol. 139, pp. 107230, 2020.

    Article  Google Scholar 

  25. J. Wu, N. Cai, F. Li, H. Jiang, & H. Wang. Automatic Detonator Code Recognition via Deep Neural Network. Expert Systems with Applications, pp. 113121, 2019.

  26. A. K. Bhunia, S. Mukherjee, A. Sain, A. K. Bhunia, P. P. Roy, & U. Pal. Indic handwritten script identification using offline-online multi-modal deep network. Information Fusion, vol. 57, p.p. 1–14, 2020.

    Article  Google Scholar 

  27. A. Blanco, O. P. de Vicaspre, A. Pйrez, & A. Casillas. Boosting ICD multi-label classification of health records with contextual embeddings and labelgranularity. Computer Methods and Programs in Biomedicine, pp. 105264, 2019.

  28. T. Meng, X. Jing, Z. Yan, W. Pedrycz. A Survey on Machine Learning for Data Fusion. Information Fusion, 2019.

  29. M. Xiang, J. Yu, Z. Yang, Y. Yang, H. Yu, & H. He. Probabilistic power flow with topology changes based on deep neural network. International Journal of Electrical Power & Energy Systems, vol. 117, pp. 105650, 2020.

    Article  Google Scholar 

  30. D. Cao, W. Hu, X. Xu, T. Dragičević, Q. Huang, Z. Liu, & F. Blaabjerg. Bidding strategy for trading wind energy and purchasing reserve of wind power producer–A DRL based approach. International Journal of Electrical Power & Energy Systems, vol. 117, pp. 105648, 2020.

    Article  Google Scholar 

  31. G. Zhang, W. Hu, D. Cao, J. Yi, Q. Huang, Z. Liu, … & F. Blaabjerg. A data-driven approach for designing STATCOM additional damping controller for wind farms. International Journal of Electrical Power & Energy Systems, vol. 117, pp.105620, 2020.

    Article  Google Scholar 

  32. C. Xiao, R. Qin, & X. Huang. Treetop detection using convolutional neural networks trained through automatically generated pseudo labels. International Journal of Remote Sensing, vol. 41, no. 8, pp. 3010–3030, 2020.

    Article  Google Scholar 

  33. S. K. Pandey, R. B. Mishra, & A. K. Tripathi. BPDET: An effective software bug prediction model using deep representation and ensemble learning techniques. Expert Systems with Applications, vol. 144, pp. 113085, 2020.

    Article  Google Scholar 

  34. K. Bijari, H. Zare, E. Kebriaei, & H. Veisi. Leveraging deep graph-based text representation for sentiment polarity applications. Expert Systems with Applications, vol. 144, pp. 113090, 2020.

    Article  Google Scholar 

  35. S. Bera, & V. K. Shrivastava. Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remotesensing image classification. International Journal of Remote Sensing, vol. 41, no. 7, pp. 2664–2683, 2019.

    Article  Google Scholar 

  36. M. A. Ebrahimighahnavieh, S. Luo, & R. Chiong. Deep learning to detect Alzheimer’s disease from neuroimaging: A systematic literature review. Computer Methods and Programs in Biomedicine, vol. 187, pp. 105242, 2020.

    Article  Google Scholar 

  37. Y. Ren, J. Huang, Z. Hong, W. Lu, J. Yin, L. Zou, & X. Shen. Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials, vol. 234, pp. 117367, 2020.

    Article  Google Scholar 

  38. N. S. Gulgec, M. Takбč, & S. N. Pakzad. Experimental Study on Digital Image Correlation for Deep Learning-Based Damage Diagnostic. In Dynamics of Civil Structures, Volume 2 (pp. 205–210). Springer, Cham, 2020.

    Google Scholar 

  39. W. Deng, Y. Mou, T. Kashiwa, S. Escalera, K. Nagai, K. Nakayama, … & H. Prendinger. Vision based pixellevel bridge structural damage detection using a link ASPP network. Automation in Construction, vol. 110, pp. 102973, 2020.

    Article  Google Scholar 

  40. J. J. Rubio, T. Kashiwa, T. Laiteerapong, W. Deng, K. Nagai, S. Escalera, … & H. Prendinger. Multi-class structural damage segmentation using fully convolutional networks. Computers in Industry, vol. 112, pp. 103121, 2019.

    Article  Google Scholar 

  41. W. Deng, Y. Mou, T. Kashiwa, S. Escalera, K. Nagai, K. Nakayama, … & H. Prendinger. Vision based pixellevel bridge structural damage detection using a link ASPP network. Automation in Construction, vol. 110, pp. 102973, 2020.

    Article  Google Scholar 

  42. Y. Bao, Z. Tang, H. Li, & Y. Zhang). Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Structural Health Monitoring, vol. 18, vol. 2, pp. 401–421, 2019.

  43. Y. Xia, X. Jian, B. Yan, & D. Su. Infrastructure Safety Oriented Traffic Load Monitoring Using Multi-Sensor and Single Camera for Short and Medium Span Bridges. Remote Sensing, vol. 11, no. 22, pp. 2651, 2019.

    Article  Google Scholar 

  44. X. Jian, Y. Xia, J. A. Lozano-Galant, & L. Sun. Traffic Sensing Methodology Combining Influence Line Theory and Computer Vision Techniques for Girder Bridges. Journal of Sensors, 2019.

  45. D. K. Amara, R. Karthika, & K. P. Soman, DeepTrackNet: Camera Based End to End Deep Learning Framework for Real Time Detection, Localization and Tracking for Autonomous Vehicles. In International Conference on Intelligent Computing, Information and Control Systems (pp. 299–307). Springer, Cham, (2019, June).

  46. C. Cardenas, & M. Gonzalez-Mendoza. Distributed System Based on Deep Learning for Vehicular Rerouting and Congestion Avoidance. In Trends and Applications in Software Engineering: Proceedings of the 8th International Conference on Software Process Improvement (CIMPS 2019) (Vol. 1071, p. 159). Springer Nature, (2019, October).

  47. S. Zhang, C. Wang, Z. He, Q. Li, X. Lin, X. Li, … & J. Li. Vehicle global 6-DoF pose estimation under traffic surveillance camera. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 159, pp. 114–128, 2020.

    Article  Google Scholar 

  48. C. K. Ng, S. N. Cheong, & Y. L. Foo. Low Latency Deep Learning Based Parking Occupancy Detection By Exploiting Structural Similarity. In Computational Science and Technology (pp. 247–256). Springer, Singapore, 2020.

    Google Scholar 

  49. J. E. Miranda-Vega, W. Flores-Fuentes, O. Sergiyenko, M. Rivas-Lypez, L. Lindner, J. C. Rodrнguez-Quiconez, & D. Hernбndez-Balbuena. Optical cyber-physical system embedded on an FPGA for 3D measurement in structural health monitoring tasks. Microprocessors and Microsystems, vol. 56, pp. 121–133, 2018.

    Article  Google Scholar 

  50. V. Tyrsa, O. Sergiyenko, L. Burtseva, M. Bravo-Zanoguera, L. Devia, I. Rendon, & V. Tyrsa. Mobile transport object control by technical vision means. In Electronics, Robotics and Automotive Mechanics Conference (CERMA'06) (Vol. 2, pp. 74–82). IEEE. (2006, September).

  51. Ivanov, M., Sergiyenko, O., Tyrsa, V., Lindner, L., Rodriguez-Quiconez, J. C., Flores-Fuentes, W., … & Hipylito, J. N. (2019). Software Advances using nagents Wireless Communication Integration for Optimization of Surrounding Recognition and Robotic Group Dead Reckoning. Programming and Computer Software, 45(8), 557–569.

    Article  Google Scholar 

  52. M. Rivas, O. Sergiyenko, M. Aguirre, L. Devia, V. Tyrsa, & I. Rendyn. Spatial data acquisition by laser scanning for robot or SHM task. In 2008 IEEE International Symposium on Industrial Electronics (pp. 1458–1462). IEEE. (2008, June).

  53. M. R. Lypez, O. Sergiyenko, & V. Tyrsa. Machine vision: approaches and limitations. In Computer vision. IntechOpen. 2008.

    Google Scholar 

  54. O. Sergiyenko, V. Tyrsa, D. Hernandez-Balbuena, M. R. Lopez, I. R. Lopez, & L. D. Cruz. Precise optical scanning for practical multi-applications. In 2008 34th Annual Conference of IEEE Industrial Electronics (pp. 1656–1661). IEEE. 2008, November.

  55. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko, J. C. Rodrнguez-Quiconez, D. Hernбndez-Balbuena, & J. Rivera-Castillo. Energy center detection in light scanning sensors for structural health monitoring accuracy enhancement. IEEE Sensors Journal, vol. 14, no. 7, pp. 2355–2361, 2014.

    Article  Google Scholar 

  56. W. Flores-Fuentes, M. Rivas-Lopez, O. Sergiyenko, F. F. Gonzalez-Navarro, J. Rivera-Castillo, D. Hernandez-Balbuena & J. C. Rodrнguez-Quiconez. Combined application of power spectrum centroid and support vector machines for measurement improvement in optical scanning systems. Signal Processing, vol. 98, pp. 37–51, 2014.

    Article  Google Scholar 

  57. M. Reyes-Garcia, C. Sepulveda-Valdez, O. Sergiyenko, M. Rivas-Lypez, J. C. Rodrнguez-Quiconez, W. Flores-Fuentes, … & M. Ivanov. Digital Control Theory Application and Signal Processing in a Laser Scanning System Applied for Mobile Robotics. In Control and Signal Processing Applications for Mobile and Aerial Robotic Systems (pp. 215–265). IGI Global.2020.

    Google Scholar 

  58. L. Lindner, O. Sergiyenko, M. Rivas-Lypez, M. Ivanov, J. C. Rodrнguez-Quiconez, D. Hernбndez-Balbuena, & P. Mercorelli. Machine vision system errors for unmanned aerial vehicle navigation. In 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) (pp. 1615–1620). IEEE. (2017, June).

  59. W. Flores-Fuentes, J. Miranda-Vega, M. Rivas-Lypez, O. Sergiyenko, J. Rodrнguez-Quiconez, & L. Lindner. Comparison between Different Types of Sensors Used in the Real Operational Environment Based on Optical Scanning System. Sensors, vol. 18, no. 6, pp. 1684, 2018.

    Article  Google Scholar 

  60. J. Krause, M. Stark, J. Deng, & L. Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 554–561), 2013.

  61. Varnovskiy, N. P., Martishin, S. A., Khrapchenko, M. V., & Shokurov, A. V. (2015). Secure cloud computing based on threshold homomorphic encryption. Programming and Computer Software, 41(4), 215–218.

    Article  MathSciNet  MATH  Google Scholar 

  62. Miranda-Lypez, V., Tchernykh, A., Cortйs-Mendoza, J. M., Babenko, M., Radchenko, G., Nesmachnow, S., & Du, Z. (2017, September). Experimental analysis of secret sharing schemes for cloud storage based on rns. In Latin American High Performance Computing Conference (pp. 370–383). Springer, Cham.

  63. Tchernykh, A., Babenko, M., Chervyakov, N., Cortйs-Mendoza, J. M., Kucherov, N., Miranda-Lypez, V., … & Radchenko, G. (2017, August). Towards mitigating uncertainty of data security breaches and collusion in cloud computing. In 2017 28th International Workshop on Database and Expert Systems Applications (DEXA) (pp. 137–141). IEEE.

  64. Tchernykh, A., Babenko, M., Chervyakov, N., Miranda-Lypez, V., Kuchukov, V., Cortйs-Mendoza, J. M., … & Avetisyan, A. (2018). AC-RRNS: Anticollusion secured data sharing scheme for cloud ISSN 0361-7688, Programming and Computer Software, 2020, Vol. .., No. .., pp. … storage. International Journal of Approximate Reasoning, 102, 60–73.

Download references

ACKNOWLEDGMENTS

The work is partially supported by Conacyt and Universidad Autónoma de Baja California.

Author information

Authors and Affiliations

  1. Universidad Autónoma de Baja California, Blvd. Benito Juárez S/N. Parcela 44, C.P. 21280, Mexicali, B.C., Mexico

    Wendy Flores-Fuentes

Authors
  1. Wendy Flores-Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wendy Flores-Fuentes.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Flores-Fuentes, W. Bridge Load Classifier Based on Deep Learning for Structural Displacement Correlation. Program Comput Soft 46, 526–535 (2020). https://doi.org/10.1134/S0361768820080101

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768820080101

Search

Navigation