Mapping Motion-Magnified Videos to Operating Deflection Shape Vectors Using Particle Filters

  • Aral Sarrafi
  • Zhu MaoEmail author
Conference paper
Part of the Conference Proceedings of the Society for Experimental Mechanics Series book series (CPSEMS)


Phase-based motion estimation and magnification are targetless methods that have been used recently to perform experimental modal analysis (EMA) and operational modal analysis (OMA) on a variety of structures. Mapping the motion-magnified sequence of images into quantified operating deflection shape (ODS) vectors is currently being conducted via edge detection methods that require intensive human supervision and interference. Within this study, a new hybrid computer vision approach is introduced to extract the quantified ODS vectors from the motion-magnified sequence of images with minimal human supervision. The particle filter point tracking method is utilized to follow the desired feature points in the motion-magnified sequence of images. Moreover, the k-means clustering algorithm is employed as an unsupervised learning approach to performing the segmentation of the particles and assigning them to specific feature points in the in the motion-magnified sequence of images. This study shows that the cluster centers can be employed to estimate the ODS vectors, and the performance of the proposed methodology is evaluated experimentally on a lab-scale cantilever beam and validated via a finite element model.


Phase-based motion estimation Video magnification Particle filter Computer vision Clustering Unsupervised learning 


  1. 1.
    Baqersad, J., Poozesh, P., Niezrecki, C., Avitabile, P.: Photogrammetry and optical methods in structural dynamics–a review. Mech. Syst. Signal Process. 86(Part B), 17 (2017)CrossRefGoogle Scholar
  2. 2.
    Feng, D., Feng, M.Q.: Computer vision for SHM of civil infrastructure: from dynamic response measurement to damage detection–a review. Eng. Struct. 156, 105–117 (2018)CrossRefGoogle Scholar
  3. 3.
    Sarrafi, A., Poozesh, P., Mao, Z.: A comparison of computer-vision-based structural dynamics characterizations. In: Model Validation and Uncertainty Quantification, Volume 3, pp. 295–301. Springer, Cham (2017)CrossRefGoogle Scholar
  4. 4.
    Castellini, P., Martarelli, M., Tomasini, E.P.: Laser Doppler Vibrometry: development of advanced solutions answering to technology’s needs. Mech. Syst. Signal Process. 20, 1265–1285 (2006)CrossRefGoogle Scholar
  5. 5.
    Cigada, A., Mazzoleni, P., Zappa, E.: Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system. Exp. Mech. 54, 255–271 (2014)CrossRefGoogle Scholar
  6. 6.
    Mazzoleni, P., Zappa, E.: Vision-based estimation of vertical dynamic loading induced by jumping and bobbing crowds on civil structures. Mech. Syst. Signal Process. 33, 1–12 (2012)CrossRefGoogle Scholar
  7. 7.
    Li, J., Xie, X., Yang, G., Zhang, B., Siebert, T., Yang, L.: Whole-field thickness strain measurement using multiple camera digital image correlation system. Opt. Lasers Eng. 90, 19–25 (2017)CrossRefGoogle Scholar
  8. 8.
    Xie, X., Zeng, D., Li, J., Dahl, J., Zhao, Q., Yang, L.: Tensile test for polymer plastics with extreme large elongation using quad-camera digital image correlation. SAE Technical Paper 0148-7191 (2016)Google Scholar
  9. 9.
    Baqersad, J., Niezrecki, C., Avitabile, P.: Full-field dynamic strain prediction on a wind turbine using displacements of optical targets measured by stereophotogrammetry. Mech. Syst. Signal Process. 62, 284–295 (2015)CrossRefGoogle Scholar
  10. 10.
    Baqersad, J., Niezrecki, C., Avitabile, P.: Extracting full-field dynamic strain on a wind turbine rotor subjected to arbitrary excitations using 3D point tracking and a modal expansion technique. J. Sound Vib. 352, 16–29 (2015)CrossRefGoogle Scholar
  11. 11.
    Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56, 221–255 (2004)CrossRefGoogle Scholar
  12. 12.
    Javh, J., Slavič, J., Boltežar, M.: The subpixel resolution of optical-flow-based modal analysis. Mech. Syst. Signal Process. 88, 89–99 (2017)CrossRefGoogle Scholar
  13. 13.
    Javh, J., Slavič, J., Boltežar, M.: High frequency modal identification on noisy high-speed camera data. Mech. Syst. Signal Process. 98, 344–351 (2018)CrossRefGoogle Scholar
  14. 14.
    Feng, D., Feng, M.Q.: Identification of structural stiffness and excitation forces in time domain using noncontact vision-based displacement measurement. J. Sound Vib. 406, 15–28 (2017)CrossRefGoogle Scholar
  15. 15.
    Feng, D., Feng, M.Q., Ozer, E., Fukuda, Y.: A vision-based sensor for noncontact structural displacement measurement. Sensors. 15, 16557–16575 (2015)CrossRefGoogle Scholar
  16. 16.
    Celik, O., Dong, C.-Z., Catbas, F.N.: A computer vision approach for the load time history estimation of lively individuals and crowds. Comput. Struct. 200, 32–52 (2018)CrossRefGoogle Scholar
  17. 17.
    Fleet, D., Weiss, Y.: Optical flow estimation. In: Handbook of Mathematical Models in Computer Vision, pp. 237–257. Springer, New York (2006)CrossRefGoogle Scholar
  18. 18.
    Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. Int. J. Comput. Vis. 5, 77–104 (1990)CrossRefGoogle Scholar
  19. 19.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Phase-based video motion processing. ACM Transactions on Graphics (TOG). 32, 80 (2013)CrossRefGoogle Scholar
  20. 20.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Riesz pyramids for fast phase-based video magnification. US Patent 9,338,331, 2016Google Scholar
  21. 21.
    Chen, J.G., Wadhwa, N., Cha, Y.-J., Durand, F., Freeman, W.T., Buyukozturk, O.: Modal identification of simple structures with high-speed video using motion magnification. J. Sound Vib. 345, 58–71 (2015)CrossRefGoogle Scholar
  22. 22.
    Yang, Y., Dorn, C., Mancini, T., Talken, Z., Theiler, J., Kenyon, G., et al.: Reference-free detection of minute, non-visible, damage using full-field, high-resolution mode shapes output-only identified from digital videos of structures. Struct. Health Monit. 17(3), 1475921717704385 (2017)Google Scholar
  23. 23.
    Yang, Y., Dorn, C., Mancini, T., Talken, Z., Kenyon, G., Farrar, C., et al.: Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification. Mech. Syst. Signal Process. 85, 567–590 (2017)CrossRefGoogle Scholar
  24. 24.
    Yang, Y., Dorn, C., Mancini, T., Talken, Z., Nagarajaiah, S., Kenyon, G., et al.: Blind identification of full-field vibration modes of output-only structures from uniformly-sampled, possibly temporally-aliased (sub-Nyquist), video measurements. J. Sound Vib. 390, 232–256 (2017)CrossRefGoogle Scholar
  25. 25.
    Chen, J.G., Wadhwa, N., Cha, Y.-J., Durand, F., Freeman, W.T., Buyukozturk, O.: Structural modal identification through high speed camera video: Motion magnification. In: Topics in Modal Analysis I, vol. 7, pp. 191–197. Springer International Publishing (2014)Google Scholar
  26. 26.
    Sarrafi, A., Poozesh, P., Niezrecki, C., Mao, Z.: Detection of natural frequency and mode shape correspondence using phase-based video magnification in large-scale structures. In: Niezrecki, C., Baqersad, J. (eds.) Structural Health Monitoring, Photogrammetry & DIC, vol. 6. Conference Proceedings of the Society for Experimental Mechanics Series. Springer, Cham (2019)Google Scholar
  27. 27.
    Sarrafi, A., Mao, Z., Niezrecki, C., Poozesh, P.: Vibration-based damage detection in wind turbine blades using Phase-based Motion Estimation and motion magnification. J. Sound Vib. 421, 300–318 (2018)CrossRefGoogle Scholar
  28. 28.
    Sarrafi, A., Poozesh, P., Niezrecki, C., Mao, Z.: Mode extraction on wind turbine blades via phase-based video motion estimation. In: SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring, pp. 101710E–101710E-12 (2017)Google Scholar
  29. 29.
    Shang, Z., Shen, Z.: Multi-point vibration measurement and mode magnification of civil structures using video-based motion processing. Autom. Constr. 93, 231–240 (2018)CrossRefGoogle Scholar
  30. 30.
    Arulampalam, M.S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50, 174–188 (2002)CrossRefGoogle Scholar
  31. 31.
    Schulz, D., Burgard, W., Fox, D., Cremers, A.B.: Tracking multiple moving targets with a mobile robot using particle filters and statistical data association. In: Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 2001, pp. 1665–1670Google Scholar
  32. 32.
    Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31, 651–666 (2010)CrossRefGoogle Scholar

Copyright information

© Society for Experimental Mechanics, Inc. 2019

Authors and Affiliations

  1. 1.Structural Dynamics and Acoustic Systems Laboratory, Department of Mechanical EngineeringUniversity of Massachusetts LowellLowellUSA

Personalised recommendations