Abstract
This paper addresses the problem of reconstructing depth and silhouette images of wind turbine from its photos of multiple views using deep learning approaches, which aims for wind turbine blade fault diagnosis. Some previous multi-view based methods have extracted each photo’s silhouette and combined them into separate channels as the input of convolution; others use LSTM to combine a series of views for reconstruction. These approaches inevitably need a fixed number of views and the output result is divergent if the order of the input views is changed. So, we refer to a network, SiDeNet (Wiles and Zisserman, Learning to predict 3d surfaces of sculptures from single and multiple views. Int J Comp Vision, 2018), which has a flexible number of input views and will not be affected by the input order. It integrates both viewpoint and image information from each view to learn a latent 3D shape representation and use it to predict the depth of wind turbine at input views. Also, this representation could generalize to the silhouette of unseen views. We make the following contributions to SiDeNet: improving the resolution of predicted images by deepening network structure, adopting 6D camera pose to increase the degrees of freedom of viewpoint to capture a wider range of views, optimizing the loss function of silhouette by applying weights on edge points, and implementing silhouette refinement with point-wise optimizing. Additionally, we conduct a set of prediction experiments and prove the network’s generalization ability to unseen views. Evaluating predicted results on a realistic wind turbine dataset confirms the high performance of the network on both given views and unseen views.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Poozesh, P., Baqersad, J., Niezrecki, C., Avitabile, P., Harvey, E., Yarala, R.: Large-area photogrammetry based testing of wind turbine blades. Mech. Syst. Signal Process. 86, 98–115 (2017)
Sarrafi, A., Mao, Z., Niezrecki, C., Poozesh, P.: Vibration-based damage detection in wind turbine blades using Phase-based Motion Estimation and motion magnification. J. Sound Vib. 421, 300–318 (2018)
Kirillov, A., Wu, Y., He, K., Girshick. R.: PointRend: Image Segmentation As Rendering. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020)
Xiaowei, L.: Research on fan Blade Detection Based on Fiber Grating Sensing Technology [D]. Nanjing University of Aeronautics and Astronautics (2013)
Wang, W., Chen, A.: Target-less approach of vibration measurement with virtual points constructed with cross ratios. Measurement 151, 107238 (2020)
Zifeng, Q.: Research on Fan Blade Surface Damage Detection Based on Computer Vision [D]. Beijing Jiaotong University (2019)
Corten, G.P., Sabel, J.C.: Optical motion analysis of wind turbines. In: Proceedings of European Union Wind Energy Conference, Goteborg, Sweden, 1996 May 20–24 (1996)
Johnson, J.T., Hughes, S., van Dam, J.: A stereo-videogrammetry system for monitoring wind turbine blade surfaces during structural testing. ASME Early Career Tech. J. 8(1), 1–1 (2009)
Ozbek, M., Rixen, D., Erne, O., Sanow, G.: Feasibility of monitoring large wind turbines using photogrammetry. Energy 35(12), 4802–4811 (2010)
Poozesh, P., Baqersad, J., Niezrecki, C., Harvey, E., Yarala, R.: Full Field Inspection of a Utility Scale Wind Turbine Blade Using Digital Image Correlation. CAMX, Orlando, FL, vol. 10, issue no. 2.1, pp. 2891–2960 (2014)
Vučina, D., Ćurković, M., Novković, T.: Classification of 3D shape deviation using feature recognition operating on parameterization control points. Comput. Ind. 65(6), 1018–1031 (2014)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the ACM SIGGRAPH Conference on Computer Graphics, pp. 187–194 (1999)
Barron, J., Malik, J.: Shape, illumination, and reflectance from shading. IEEE Trans. Patt. Anal. Mach. Intell. 37, 1670–1687 (2015)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, USA (2003)
Wiles, O., Zisserman, A.: Learning to predict 3d surfaces of sculptures from single and multiple views. Int. J. Comp. Vision (2018)
Laurentini, A.: The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)
Kolev, K., Klodt, M., Brox, T., Cremers, D.: Continuous Global Optimization in Multiview 3D Reconstruction, vol. 84, pp. 80–96. Springer (2009)
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N..: Deeper depth prediction with fully convolutional residual networks. In: Fourth International Conference on 3d Vision. IEEE (2016)
Yao, Y., Luo, Z., Li, S., Fang, T., Quan, L.: MVSNet: Depth Inference for Unstructured Multi-view Stereo (2018)
Chen, R., Han, S., Xu, J., Su, H.: Point-Based Multi-View Stereo Network, pp. 1538–1547. https://doi.org/10.1109/ICCV.2019.00162 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (2015)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: A Convolutional Neural Network for 6d Object Pose Estimation in Cluttered Scenes (2017)
Kok, F., Charles, J., Cipolla, R.: Footnet: an efficient convolutional network for multiview 3d foot reconstruction. In: Asian Conference on Computer Vision (2020)
Yan, X., Yang, J., Yumer, E., Guo, Y., Lee, H.: Perspective transformer nets: LEARNING single-view 3D object reconstruction without 3D supervision. In: Advances in Neural Information Processing Systems (2016)
Acknowledgements
The work was supported by the National Natural Science Foundation of China (Project no. 62076029) and an internal funding from United International College.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, M., Zhao, M., Bai, Y., Gao, R., Deng, R., Zhang, H. (2023). Image-Based 3D Shape Estimation of Wind Turbine from Multiple Views. In: Zhang, H., Feng, G., Wang, H., Gu, F., Sinha, J.K. (eds) Proceedings of IncoME-VI and TEPEN 2021. Mechanisms and Machine Science, vol 117. Springer, Cham. https://doi.org/10.1007/978-3-030-99075-6_82
Download citation
DOI: https://doi.org/10.1007/978-3-030-99075-6_82
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99074-9
Online ISBN: 978-3-030-99075-6
eBook Packages: EngineeringEngineering (R0)