Abstract
Skeleton construction and pose estimation through two-dimensional (2D) images may have reduced estimation accuracy due to truncation and occlusion that may occur within the image. In this paper, a 3D model is created through NeRF (Neural Radiance Fields) based Instant-NGP (Neural Graphics Primitives), a skeleton is constructed, and the artificial intelligence is trained to perform pose estimation in a 2D image. Create a 3D model by performing image view synthesis using Instant-NGP on images taken from 4 or more different angles. After that, we use DeepLabCut to build joint coordinates and skeletons. By learning the built skeleton to artificial intelligence, a pre-trained AI model is created. Using an artificial intelligence model learned in advance through a 3D model, joint coordinate recognition in a 2D image, skeleton construction, posture estimation, and classification are carried out. Through the test image dataset, the posture estimation and classification accuracy of the artificial intelligence model trained in advance with the 3D model and the artificial intelligence model learned using the existing 2D image are compared.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang Z, Hu L, Deng X, Xia S (2021) Sequential 3D human pose estimation using adaptive point cloud sampling strategy. In: Proceedings of the 30th international joint conference on artificial intelligence. IJCAI, Montreal, pp 1330–1337
Xiu Y, Li J, Wang H, Fang Y, Lu C (2018) Pose flow: efficient online pose tracking, pp 1–12. arXiv preprint arXiv:1802.00977
Colombel J, Daney D, Bonnet V, Charpillet F (2021) Markerless 3D human pose tracking in the wild with fusion of multiple depth cameras: comparative experimental study with Kinect 2 and 3. In: Activity and behavior computing, vol 204, pp 119–134
Shen X, Plested J, Yao Y, Gedeon T (2020) Pairwise-GAN: pose-based view synthesis through pair-wise training. In: International conference on neural information processing, vol 1332. Springer, Bangkok, pp 507–515
Mildenhall B, Srinivasan P, Tancik M, Barron J, Ramamoorthi R, Ng R (2021) NeRF: representing scenes as neural radiance fields for view synthesis. Commun ACM 65(1):99–106
Dang Q, Yin J, Wang B, Zheng W (2019) Deep learning based 2D human pose estimation: a survey. Tsinghua Sci Technol 24(6):663–676
Dong J, Fang Q, Jiang W, Yang Y, Huang Q, Bao H, Zhou X (2021) Fast and robust multi-person 3D pose estimation and tracking from multiple views. IEEE Trans Pattern Anal Mach Intell 44(10):6981–6992
Nath T, Mathis A, Chen A, Patel A, Bethge M, Mathis W (2019) Using DeepLabCut for 3D markerless pose estimation across species and behaviors. Nat Protoc 14(7):2152–2176
Meng Q, Chen A, Luo H, Wu M, Su H, Xu L, He X, Yu J (2021) GNeRF: GAN-based neural radiance field without posed camera. In: Proceedings of the IEEE/CVF international conference on computer vision. IEEE, Montreal, pp 6331–6341
Müller T, Evans A, Schied C, Keller A (2022) Instant neural graphics primitives with a multiresolution hash encoding, pp 1–15. arXiv preprint arXiv:2201.05989
Acknowledgements
This paper was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C2011966).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Park, C., Kim, H., Moon, N. (2023). Design of 2D to 3D Pose Estimation Using NeRF Image View Synthesis. In: Park, J.S., Yang, L.T., Pan, Y., Park, J.H. (eds) Advances in Computer Science and Ubiquitous Computing. CUTECSA 2022. Lecture Notes in Electrical Engineering, vol 1028. Springer, Singapore. https://doi.org/10.1007/978-981-99-1252-0_18
Download citation
DOI: https://doi.org/10.1007/978-981-99-1252-0_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1251-3
Online ISBN: 978-981-99-1252-0
eBook Packages: EngineeringEngineering (R0)