Abstract
In spatial missions, it is important to estimate kinematic state and to identify the shape of non-cooperative targets. In order to improve the accuracy and generality of the existing algorithm, a real-time recognition and detection method for tumbling non-cooperative target is proposed. First, we design a key point detection network to identify non-cooperative targets and their feature points in the process of long-distance non-cooperative target recognition. At the same time, the detected image is combined with the PNP algorithm to obtain the target’s 6D attitude. Then, a BiSeNet based model is trained for real-time semantic segmentation of satellite components as the base satellite is pursued close to its target. The segmented image is selected with depth information and the relative position of the capturing point is transmitted to the manipulator. Finally, we complete physical experiments under different lighting conditions with a spinning non-cooperative target on a 6-DOF air-bearing table. The experimental results show that the satellite recognition accuracy and object segmentation accuracy are 99.48% and 98.11%, respectively. The position measurement error is less than 1 mm, which achieves more than 50% improvement over the conventional methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Our dataset and program code are public; you can contact us by email if you need. duhang2020@foxmail.com.
References
Benninghoff H, Rems F, Risse EA et al (2017) European proximity operations simulator 2.0 (EPOS)—a robotic-based rendezvous and docking simulator. J Large Scale Res Fac JLSRF 3:107. https://doi.org/10.17815/jlsrf-3-155
Flores-Abad A, Ma O, Pham K, Ulrich S (2014) A review of space robotics technologies for on-orbit servicing. Prog Aerosp Sci 68:1–26. https://doi.org/10.1016/j.paerosci.2014.03.002
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision & pattern recognition. IEEE Computer Society, pp 770–778
Inaba N, Oda M, Asano M (2006) Rescuing a stranded satellite in space—experimental robotic capture of non-cooperative satellites. Trans Jpn Soc Aeronaut Space Sci 48(162):213–220. https://doi.org/10.2322/tjsass.48.213
Jigalin A, Gurfil P (2016) Investigation of multiple-baseline stereovision for state estimation of unknown dynamic space targets. Proc Inst Mech Eng Part G J Aerosp Eng 230(2):207–233. https://doi.org/10.1177/0954410015590636
Lan R, Zhou Y, Liu Z et al (2020) Prior knowledge-based probabilistic collaborative representation for visual recognition. IEEE Trans Cybern 50(4):1498–1508. https://doi.org/10.1109/TCYB.2018.2880290
Lan R, Sun L, Liu Z et al (2021) Madnet: a fast and lightweight network for single-image super resolution. IEEE Trans Cybern 51(3):1443–1453. https://doi.org/10.1109/TCYB.2020.2970104
Li WJ, Cheng DY, Liu XG et al (2019) On-orbit service (OOS) of spacecraft: a review of engineering developments. Prog Aerosp Sci 108:32–120. https://doi.org/10.1016/j.paerosci.2019.01.004
Lu H, Li Y, Mu S et al (2017) Motor anomaly detection for unmanned aerial vehicles using reinforcement learning. IEEE Internet Things J 5(4):2315–2322. https://doi.org/10.1109/JIOT.2017.2737479
Lu H, Li Y, Chen M et al (2018) Brain intelligence: go beyond artificial intelligence. Mob Netw Appl 23(2):368–375. https://doi.org/10.1007/s11036-017-0932-8
Lu H, Wang D, Li Y et al (2019) CONet: a cognitive ocean network. IEEE Wirel Commun 26(3):90–96. https://doi.org/10.1109/MWC.2019.1800325
Ng WH, Tran HT, Martorella M et al (2017) Estimation of the total rotational velocity of a non-cooperative target with a high cross-range resolution three-dimensional interferometric inverse synthetic aperture radar system. IET Radar Sonar Navig 11(6):1020–1029. https://doi.org/10.1049/iet-rsn.2016.0462
Oumer NW, Panin G, Mülbauer Q et al (2015) Vision-based localization for on-orbit servicing of a partially cooperative satellite. Acta Astronaut 117:19–37. https://doi.org/10.1016/j.actaastro.2015.07.025
Peng J, Xu W, Liang B et al (2018) Pose measurement and motion estimation of space non-cooperative targets based on laser radar and stereo-vision fusion. IEEE Sens J 19(8):3008–3019. https://doi.org/10.1109/JSEN.2018.2889469
Serikawa S, Lu H (2014) Underwater image dehazing using joint trilateral filter. Comput Electr Eng 40(1):41–50. https://doi.org/10.1016/j.compeleceng.2013.10.016
Shan M, Guo J, Gill E (2016) Review and comparison of active space debris capturing and removal methods. Prog Aerosp Sci 80:18–32. https://doi.org/10.1016/j.paerosci.2015.11.001
Sharma S, D’Amico S (2016) Comparative assessment of techniques for initial pose estimation using monocular vision. Acta Astronaut 123:435–445. https://doi.org/10.1016/j.actaastro.2015.12.032
Sharma S, Beierle C, D'Amico S (2018) Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks. In: 2018 IEEE aerospace conference. IEEE, pp 1–12. https://doi.org/10.1109/AERO.2018.8396425
Simonyan K, Zisserman A (2014) A very deep convolutional networks for large-scale image recognition. Comput Sci
Song J, Cao C (2015) Pose self-measurement of noncooperative spacecraft based on solar panel triangle structure. J Robot. https://doi.org/10.1155/2015/472461
Thienel J, Van Eepoel J, Sanner R (2006) Accurate state estimation and tracking of a non-cooperative target vehicle. In: AIAA guidance, navigation, and control conference and exhibit, p 6802. https://doi.org/10.2514/6.2006-6802
Tweddle BE, Saenz-Otero A, Leonard J et al (2015) Factor graph modeling of rigid-body dynamics for localization, mapping, and parameter estimation of a spinning object in space. J Field Robot 32(6):897–933. https://doi.org/10.1002/rob.21548
Volpe R, Palmerini GB, Sabatini M (2018) A passive camera based determination of a non-cooperative and unknown satellite’s pose and shape. Acta Astronaut 151:805–817. https://doi.org/10.1016/j.actaastro.2018.06.061
Wang K, Liu H, Guo B et al (2016) A 6D-ICP approach for 3D reconstruction and motion estimate of unknown and non-cooperative target. In: 2016 Chinese control and decision conference (CCDC). IEEE, pp 6366–6370. https://doi.org/10.1109/CCDC.2016.7532144
Wei SE, Ramakrishna V, Kanade T et al (2016) Convolutional pose machines. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4724-4732. https://doi.org/10.1109/CVPR.2016.511
Woods JO, Christian JA (2016) Lidar-based relative navigation with respect to non-cooperative objects. Acta Astronaut 136:298–311. https://doi.org/10.1016/j.actaastro.2016.05.007
Xiong J, Gao H, Wang M, Li H, Lin W (2021) Occupancy map guided fast video-based dynamic point cloud coding. IEEE Trans Circ Syst Video Technol. https://doi.org/10.1109/tcsvt.2021.3063501
Yang L, Zongwu X, Hong L (2020) Three-line structured light vision system for non-cooperative satellites in proximity operations. Chin J Aeronaut 33(5):1494–1504. https://doi.org/10.1016/j.cja.2019.08.024
Yin F, Chou W, Wu Y et al (2020) Relative pose determination of uncooperative known target based on extracting region of interest. Meas Control 53(3–4):589–600. https://doi.org/10.1177/0020294019858105
Yingying G, Li W (2020) Fast-swirl space non-cooperative target spin state measurements based on a monocular camera. Acta Astronaut 166:156–161. https://doi.org/10.1016/j.actaastro.2019.08.011
Yu C, Wang J, Peng C et al (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 325–341
Zhang H, Wei Q, Jiang Z (2017) 3D reconstruction of space objects from multi-views by a visible sensor. Sensors 17(7):1689. https://doi.org/10.3390/s17071689
Zhang L, Wu DM, Ren Y (2019) Pose measurement for non-cooperative target based on visual information. IEEE Access 7:106179–106194. https://doi.org/10.1109/ACCESS.2019.2932835
Acknowledgements
This work was supported by the National Key Research and Development Program of China (Grant No. 2018AAA0102700, 2018AAA0102702), the Nature Program of the National Natural Science Foundation of China (Grant No. 61690210, 61690215), Science and Technology on Space Intelligent Control Laboratory (KGJZDSYS-2018-02), Beijing Major Science and Technology Projects (No. Z181100003118011), and Civil Space Technology Advanced Research Projects of China.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Du, H., Hu, H., Wang, D. et al. Autonomous measurement and semantic segmentation of non-cooperative targets with deep convolutional neural networks. J Ambient Intell Human Comput 14, 6959–6973 (2023). https://doi.org/10.1007/s12652-021-03553-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03553-7