Abstract
Venipuncture robots have superior perception and stability to humans and are expected to replace manual venipuncture. However, their use is greatly restricted because they cannot make decisions regarding the puncture sites. Thus, this study presents a multi-information fusion method for determining puncture sites for venipuncture robots to improve their autonomy in the case of limited resources. Here, numerous images have been gathered and processed to establish an image dataset of human forearms for training the U-Net with the soft attention mechanism (SAU-Net) for vein segmentation. Then, the veins are segmented from the images, feature information is extracted based on near-infrared vision, and a multiobjective optimization model for puncture site decision is provided by considering the depth, diameter, curvature, and length of the vein to determine the optimal puncture site. Experiments demonstrate that the method achieves a segmentation accuracy of 91.2% and a vein extraction rate of 86.7% while achieving the Pareto solution set (average time: 1.458 s) and optimal results for each vessel. Finally, a near-infrared camera is applied to the venipuncture robot to segment veins and determine puncture sites in real time, with the results transmitted back to the robot for an attitude adjustment. Consequently, this method can enhance the autonomy of venipuncture robots if implemented dramatically.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Horattas M C, Trupiano J, Hopkins S, et al. Changing concepts in long-term central venous access: Catheter selection and cost savings. Am J Infect Control, 2001, 29: 32–40
Sampalis J S, Lavoie A, Williams J I, et al. Impact of on-site care, prehospital time, and level of in-hospital care on survival in severely injured patients. J Trauma-Injury Infect Crit Care, 1993, 34: 252–261
Armenteros-Yeguas V, Gárate-Echenique L, Tomás-López M A, et al. Prevalence of difficult venous access and associated risk factors in highly complex hospitalised patients. J Clin Nurs, 2017, 26: 4267–4275
Lamperti M, Pittiruti M. Difficult peripheral veins: Turn on the lights. Br J Anaesthesia, 2013, 110: 888–891
Hulse E J, Thomas G O R. Vascular access on the 21st century military battlefield. J R Army Med Corps, 2010, 156: S385–390
Chen A I, Balter M L, Maguire T J, et al. Deep learning robotic guidance for autonomous vascular access. Nat Mach Intell, 2020, 2: 104–115
Chen A I, Balter M L, Maguire T J, et al. Real-time needle steering in response to rolling vein deformation by a 9-DOF image-guided autonomous venipuncture robot. In: Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE, 2015. 2633–2638
Chen A, Nikitczuk K, Nikitczuk J, et al. Portable robot for autonomous venipuncture using 3D near infrared image guidance. Technology, 2013, 1: 72–87
Balter M L, Chen A I, Maguire T J, et al. Adaptive kinematic control of a robotic venipuncture device based on stereo vision, ultrasound, and force guidance. IEEE Trans Ind Electron, 2017, 64: 1626–1635
Chen A I. Image-guided robotics for autonomous venipuncture. Dissertation for the Doctoral Degree. New Brunswick: Rutgers University. 2016. 25–28
Wang D H, Feng G L, Wang X, et al. Research on image segmentation algorithm based on features of venous gray value. Opto Electron Eng, 2018, 45: 180066
Ji J, Zhao Y, Xie T, et al. Automated vein segmentation from NIR images using a mixer-UNet model. In: Liu H, Yin Z, Liu L, et al., eds. Intelligent Robotics and Applications. Cham: Springer International Publishing, 2022. 64–75
Lin G, Wang H, Sha M, et al. Design of a Multi-data fusion intelligent venipuncture blood sampling robot. In: Proceedings of 2022 7th International Conference on Control and Robotics Engineering (ICCRE). Beijing, China: IEEE, 2022. 10–15
Sha M, Wang H, Lin G, et al. Design of multi-sensor vein data fusion blood sampling robot based on deep learning. In: Proceedings of 2022 2nd International Conference on Computer, Control and Robotics (ICCCR). Shanghai, China: IEEE, 2022. 46–51
Zivanovic A, Davies B L. A robotic system for blood sampling. IEEE Trans Inform Technol Biomed, 2000, 4: 8–14
Cheng Z, Davies B L, Caldwell D G, et al. A hand-held robotic device for peripheral intravenous catheterization. Proc Inst Mech Eng H, 2017, 231: 1165–1177
Kobayashi Y, Hamano R, Watanabe H, et al. Use of puncture force measurement to investigate the conditions of blood vessel needle insertion. Med Eng Phys, 2013, 35: 684–689
Zhuang Y, Chen J, Liu Q, et al. Preliminary study on mechanical characteristics of maxillofacial soft and hard tissues for virtual surgery. Int J CARS, 2021, 16: 151–160
Zhao Y, Ji J, Xie T, et al. Vessel site selection for autonomous cannulation under NIR image guidance. In: Liu H, Yin Z, Liu L, et al., eds. Intelligent Robotics and Applications. Cham: Springer International Publishing, 2022. 88–99
He T, Guo C, Jiang L, et al. Automatic venous segmentation in venipuncture robot using deep learning. In: Proceedings of 2021 IEEE International Conference on Real-Time Computing and Robotics (RCAR). Xining, China: IEEE, 2021. 614–619
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015. 3431–3440
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. arXiv: 150504597
Valipour S, Siam M, Jagersand M, et al. Recurrent fully convolutional networks for video segmentation. In: Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Santa Rosa: IEEE, 2017. 29–36
Jain H, Deb K. An evolutionary many-objective optimization algorithm using reference-point based nondominated sorting approach, Part II: Handling constraints and extending to an adaptive approach. IEEE Trans Evol Computat, 2014, 18: 602–622
Deb K, Jain H. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: Solving problems with box constraints. IEEE Trans Evol Computat, 2014, 18: 577–601
Deb K, Pratap A, Agarwal S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Computat, 2002, 6: 182–197
Li W L, Xie H, Zhang G, et al. Hand—eye calibration in visually-guided robot grinding. IEEE Trans Cybern, 2016, 46: 2634–2642
Wang G, Li W, Jiang C, et al. Simultaneous calibration of multi-coordinates for a dual-robot system by solving the AXB = YCZ problem. IEEE Trans Robot, 2021, 37: 1172–1185
Fu H, Xu Y, Wong D W K, et al. Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. In: Proceedings of 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). Prague: IEEE, 2016. 698–701
Al-Bander B, Williams B, Al-Nuaimy W, et al. Dense fully convolutional segmentation of the optic disc and cup in colour fundus for glaucoma diagnosis. Symmetry, 2018, 10: 87
Balter M L, Chen A I, Fromholtz A, et al. System design and development of a robotic device for automated venipuncture and diagnostic blood cell analysis. In: Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea: IEEE, 2016. 514–520
Balter M L, Chen A I, Maguire T J, et al. The system design and evaluation of a 7-DOF image-guided venipuncture robot. IEEE Trans Robot, 2015, 31: 1044–1053
Qiao Z, Li Y, Wu Z, et al. Automatic puncture system based on NIR image and ultrasonic image. In: Proceedings of International Conference on Mechanical, Aeronautical and Automotive Engineering (ICMAA). Malacca, 2017, 108: 15002
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (Grant No. U1813209) and Self-Planned Task of State Key Laboratory of Robotics and System (Harbin Institute of Technology) (Grant No. SKLRS202112B).
Rights and permissions
About this article
Cite this article
He, T., Guo, C. & Jiang, L. Puncture site decision method for venipuncture robot based on near-infrared vision and multiobjective optimization. Sci. China Technol. Sci. 66, 13–23 (2023). https://doi.org/10.1007/s11431-022-2232-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11431-022-2232-5