Multimedia Tools and Applications

, Volume 75, Issue 22, pp 14247–14261 | Cite as

Robot-assisted mirror ultrasound scanning for deep venous thrombosis detection using RGB-D sensor

  • Bo Meng
  • Ying-ying Zhao
  • Lei Chen
  • Fereshteh Aalamifar
  • Xue-jun Liu
  • Emad Boctor


Deep Venous Thrombosis (DVT) is a major cause of morbidity and mortality. Manually scanning with Ultrasound (US) probe brings heavy work load for the sonographers. This paper proposes a novel “Mirror robotic US scanning system. The system is composed of two-arm robot, linear US probes for master and slave side, Kinect sensor as a vision servo. On the master side, sonographers hold one robot arm and operate probe to inspect one leg. On the slave side, the robot follows the master probe on the navigation of Kinect and scans the other leg. 3D images of legs are segmented and register to get mirror matrix. Both Clustered Viewpoint Feature Histogram (CVFH) descriptors of segmented probe and CAD (Computer-Aided Design) training data were calculated to probe recognition. The leg phantom platform was built up. The mirror matrixes were obtained. The correlation coefficients between the two legs are calculated. Times of ICP (Iterative Closest Point) have been calculated for the platform. Results from the initial experiment indicate the idea is feasible and promising greatly by improving the inspection efficiently. Clinically, the method can be implemented for pre-operative procedures to predict the risk of DVT, and this may improve the US scanning efficacy.


RGB-D sensor Vision servo Robot Camera calibration CAD model ICP registraion 


  1. 1.
    “DVT overview”, WebMD, available online: (accessed at 2014-12-17)
  2. 2.
    Aalamifar F, Khurana R, Cheng A et al. (2014) Enabling technologies for robot assisted ultrasound tomography: system setup and calibration[C]//SPIE medical imaging. Int Soc Optics Photo 90401X–90401X-9Google Scholar
  3. 3.
    Bauer S, Wasza J, Haase S et al. (2011) “Multi-modal suface registration for markerless initial patient setup in radiation therapy using Microsoft’s Kinect sensor”. IEEE Int Conf Comput Vision Workshops (ICCV), Barcelona, Spain 1175–1181Google Scholar
  4. 4.
    Benavidez P, Jamshidi M (2011) “Mobile robot navigation and target tracking system”. Proc 6th Int Conf Syst Syst Eng: SoSE Cloud Comput, Smart Grid, Cyber Secur, SoSE 2011, Albuquerque, NM, USA 299–304Google Scholar
  5. 5.
    Besl PJ, MacKay N (1992) A method for registration of 3-d shapes”. IEEE Trans Pattern Anal Mach Intell 14(2):239–256CrossRefGoogle Scholar
  6. 6.
    Boctor EM, Fischer G, Choti MA et al (2004) A dual-armed robotic system for intraoperative ultrasound guided hepatic ablative therapy: a prospective study[C]//robotics and automation, 2004. Proc ICRA'04 2004 IEEE Int Conf IEEE 3:2517–2522Google Scholar
  7. 7.
    Burgin W, Pantofaru C, Smart WD et al. (2011) “Using depth information to improve face detection”. Proc 6th Int Conf Human-Robot Interact, Lausanne, Switzerland 6–9Google Scholar
  8. 8.
    De Martino RR, Beck AW, Edwards MW (2012) Impact of screening versus symptomatic measurement of deep vein thrombosis in a national quality improvement registry”. J Vasc Surg 10:1045–1051CrossRefGoogle Scholar
  9. 9.
    Drost B, Ulrich M, Navab N (2010) ” Model globally, match locally: efficient and robust 3D object recognition”. IEEE Conf Comput Vision Pattern Recognition(CVPR), 2010, San Francisco, CA, USA 998–1005Google Scholar
  10. 10.
    Ensor J, Riley RD, Moore D et al (2013) Protocol for a systematic review of prognostic models for the recurrence of venous thromboembolism (VTE) following treatment for a first unprovoked VTE[J]. Syst Rev 2(91):10.1186Google Scholar
  11. 11.
    Gornik HL, Sharma AM (2014) Duplex ultrasound in the diagnosis of lower-extremity deep venous thrombosis”. Circulat, J Am Heart Assoc 129:917–921Google Scholar
  12. 12.
    Gupta S, Arbeláez P, Girshick R et al. Inferring 3D object pose in RGB-D images[J]. arXiv prGoogle Scholar
  13. 13.
    Izadi S, Kim D, Hilliges O et al. (2011) “KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera”. UIST’11 Proc 24th Ann ACM Symp User Interface Software Technol, ACM New York, NY, USA 559–568Google Scholar
  14. 14.
    Krupa A, Folio D, Novales C et al (2014) Robotized tele-echography: an assisting visibility tool to support expert diagnostic[J]. IEEE Syst J 99:1–10Google Scholar
  15. 15.
    Microsoft. Kinect for Windows. Available noline: (accessed at 2014-12-17)Google Scholar
  16. 16.
    Motohashi M, Adachi A, Takigami K et al (2012) Deep vein thrombosis in orthopedic surgery of the lower extremities[J]. Ann Vasc Dis 5(3):328CrossRefGoogle Scholar
  17. 17.
    Mylonas GP, Giataganas P, Chaudery M et al. (2013) Autonomous eFAST ultrasound scanning by a robotic manipulator using learning from demonstrations[C]//Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on. IEEE 3251–3256Google Scholar
  18. 18.
    Nadvoretskiy V, Ermilov S, Brecht HP et al. (2011) Image processing and analysis in a dual-modality optoacoustic/ultrasonic system for breast cancer diagnosis[C]//SPIE BiOS. Int Soc Optics Photon 789909–789909-6Google Scholar
  19. 19.
    Patel AP, Koltz MT, Sansur CA (2013) “An analysis of deep vein thrombosis in 1277 consecutive neurosurgical patients undergoing routine weekly ultrasonography”. J Neurosurg 118:505–509CrossRefGoogle Scholar
  20. 20.
    Shen J, Cheung S (2013) Layer depth denoising and completion for structured-light rgb-d cameras[C]//Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE 1187–1194Google Scholar
  21. 21.
    Talbot S (1982) Use of real-time imaging in identifying deep venous obstruction: a preliminary report”. Bruit 7:41–42Google Scholar
  22. 22.
    Vanderpool HE, Friis EA, Smith BS, Harms KL (1993) Prevalence of carpal tunnel syndrome and other work-related musculoskeletal problems in cardiac sonographers”. JOEM: J Occupation Environ Med 35:97–113Google Scholar
  23. 23.
    Wen J, Zhu X, Li D (2013) Improved bounds on restricted isometry constant for orthogonal matching pursuit[J]. Electron Lett 49(23):1487–1489CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Bo Meng
    • 1
  • Ying-ying Zhao
    • 1
  • Lei Chen
    • 2
    • 4
  • Fereshteh Aalamifar
    • 3
  • Xue-jun Liu
    • 1
  • Emad Boctor
    • 3
    • 4
  1. 1.Northeast Dianli UniversityJilinChina
  2. 2.Engineering Research Center, Johns Hopkins UniversityBaltimoreUSA
  3. 3.Department of Electrical & Computer Engineering Johns Hopkins UniversityBaltimoreUSA
  4. 4.Department of Radiology and Radiological Science, Johns Hopkins Medical InstitutesBaltimoreUSA

Personalised recommendations