Medical & Biological Engineering & Computing

, Volume 55, Issue 8, pp 1401–1414 | Cite as

A data-driven soft sensor for needle deflection in heterogeneous tissue using just-in-time modelling

  • Carlos RossaEmail author
  • Thomas Lehmann
  • Ronald Sloboda
  • Nawaid Usmani
  • Mahdi Tavakoli
Original Article


Global modelling has traditionally been the approach taken to estimate needle deflection in soft tissue. In this paper, we propose a new method based on local data-driven modelling of needle deflection. External measurement of needle–tissue interactions is collected from several insertions in ex vivo tissue to form a cloud of data. Inputs to the system are the needle insertion depth, axial rotations, and the forces and torques measured at the needle base by a force sensor. When a new insertion is performed, the just-in-time learning method estimates the model outputs given the current inputs to the needle–tissue system and the historical database. The query is compared to every observation in the database and is given weights according to some similarity criteria. Only a subset of historical data that is most relevant to the query is selected and a local linear model is fit to the selected points to estimate the query output. The model outputs the 3D deflection of the needle tip and the needle insertion force. The proposed approach is validated in ex vivo multilayered biological tissue in different needle insertion scenarios. Experimental results in five different case studies indicate an accuracy in predicting needle deflection of 0.81 and 1.24 mm in the horizontal and vertical lanes, respectively, and an accuracy of 0.5 N in predicting the needle insertion force over 216 needle insertions.


Surgical needles Needle steering Data-driven Just-in-time modelling Soft sensor 



This work was supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada under Grant CHRP 446520, the Canadian Institutes of Health Research (CIHR) under Grant CPG 127768, and Alberta Innovates—Health Solutions (AIHS) under Grant CRIO 201201232.


  1. 1.
    Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431CrossRefPubMedGoogle Scholar
  2. 2.
    Atkeson CG, Moore AW, Schaal S (1997) Locally weighted learning for control in Lazy learning. Springer, Berlin, pp 75–113CrossRefGoogle Scholar
  3. 3.
    Azar F, Metaxas DN, Schnall MD (2001) A deformable finite element model of the breast for predicting mechanical deformations under external perturbations. Acad Radiol 8(10):965–975CrossRefPubMedGoogle Scholar
  4. 4.
    Bengio Y, Grandvalet Y (2004) No unbiased estimator of the variance of k-fold cross-validation. J Mach Learn Res 5:1089–1105Google Scholar
  5. 5.
    Cheng C, Chiu M-S (2004) A new data-based methodology for nonlinear process modeling. Chem Eng Sci 59(13):2801–2810CrossRefGoogle Scholar
  6. 6.
    DiMaio S, Salcudean S (2003) Needle insertion modeling and simulation. IEEE Trans Robot Autom 19(5):864–875CrossRefGoogle Scholar
  7. 7.
    Dobler B, Mai S, Ross C, Wolff D, Wertz H, Lohr F, Wenz F (2006) Evaluation of possible prostate displacement induced by pressure applied during transabdominal ultrasound image acquisition. Strahlenther Onkol 182(4):240–246CrossRefPubMedGoogle Scholar
  8. 8.
    Eberhart RC, Kennedy J, et al. (1995) A new optimizer using particle swarm theory. In proceedings of the sixth international symposium on micro machine and human science, vol. 1. New York, NY, 1995, pp 39–43Google Scholar
  9. 9.
    Goksel O, Salcudean S, Dimaio SP (2006) 3D simulation of needle-tissue interaction with application to prostate brachytherapy. Comput Aided Surg 11(6):279–288CrossRefPubMedGoogle Scholar
  10. 10.
    Khadem M, Rossa C, Usmani N, Sloboda RS, Tavakoli M (2016) A two-body rigid/flexible model of needle steering dynamics in soft tissue. IEEE/ASME Trans Mechatron 21(5):2352–2364CrossRefGoogle Scholar
  11. 11.
    Khadem M, Rossa C, Sloboda R, Usmani N, Tavakoli M (2016) Ultrasound-guided model predictive control of needle steering in biological tissue. J Med Robot Res 01(01):1640007CrossRefGoogle Scholar
  12. 12.
    Khadem M, Rossa C, Sloboda RS, Usmani N, Tavakoli M (2016) Mechanics of tissue cutting during needle insertion in biological tissue. IEEE Robot Autom Lett 1(2):800–807CrossRefGoogle Scholar
  13. 13.
    Lee H, Kim J (2014) Estimation of flexible needle deflection in layered soft tissues with different elastic moduli. Med Biol Eng Comput 52(9):729–740CrossRefPubMedGoogle Scholar
  14. 14.
    Lehmann T, Rossa C, Usmani N, Sloboda R, Tavakoli M (2016) A real-time estimator for needle deflection during insertion into soft tissue based on adaptive modeling of needle-tissue interactions. IEEE/ASME Trans Mechatron (in press)Google Scholar
  15. 15.
    Moreira P, Misra S (2015) Biomechanics-based curvature estimation for ultrasound-guided flexible needle steering in biological tissues. Ann Biomed Eng 43(8):1716–1726CrossRefPubMedGoogle Scholar
  16. 16.
    Okamura A, Simone C, Leary M (2004) Force modeling for needle insertion into soft tissue. IEEE Trans Biomed Eng 51(10):1707–1716CrossRefPubMedGoogle Scholar
  17. 17.
    Rosen J, Hannaford B, Richards CG, Sinanan MN (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48(5):579–591CrossRefPubMedGoogle Scholar
  18. 18.
    Rossa C, Khadem M, Sloboda R, Usmani N, Tavakoli M (2016) Constrained optimal control of needle deflection for semi-manual steering. In 2016 IEEE international conference on advanced intelligent mechatronics (AIM), July 2016, pp 1198–1203Google Scholar
  19. 19.
    Rossa C, Khadem M, Sloboda R, Usmani N, Tavakoli M (2016) Adaptive quasi-static modelling of needle deflection during steering in soft tissue. IEEE Robot Autom Lett 1(2):916–923CrossRefGoogle Scholar
  20. 20.
    Rossa C, Usmani N, Sloboda R, Tavakoli M (2016) A hand-held assistant for semi-automated percutaneous needle steering. IEEE Trans Biomed Eng (in press)Google Scholar
  21. 21.
    Rossa C, Sloboda R, Usmani N, Tavakoli M (2016) Estimating needle tip deflection in biological tissue from a single transverse ultrasound image: application to brachytherapy. Int J Comput Assist Radiol Surg 11(7):1347–1359CrossRefPubMedGoogle Scholar
  22. 22.
    Shao J (1993) Linear model selection by cross-validation. J Am Stat Assoc 88(422):486–494CrossRefGoogle Scholar
  23. 23.
    Singhal A, Seborg DE (2002) Pattern matching in multivariate time series databases using a moving-window approach. Ind Eng Chem Res 41(16):3822–3838CrossRefGoogle Scholar
  24. 24.
    Vidal FP, John NW, Healey AE, Gould DA (2008) Simulation of ultrasound guided needle puncture using patient specific data with 3D textures and volume haptics. Comput Animat Virtual Worlds 19(2):111–127CrossRefGoogle Scholar
  25. 25.
    Waine M, Rossa C, Sloboda R, Usmani N, Tavakoli M (2015) 3D needle shape estimation in TRUS-guided prostate brachytherapy using 2D ultrasound images. IEEE J Biomed Health Inform PP(99):1–1Google Scholar
  26. 26.
    Waine M, Rossa C, Sloboda R, Usmani N, Tavakoli M (2016) Needle tracking and deflection prediction for robot-assisted needle insertion using 2D ultrasound images. J Med Robot Res 01(01):1640001CrossRefGoogle Scholar
  27. 27.
    Wang H, Yuan J (2015) Collaborative multifeature fusion for transductive spectral learning. IEEE Trans Cybern 45(3):451–461CrossRefGoogle Scholar
  28. 28.
    Webster R, Kim JS, Cowan NJ, Chirikjian GS, Okamura AM (2006) Nonholonomic modeling of needle steering. Int J Robot Res 25(5–6):509–525CrossRefGoogle Scholar
  29. 29.
    Yoon S, MacGregor JF (2001) Fault diagnosis with multivariate statistical models part I: using steady state fault signatures. J Process Control 11(4):387–400CrossRefGoogle Scholar

Copyright information

© International Federation for Medical and Biological Engineering 2016

Authors and Affiliations

  • Carlos Rossa
    • 1
    Email author
  • Thomas Lehmann
    • 1
  • Ronald Sloboda
    • 2
  • Nawaid Usmani
    • 2
  • Mahdi Tavakoli
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of AlbertaEdmontonCanada
  2. 2.Cross Cancer Institute and the Department of OncologyUniversity of AlbertaEdmontonCanada

Personalised recommendations