Kalman filter-based EM-optical sensor fusion for needle deflection estimation

  • Baichuan Jiang
  • Wenpeng Gao
  • Daniel Kacher
  • Erez Nevo
  • Barry Fetics
  • Thomas C. Lee
  • Jagadeesan Jayender
Original Article



In many clinical procedures such as cryoablation that involves needle insertion, accurate placement of the needle’s tip at the desired target is the major issue for optimizing the treatment and minimizing damage to the neighboring anatomy. However, due to the interaction force between the needle and tissue, considerable error in intraoperative tracking of the needle tip can be observed as needle deflects.


In this paper, measurements data from an optical sensor at the needle base and a magnetic resonance (MR) gradient field-driven electromagnetic (EM) sensor placed 10 cm from the needle tip are used within a model-integrated Kalman filter-based sensor fusion scheme. Bending model-based estimations and EM-based direct estimation are used as the measurement vectors in the Kalman filter, thus establishing an online estimation approach.


Static tip bending experiments show that the fusion method can reduce the mean error of the tip position estimation from 29.23 mm of the optical sensor-based approach to 3.15 mm of the fusion-based approach and from 39.96 to 6.90 mm, at the MRI isocenter and the MRI entrance, respectively.


This work established a novel sensor fusion scheme that incorporates model information, which enables real-time tracking of needle deflection with MRI compatibility, in a free-hand operating setup.


Sensor fusion Needle deflection estimation Kalman filter (KF) Surgical navigation 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Rossa C, Tavakoli M (2017) Issues in closed-loop needle steering. Control Eng Pract 62:55–69CrossRefGoogle Scholar
  2. 2.
    Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431CrossRefPubMedGoogle Scholar
  3. 3.
    Dupuy DE, Zagoria RJ, Akerley W, Mayo-Smith WW, Kavanagh PV, Safran H (2000) Percutaneous radiofrequency ablation of malignancies in the lung. Am J Roentgenol 174(1):57–59CrossRefGoogle Scholar
  4. 4.
    Park YL, Elayaperumal S, Daniel B, Ryu SC, Shin M, Savall J, Black RJ, Moslehi B, Cutkosky MR (2010) Real-time estimation of 3-D needle shape and deflection for MRI-guided interventions. IEEE/ASME Trans Mechatron 15(6):906–915Google Scholar
  5. 5.
    Bartynski WS, Grahovac SZ, Rothfus WE (2005) Incorrect needle position during lumbar epidural steroid administration: inaccuracy of loss of air pressure resistance and requirement of fluoroscopy and epidurography during needle insertion. Am J Neuroradiol 26(3):502–505PubMedGoogle Scholar
  6. 6.
    Mala T, Edwin B, Mathisen Ø, Tillung T, Fosse E, Bergan A, SØreide O, Gladhaug I (2004) Cryoablation of colorectal liver metastases: minimally invasive tumour control. Scand J Gastroenterol 39(6):571–578CrossRefPubMedGoogle Scholar
  7. 7.
    Charboneau JW, Reading CC, Welch TJ (1990) Ct and sonographically guided needle biopsy: current techniques and new innovations. AJR Am J Roentgenol 154(1):1–10CrossRefPubMedGoogle Scholar
  8. 8.
    Tesei M, Saccomandi P, Massaroni C, Quarta R, Carassiti M, Schena E, Setola R (2016) A cost-effective, non-invasive system for pressure monitoring during epidural needle insertion: Design, development and bench tests. In: IEEE 38th annual international conference of the engineering in medicine and biology society (EMBC), IEEE, pp 194–197Google Scholar
  9. 9.
    Gao W, Jiang B, Kacher DF, Fetics B, Nevo E, Lee TC, Jayender J (2017) Real-time probe tracking using EM-optical sensor for MRI-guided cryoablation. Int J Med Rob.
  10. 10.
    Dorileo E, Zemiti N, Poignet P (2015) Needle deflection prediction using adaptive slope model. In: International conference on advanced robotics (ICAR), IEEE, pp 60–65Google Scholar
  11. 11.
    Roesthuis RJ, Van Veen YR, Jahya A, Misra S (2011) Mechanics of needle-tissue interaction. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE, pp 2557–2563Google Scholar
  12. 12.
    Asadian A, Kermani MR, Patel RV (2011) An analytical model for deflection of flexible needles during needle insertion. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE, pp 2551–2556Google Scholar
  13. 13.
    Park W, Reed KB, Okamura AM, Chirikjian GS (2010) Estimation of model parameters for steerable needles. In: IEEE International conference on robotics and automation (ICRA), 2010, IEEE, pp 3703–3708Google Scholar
  14. 14.
    Taffoni F, Formica D, Saccomandi P, Pino GD, Schena E (2013) Optical fiber-based MR-compatible sensors for medical applications: an overview. Sensors 13(10):14105–14120CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Sadjadi H, Hashtrudi-Zaad K, Fichtinger G (2013) Fusion of electromagnetic trackers to improve needle deflection estimation: simulation study. IEEE Trans Biomed Eng 60(10):2706–2715CrossRefPubMedGoogle Scholar
  16. 16.
    Jiang B, Gao W, Kacher DF, Lee TC, Jayender J (2016) Kalman filter based data fusion for needle deflection estimation using optical-em sensor. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 457–464Google Scholar
  17. 17.
    Roth A, Nevo E (2015) Method and apparatus to estimate location and orientation of objects during magnetic resonance imaging. US Patent 9,037,213Google Scholar
  18. 18.
    Goksel O, Dehghan E, Salcudean SE (2009) Modeling and simulation of flexible needles. Med Eng Phys 31(9):1069–1078CrossRefPubMedGoogle Scholar
  19. 19.
    Wan G, Wei Z, Gardi L, Downey DB, Fenster A (2005) Brachytherapy needle deflection evaluation and correction. Med Phys 32(4):902–909CrossRefPubMedGoogle Scholar
  20. 20.
    Chui CK, Chen G (2008) Kalman filtering: with real-time applications. Springer, BerlinGoogle Scholar
  21. 21.
    Hall DL, Llinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85(1):6–23CrossRefGoogle Scholar
  22. 22.
    Yan KG, Podder T, Xiao D, Yu Y, Liu TI, Ling KV, Ng WS (2006) Online parameter estimation for surgical needle steering model. In: Medical image computing and computer-assisted intervention–MICCAI 2006, Springer, pp 321–329Google Scholar
  23. 23. Accessed 14 Mar 2016
  24. 24.
    Nevo E (2003) Method and apparatus to estimate location and orientation of objects during magnetic resonance imaging. US Patent 6,516,213Google Scholar
  25. 25.
    Yaniv Z (2015) Which pivot calibration? In: SPIE medical imaging, international society for optics and photonics, pp 941527–941527Google Scholar
  26. 26.
    Crouch JR, Schneider CM, Wainer J, Okamura AM (2005) A velocity-dependent model for needle insertion in soft tissue. In: Medical image computing and computer-assisted intervention–Miccai 2005, Springer, pp 624–632Google Scholar
  27. 27.
    Bar-Shalom Y, Li XR, Kirubarajan T (2004) Estimation with applications to tracking and navigation: theory algorithms and software. Wiley, HobokenGoogle Scholar
  28. 28.
    Cerveri P, Pedotti A, Ferrigno G (2003) Robust recovery of human motion from video using kalman filters and virtual humans. Hum Mov Sci 22(3):377–404CrossRefPubMedGoogle Scholar
  29. 29.
    Lagarias JC, Reeds JA, Wright MH, Wright PE (1998) Convergence properties of the nelder-mead simplex method in low dimensions. SIAM J Optim 9(1):112–147CrossRefGoogle Scholar
  30. 30.
    Khadem M, Rossa C, Sloboda RS, Usmani N, Tavakoli M (2016) Ultrasound-guided model predictive control of needle steering in biological tissue. J Med Robot Res 1(01):1640007CrossRefGoogle Scholar
  31. 31.
    Coleman TF, Li Y (1996) An interior trust region approach for nonlinear minimization subject to bounds. SIAM J Optim 6(2):418–445CrossRefGoogle Scholar
  32. 32.
    Birkfellner W, Watzinger F, Wanschitz F, Ewers R, Bergmann H (1998) Calibration of tracking systems in a surgical environment. IEEE Trans Med Imaging 17(5):737–742CrossRefPubMedGoogle Scholar
  33. 33.
    Jayender J, Lee TC, Ruan DT (2015) Real-time localization of parathyroid adenoma during parathyroidectomy. N Engl J Med 373(1):96–98CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  1. 1.School of Mechanical EngineeringTianjin UniversityTianjinChina
  2. 2.School of Life Science and TechnologyHarbin Institute of TechnologyHarbinChina
  3. 3.Robin Medical Inc.BaltimoreUSA
  4. 4.Department of Neuroradiology, Brigham and Women’s HospitalHarvard Medical SchoolBostonUSA
  5. 5.Department of Radiology, Brigham and Women’s HospitalHarvard Medical SchoolBostonUSA

Personalised recommendations