Needle Tip Force Estimation Using an OCT Fiber and a Fused convGRU-CNN Architecture
Needle insertion is common during minimally invasive interventions such as biopsy or brachytherapy. During soft tissue needle insertion, forces acting at the needle tip cause tissue deformation and needle deflection. Accurate needle tip force measurement provides information on needle-tissue interaction and helps detecting and compensating potential misplacement. For this purpose we introduce an image-based needle tip force estimation method using an optical fiber imaging the deformation of an epoxy layer below the needle tip over time. For calibration and force estimation, we introduce a novel deep learning-based fused convolutional GRU-CNN model which effectively exploits the spatio-temporal data structure. The needle is easy to manufacture and our model achieves a mean absolute error of \(1.76\,\pm \,1.5\) mN with a cross-correlation coefficient of 0.9996, clearly outperforming other methods. We test needles with different materials to demonstrate that the approach can be adapted for different sensitivities and force ranges. Furthermore, we validate our approach in an ex-vivo prostate needle insertion scenario.
KeywordsForce estimation Optical coherence tomography Convolutional GRU Convolution Neural Network Needle placement
This work was partially supported by DFG grants SCHL 1844/2-1 and SCHL 1844/2-2.
- 3.Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: CVPR, pp. 2625–2634 (2015)Google Scholar
- 4.Hatzfeld, C., Wismath, S., Hessinger, M., Werthschtzky, R., Schlaefer, A., Kupnik, M.: A miniaturized sensor for needle tip force measurements. Biomed. Eng. 62(1), 109–115 (2017)Google Scholar
- 5.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
- 6.Kataoka, H., Washio, T., Chinzei, K., Mizuhara, K., Simone, C., Okamura, A.M.: Measurement of the tip and friction force acting on a needle during penetration. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2488, pp. 216–223. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45786-0_27CrossRefzbMATHGoogle Scholar
- 7.Kennedy, K.M., et al.: Quantitative micro-elastography: imaging of tissue elasticity using compression optical coherence elastography. Sci. Rep. 5(15), 538 (2015)Google Scholar
- 9.Mo, Z., Xu, W., Broderick, N.G.: Capability characterization via ex-vivo experiments of a fiber optical tip force sensing needle for tissue identification. IEEE Sens. J. 18, 1195–1202 (2017)Google Scholar
- 13.Sun, L., Jia, K., Yeung, D.Y., Shi, B.E.: Human action recognition using factorized spatio-temporal convolutional networks. In: CVPR, pp. 4597–4605 (2015)Google Scholar
- 15.Xingjian, S., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)Google Scholar