Abstract
Purpose
The robot-assisted automated puncture system under ultrasound guidance can well improve the puncture accuracy in ablation surgery. The automated puncture system requires advanced definition of the puncture location, while the displacement of thoracic-abdominal tumors caused by respiratory motion makes it difficult for the system to locate the best puncture position. Predicting tumor motion is an effective way to help the automated puncture system output a more accurate puncture position.
Methods
In this paper, we propose a self-attention-based feature pyramid algorithm FPSANet for time-series forecasting, which can extract both linear and nonlinear dependencies of time series. Firstly, we use the temporal convolutional network as the backbone to extract different scale time-series features, and the self-attention module is followed to weigh more significant features to improve nonlinear prediction. Secondly, we use autoregressive models to perform linear prediction. Finally, we directly combine the above two kinds of predictions as the final prediction.
Results
FPSANet is trained and tested on our private datasets captured from clinical individuals, and we predict the target position after 50 ms, 150 ms, 300 ms and 400 ms. The result shows the evaluation criteria of the MAE is less than 1 mm at 50 ms and 150 ms, and less than 2 mm at 300 ms. Compared with the AR model, bidirectional LSTM and RVM, our method not only outperforms both models in accuracy (AR: ~ 7.7%; bidirectional LSTM: ~ 75.9%; RVM: ~ 76.5%) but is also more stable on different types of respiratory curves.
Conclusion
Respiratory motion in the liver in actual clinical practice vary widely from person to person, while sometimes having less distinct periodic patterns. Under these conditions, our algorithm has the advantage of excellent stability for prediction on various sequences, and its running time of performing single sequence prediction can meet clinical requirements.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Code availability
Private code repository.
References
Soni NJ, Franco-Sadud R, Schnobrich D, Dancel R, Tierney DM, Salame G, ... McHardy P (2016) Ultrasound guidance for lumbar puncture. Neurol Clin Pract 6(4):358–368
Chen S, Wang F, Lin Y, Shi Q, Wang Y (2021) Ultrasound-guided needle insertion robotic system for percutaneous puncture. Int J Comput Assist Radiol Surg 16(3):475–484
Ramrath L, Schlaefer A, Ernst F, Dieterich S, Schweikard A (2007) Prediction of respiratory motion with a multi-frequency based Extended Kalman Filter. In: Proceedings of the 21st international conference and exhibition on computer assisted radiology and surgery (CARS’07) (vol 21, pp 56–58)
Ernst F, Schlaefer A, Schweikard A (2007, October) Prediction of respiratory motion with wavelet-based multiscale autoregression. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, Heidelberg, pp 668–675
McCall KC, Jeraj R (2007) Dual-component model of respiratory motion based on the periodic autoregressive moving average (periodic ARMA) method. Phys Med Biol 52(12):3455
Ernst F, Schweikard A (2008) Prediction of respiratory motion using a modified Recursive Least Squares algorithm. CURAC 8(7):157–160
Widrow B, Hoff ME (1960) Adaptive switching circuits. Stanford Univ Ca Stanford Electronics Labs
Ernst F, Schweikard A (2008) Predicting respiratory motion signals for image-guided radiotherapy using multi-step linear methods (MULIN). Int J Comput Assist Radiol Surg 3(1):85–90
Ernst F, Schweikard A (2009) Forecasting respiratory motion with accurate online support vector regression (SVRpred). Int J Comput Assist Radiol Surg 4(5):439–447
Dürichen R, Wissel T, Schweikard A (2012, September) Prediction of respiratory motion using wavelet based support vector regression. In: 2012 IEEE international workshop on machine learning for signal processing. IEEE, pp. 1–6
Dürichen R, Wissel T, Ernst F, Schweikard A (2013, September) Respiratory motion compensation with relevance vector machines. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, Heidelberg, pp 108–115
Wang R, Liang X, Zhu X, Xie Y (2018) A feasibility of respiration prediction based on deep Bi-LSTM for real-time tumor tracking. IEEE Access 6:51262–51268
Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. http://arxiv.org/abs/1803.01271
Oord AVD, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, ... Kavukcuoglu K (2016) Wavenet: a generative model for raw audio. http://arxiv.org/abs/1609.03499
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, ... Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
Huang S, Wang D, Wu X, Tang A (2019, November) DSANet: dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2129–2132
Ernst F (2011) Compensating for quasi-periodic motion in robotic radiosurgery. Springer, Berlin
De Luca V, Banerjee J, Hallack A, Kondo S, Makhinya M, Nouri D, ... Tanner C (2018) Evaluation of 2D and 3D ultrasound tracking algorithms and impact on ultrasound‐guided liver radiotherapy margins. Med Phys 45(11):4986–5003
Shen C, He J, Huang Y, Wu J (2019, October) Discriminative correlation filter network for robust landmark tracking in ultrasound guided intervention. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 646–654
Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2021, February) Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of AAAI
Acknowledgements
Work supported by the Knowledge Innovation Program of Basic Research Projects of Shenzhen [JCY20200109142805928, JCYJ20160428182053361], in part by National Key R&D Program of China (2019YFC0119500).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
For this type of study formal consent is not required.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yao, C., He, J., Che, H. et al. Feature pyramid self-attention network for respiratory motion prediction in ultrasound image guided surgery. Int J CARS 17, 2349–2356 (2022). https://doi.org/10.1007/s11548-022-02697-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-022-02697-x