Skip to main content

Advertisement

SpringerLink
Log in

A Hybrid RF and Vision Aware Fusion Scheme for Multi-Sensor Wireless Capsule Endoscopic Localization

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Localization Wireless Capsule Endoscopy (WCE) inside the human small intestine is a hard issue for a decade. This is due to long, curly, and compact structure small intestine. Some of the techniques as Radio Frequency (RF), Vision based and Magnetic type have been proposed. To be more important, any one of the techniques as RF, Vision or Magnetic shows that the poor performance in terms of localization error and accuracy. To address these issues, in this paper a hybrid RF with Vision aware Fusion scheme (RF-VaF) is proposed under multisensor. In RF based approach, Time of Flight and Received Signal Strength Indicator are presented. In vision based approach, Siamese CapsNet is proposed for frames registration, correlation maps generation, and pixel based matching point’s prediction. A multi-feature extraction (color, edge, intensity and texture) is executed by Spatial Transformer Network for consecutive frames. In particular, this will be fed into the Siamese CapsNet. Similarly, Canberra distance is computed in the softmax layer for localization. The results from RF and Vision are fused into find the accurate position. In this step, hydrological cycle optimization algorithm is proposed. With this step, WCE can be accurately predicted at the end. One of the novel steps here is adjusting the Receiver’s Position by Positioning Metric. Finally, the performance is computed by using Matlab R2019b. From the results, it is proved that the RF-VaF is outperforms than the previous works by following metrics as Average Localization Error [5.41], Root Mean Square Error [6.76], Normalized Error [6.775], Localization Accuracy [96.43%], Localization Error [5.14%], Sensitivity [96.6%] and also Specificity [96.5%].

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Mateen, H., Basar, R., Ahmed, A. U., & Ahmad, M. Y. (2017). Localization of wireless capsule endoscope: A systematic review. IEEE Sensors Journal, 17(5), 1197–1206.

    Article  Google Scholar 

  2. Than, T. D., Alici, G., Zhou, H., Harvey, S., & Li, W. (2017). Enhanced Localization of Robotic Capsule Endoscopes Using Positron Emission Markers and Rigid-Body Transformation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1–15. https://doi.org/10.1109/tsmc.2017.2719050.

  3. Marya, N., Karellas, A., Foley, A., Roychowdhury, A., & Cave, D. (2014). Computerized 3-dimensional localization of a video capsule in the abdominal cavity: Validation by digital radiography. Gastrointestinal Endoscopy, 79(4), 669–674.

    Article  Google Scholar 

  4. Yang, Y. J. (2020). The future of capsule endoscopy: The role of artificial intelligence and other technical advancements. Clinical Endoscopy, 53, 387–394.

    Article  Google Scholar 

  5. Gomes, S., Valério, M. T., Salgado, M., Oliveira, H. P., & Cunha, A. (2019). Unsupervised neural network for homography estimation in capsule endoscopy frames. Procedia Computer Science, 164, 602–609.

    Article  Google Scholar 

  6. Karargyris, A., & Koulaouzidis, A. (2015). OdoCapsule: Next-generation wireless capsule endoscopy with accurate lesion localization and video stabilization capabilities. IEEE Transactions on Biomedical Engineering, 62(1), 352–360.

    Article  Google Scholar 

  7. Dey, N., Ashour, A. S., Shi, F., & Sherratt, R. S. (2017). Wireless capsule gastrointestinal endoscopy: Direction-of-arrival estimation based localization survey. IEEE Reviews in Biomedical Engineering, 10, 2–11.

    Article  Google Scholar 

  8. Khan, U., Ye, Y., Aisha, A.-U., Swar, P., & Pahlavan, K. (2018). Precision of EM simulation based wireless location estimation in multi-sensor capsule endoscopy. IEEE Journal of Translational Engineering in Health and Medicine, 6, 1–11.

    Article  Google Scholar 

  9. Dimas, G., Iakovidis, D. K., Ciuti, G., Karargyris, A., & Koulaouzidis, A. (2017). Visual localization of wireless capsule endoscopes aided by artificial neural networks. 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS).

  10. Hwang, S.-N., Kim, R., & Lim, H. (2014). Bayesian inference-based tracking for wireless capsule endoscopes. 2014 International Conference on Information and Communication Technology Convergence

  11. Yishuang Geng, & Pahlavan, K. (2015). On the accuracy of RF and image processing based hybrid localization for wireless capsule endoscopy. 2015 IEEE Wireless Communications and Networking Conference (WCNC).

  12. Barbi, M., Garcia-Pardo, C., Cardona, N., Nevarez, A., Pons, V., & Frasson, M. (2018). Impact of receivers location on the accuracy of capsule endoscope localization. 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications

  13. Garcia-Pardo, C., Barbi, M., Pérez-Simbor, S., & Cardona, N. (2020). UWB channel characterization for wireless capsule endoscopy localization. 2020 IEEE International Conference on Communications Workshops (ICC Workshops), 1-6

  14. Iakovidis, D. K., Spyrou, E., Diamantis, D., & Tsiompanidis, I. (2013). Capsule endoscope localization based on visual features. 13th IEEE International Conference on BioInformatics and BioEngineering.

  15. Dimas, G., Iakovidis, D. K., Karargyris, A., Ciuti, G., & Koulaouzidis, A. (2017). An artificial neural network architecture for non-parametric visual odometry in wireless capsule endoscopy. Measurement Science and Technology, 28(9), 094005.

    Article  Google Scholar 

  16. Guanqun Bao, Liang Mi, Yishuang Geng, Mingda Zhou, & Pahlavan, K. (2014). A video-based speed estimation technique for localizing the wireless capsule endoscope inside gastrointestinal tract. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

  17. Hany, U., & Wahid, K. A. (2015). An Adaptive Linearized Method for Localizing Video Endoscopic Capsule Using Weighted Centroid Algorithm. International Journal of Distributed Sensor Networks, 1–18. https://doi.org/10.1155/2015/342428.

  18. Shao, G., Tang, Y., Tang, L., Dai, Q., & Guo, Y. (2019). A novel passive magnetic localization wearable system for wireless capsule endoscopy. IEEE Sensors Journal, 19, 3462–3472.

    Article  Google Scholar 

  19. Suveren, M., & Kanaan, M. (2019). 5D magnetic localization for wireless capsule endoscopy using the Levenberg-Marquardt Method and Artificial Bee Colony Algorithm. 2019 IEEE 30th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC Workshops), 1–6.

  20. Wang, J., Wang, Z., Leach, M., Lee, S., Lim, E. G., & Huang, Y. (2016). RF characteristics of wireless capsule endoscopy in human body. Journal of Central South University, 23(5), 1198–1207.

    Article  Google Scholar 

  21. Ali, H., Sharif, M., Yasmin, M., Rehmani, M. H., & Riaz, F. (2019). A survey of feature extraction and fusion of deep learning for detection of abnormalities in video endoscopy of gastrointestinal-tract. Artificial Intelligence Review. https://doi.org/10.1007/s10462-019-09743-2.

  22. Soffer, S., Klang, E., Shimon, O., Nachmias, N., Eliakim, R., Ben-Horin, S., Kopylov, U., & Barash, Y. (2020). Deep learning for wireless capsule endoscopy: A systematic review and meta-analysis. Gastrointestinal Endoscopy. https://doi.org/10.1016/j.gie.2020.04.039

    Article  Google Scholar 

  23. Xu, Y., & Meng, M. (2018). Free sensor array based relative localization system for wireless capsule endoscopy. IEEE International Conference on Robotics and Biomimetics (ROBIO), 2018, 550–555.

    Article  Google Scholar 

  24. Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E., & Sitti, M. (2017). Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based vi-sual odometry approach for endoscopic capsule robots. Neurocomputing. https://doi.org/10.1016/j.neucom.2017.10.014

    Article  Google Scholar 

  25. Kundu, A. K., & Fattah, S. A. (2019). Probability density function based modeling of spatial feature variation in capsule endoscopy data for automatic bleeding detection. Computers in Biology and Medicine. https://doi.org/10.1016/j.compbiomed.2019.103478

    Article  Google Scholar 

  26. Barbi, M., Pérez-Simbor, S., Garcia-Pardo, C., & Cardona, N. (2019). Analysis of the localization error for capsule endoscopy applications at UWB frequencies. 2019 13th International Symposium on Medical Information and Communication Technology (ISMICT), 1–6.

  27. Ara, P., Yu, K., Cheng, S., Dutkiewicz, E., & Heimlich, M. C. (2018). Human abdomen path-loss modeling and location estimation of wireless capsule endoscope using round-trip propagation loss. IEEE Sensors Journal, 18(8), 3266–3277.

    Article  Google Scholar 

  28. Hany, U., & Akter, L. (2017). Local parametric approach of wireless capsule endoscope localization using randomly scattered path loss based WCL. Wireless Communications and Mobile Computing, 2017, 1–17.

    Article  Google Scholar 

  29. Ara, P., Yu, K., Cheng, S., Dutkiewicz, E., & Heimlich, M. C. (2016). Derivation of CRLB for wireless capsule endoscope localization using received signal strength. IEEE Sensors Journal, 16(24), 9064–9074.

    Article  Google Scholar 

  30. Barbi, M., Garcia-Pardo, C., Nevarez, A., Pons, V., & Cardona, N. (2019). UWB RSS-based localization for capsule endoscopy using a multilayer phantom and in vivo measurements. IEEE Transactions on Antennas and Propagation. https://doi.org/10.1109/TAP.2019.2916629

    Article  Google Scholar 

  31. Hany, U., Akter, L., & Hossain, M. F. (2017). Degree-based WCL for video endoscopic capsule localization. IEEE Sensors Journal, 17, 2904–2916.

    Article  Google Scholar 

  32. Iida, T., Anzai, D., & Wang, J. (2017). A three-dimensional em-based implant device localization method improved by genetic algorithm. International Journal of Wireless Information Networks, 24(2), 180–188.

    Article  Google Scholar 

  33. Jeong, S., Kang, J., Pahlavan, K., & Tarokh, V. (2017). Fundamental limits of TOA/DOA and inertial measurement unit-based wireless capsule endoscopy hybrid localization. International Journal of Wireless Information Networks, 24(2), 169–179.

    Article  Google Scholar 

  34. Kissi, C., Sarestoniemi, M., Kumpuniemi, T., Sonkki, M., Myllymaki, S., Srifi, M., & Pomalaza-raez, C. (2019). On-body Cavity-Backed Low-UWB Antenna for Capsule Localization. International Journal of Wireless Information Networks. https://doi.org/10.1007/s10776-019-00460-9.

  35. Hany, U., & Akter, L. (2017). Non-parametric approach of video capsule endoscope localization using suboptimal method of position bounded CWCL. IEEE Sensors Journal, 17(20), 6806–6815.

    Article  Google Scholar 

  36. Hany, U., & Akter, L. (2017). Non-parametric method of path loss estimation for endoscopic capsule localization. International Journal of Wireless Information Networks, 25(1), 44–56.

    Article  Google Scholar 

  37. Aghanouri, M., Ghaffari, A., & Dadashi Serej, N. (2018). Image Based High-Level Control System Design for Steering and Controlling of an Active Capsule Endoscope. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-018-0956-8.

  38. Dimas, G., Spyrou, E., Iakovidis, D. K., & Koulaouzidis, A. (2017). Intelligent visual localization of wireless capsule endoscopes enhanced by color information. Computers in Biology and Medicine, 89, 429–440.

    Article  Google Scholar 

  39. Pahlavan, K., Geng, Y., Cave, D. R., Bao, G., Mi, L., Agu, E., & Tarokh, V. (2015). A novel cyber physical system for 3-D imaging of the small intestine in vivo. IEEE Access, 3, 2730–2742.

    Article  Google Scholar 

  40. Umay, I., & Fidan, B. (2017). Adaptive wireless biomedical capsule tracking based on magnetic sensing. International Journal of Wireless Information Networks, 24(2), 189–199.

    Article  Google Scholar 

  41. Bao, G., Pahlavan, K., & Mi, L. (2015). Hybrid localization of microrobotic endoscopic capsule inside small intestine by data fusion of vision and RF sensors. IEEE Sensors Journal, 15(5), 2669–2678.

    Article  Google Scholar 

  42. Hany, U., & Akter, L. (2018). Non-parametric approach using ml estimated path loss bounded WCL for video capsule endoscope localization. IEEE Sensors Journal, 18(11), 4761–4769.

    Article  Google Scholar 

  43. Iakovidis, D. K., Dimas, G., Karargyris, A., Bianchi, F., Ciuti, G., & Koulaouzidis, A. (2018). Deep Endoscopic Visual Measurements. IEEE Journal of Biomedical and Health Informatics, 23(6), 2211–2219.

    Article  Google Scholar 

  44. Figueiredo, I. N., Leal, C., Pinto, L., Figueiredo, P. N., & Tsai, R. (2018). Hybrid multiscale affine and elastic image registration approach towards wireless capsule endoscope localization. Biomedical Signal Processing and Control, 39, 486–502.

    Article  Google Scholar 

  45. Geng, Y., & Pahlavan, K. (2016). Design, implementation, and fundamental limits of image and RF based wireless capsule endoscopy hybrid localization. IEEE Transactions on Mobile Computing, 15(8), 1951–1964.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Enago (www.enago.com) for the English language review.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Excel Engineering College, Komarapalayam, Tamil Nadu, 637303, India

    P. Narmatha

  2. Electrical and Electronics Engineering Department, K.S. Rangasamy College of Technology, Tiruchengode, Tamil Nadu, 637215, India

    Venkatesan Thangavel & D. Sri Vidhya

Authors
  1. P. Narmatha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Venkatesan Thangavel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Sri Vidhya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Narmatha.

Ethics declarations

Conflict of interest

No any conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Narmatha, P., Thangavel, V. & Vidhya, D.S. A Hybrid RF and Vision Aware Fusion Scheme for Multi-Sensor Wireless Capsule Endoscopic Localization. Wireless Pers Commun 123, 1593–1624 (2022). https://doi.org/10.1007/s11277-021-09205-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-09205-5

Keywords

Search

Navigation