Multimedia Tools and Applications

, Volume 78, Issue 5, pp 5107–5135 | Cite as

A novel registration and super-resolution jointed paradigm for medical images under internet of thing environment

  • Yu Liu
  • Jie Yang
  • Jing Mi
  • Jingjing Yang
  • Xiao ZhangEmail author


This paper proposes a novel registration and super-resolution jointed paradigm for medical images under the Internet of thing environment. In the medical image processing, the matching issue is one catches wide attention with the domain of research. Image registration technique can be divided into similarity measure, optimization, geometric transformation, and interpolation, etc. As the first essential clue of our model, we propose the novel registration algorithm based on energy feature extraction. Generally, the matching energy function by the similarity measurement and a penalty constitution is called the external force and endogenic force separately. The matching is an external force and endogenic force mutual competition, eventually achieves the balanced process. Furtherly, we integrate the game analysis and area feature selection to achieve the better image super-resolution mode through the pretreatment of the image to change the initial value, so as to achieve the purpose of improving the performance. Besides the algorithm level innovation, we integrate the GPU and the IOT to construct the hardware based implementation of the proposed medical image processing system. The latency of registers to read and write data across a GPU’s entire storage system is minimal, it is private to each thread, and can only be accessed by its owning thread. For each thread, the local memory is also private and it is often used to deal with the problem of overflow register, reducing the buffer overflow caused by the entire application of a substantial decline in the possibility and shared memory is visible to all threads within the thread block. We then achieve the optimal integration of IOT and GPU. The experimental result proves the robustness of the method.


Medical image Image registration Image super-resolution Internet of things Data storage Algorithm efficiency Jointed paradigm 



The authors thank Prof. Xiao Zhang and Prof. Wei Peng for the valuable discussion and recommendation.

This project was supported partially by the Population Health Informatization in Hebei Province Engineering Technology Research Center, Medical Informatics in Hebei Universities Application Technology Research and Development Center, Hebei province department of science and technology project(15217747D), Hebei province department of health project(20160029), Zhangjiakou department of science and technology project (1421012B), and Youth Foundation of the Education Department of Hebei Province (QN2016190).


  1. 1.
    Alaei A, Roy PP, Pal U (2016) Logo and seal based administrative document image retrieval: a survey. Comput Sci RevGoogle Scholar
  2. 2.
    Albalawi U, Mohanty SP, Kougianos E (2016) Energy-Efficient Design of the Secure Better Portable Graphics Compression Architecture for Trusted Image Communication in the IoT. VLSI (ISVLSI), 2016 I.E. computer society annual symposium on. IEEEGoogle Scholar
  3. 3.
    Badshah G et al (2016) Watermarking of ultrasound medical images in teleradiology using compressed watermark. J Med Image 3(1):017001–017001Google Scholar
  4. 4.
    Batten CF et al (2001) Sharpness search algorithms for automatic focusing in the scanning electron microscope. Scanning 23(2):112–113Google Scholar
  5. 5.
    Bi C et al. (2015a) SAR image restoration and change detection based on game theory. Intelligent Computing and Internet of Things (ICIT), 2014 International conference on. IEEEGoogle Scholar
  6. 6.
    Bi C, Zhang Q, Bao R, Wang H (2015b) SAR image restoration and change detection based on game theory. In: Intelligent Computing and Internet of Things (ICIT), 2014 International Conference on. IEEE, p 55–58Google Scholar
  7. 7.
    Blunt MJ et al (2013) Pore-scale imaging and modelling. Adv Water Resour 51:197–216Google Scholar
  8. 8.
    Cao J et al (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102Google Scholar
  9. 9.
    Chae S et al. (2016) Personal Smart Space: IoT Based User Recognition and Device Control. 2016 I.E. tenth international conference on semantic computing (ICSC). IEEEGoogle Scholar
  10. 10.
    Chen H et al (2016a) Single image super resolution using local smoothness and nonlocal self-similarity priors. Signal Process Image Commun 43:68–81Google Scholar
  11. 11.
    Chen H, Liu X, Xu H, Wang C (2016b) A Cloud Service Broker Based on Dynamic Game Theory for Bilateral SLA Negotiation in Cloud EnvironmentGoogle Scholar
  12. 12.
    Cho C, Lee S (2016) Effective five directional partial derivatives-based image smoothing and a parallel structure design. IEEE Trans Image Process 25(4):1617–1625MathSciNetzbMATHGoogle Scholar
  13. 13.
    Cho O et al (2013) Can initial diagnostic PET-CT aid to localize tumor bed in breast cancer radiotherapy: feasibility study using deformable image registration. Radiat Oncol 8(1):1MathSciNetGoogle Scholar
  14. 14.
    Chuang TW, Chen CC, Chien B (2016) Image Sharing and Recovering Based on Chinese Remainder Theorem. Computer, Consumer and Control (IS3C), 2016 International symposium on. IEEEGoogle Scholar
  15. 15.
    Oro D et al. (2016) Work-efficient parallel non-maximum suppression for embedded GPU architectures. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEEGoogle Scholar
  16. 16.
    De Nigris D, Louis Collins D, Arbel T (2012) Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans Med Imaging 31(12):2343–2354Google Scholar
  17. 17.
    Debita G et al. (2016) Analysis and Characteristics of Automatic Reconfiguration Mechanisms in IoT Devices Network. Information Systems Architecture and Technology: Proceedings of 36th International Conference on Information Systems Architecture and Technology–ISAT 2015–Part II. Springer International PublishingGoogle Scholar
  18. 18.
    Dimitrovski I et al (2016) Improving bag-of-visual-words image retrieval with predictive clustering trees. Inf Sci 329:851–865Google Scholar
  19. 19.
    Do NH (2016) Parallel processing for adaptive optics optical coherence tomography (AO-OCT) image registration using GPU. DissGoogle Scholar
  20. 20.
    Dong W et al (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630MathSciNetzbMATHGoogle Scholar
  21. 21.
    Dong-Xiao Z et al. (2016) Image Super Resolution Using Expansion Move Algorithm. Quantitative Logic and Soft Computing 2016. Springer International Publishing. p 641–657Google Scholar
  22. 22.
    Elfes A (2013) Occupancy grids: A stochastic spatial representation for active robot perception. arXiv preprint arXiv:1304.1098Google Scholar
  23. 23.
    Esposito C, Ficco M, Palmieri F, Castiglione A (2015) Smart cloud storage service selection based on fuzzy logic, theory of evidence and game theoryGoogle Scholar
  24. 24.
    Gao K et al (2013) An efficient of coal and gangue recognition algorithm. Int J Signal Process Image Process Pattern Recogn 6(4):345–354Google Scholar
  25. 25.
    Gao Y et al. (2016) Image registration and segmentation in longitudinal MRI using temporal appearance modeling. International Symposium on Biomedical Imaging-ISBIGoogle Scholar
  26. 26.
    Garg, R, and Chaudhury S (2015) A Novel Approach for Image Super Resolution Using Kernel Methods. Pattern Recognition and Machine Intelligence: 6th International Conference, PReMI 2015, Warsaw, June 30–July 3, 2015, Proceedings, vol 9124. SpringerGoogle Scholar
  27. 27.
    Glowacka, D, Teh YW, Shawe-Taylor J (2016) Image Retrieval with a Bayesian Model of Relevance Feedback. arXiv preprint arXiv:1603.09522Google Scholar
  28. 28.
    Guler P et al (2016) Real-time multi-camera video analytics system on GPU. J Real-Time Image Proc 11(3):457–472Google Scholar
  29. 29.
    Hindia MN et al (2016) Enabling remote health-caring utilizing IoT concept over LTE-femtocell networks. PLoS One 11(5):e0155077Google Scholar
  30. 30.
    Hipwell JH et al (2016) A review of biomechanically informed breast image registration. Phys Med Biol 61(2):R1Google Scholar
  31. 31.
    Hong S, Jeong W-K (2016) A Multi-GPU Fast Iterative Method for Eikonal Equations Using On-the-fly Adaptive Domain Decomposition. Procedia Computer Science 80:190–200Google Scholar
  32. 32.
    Hu YL et al. (2015) A programming framework for implementing fault-tolerant mechanism in IoT applications. International Conference on Algorithms and Architectures for Parallel Processing. Springer International PublishingGoogle Scholar
  33. 33.
    Hu J, Wu X, Zhou J (2016) Single image super resolution of 3D MRI using local regression and intermodality priors. Eighth International Conference on Digital Image Processing (ICDIP 2016). International Society for Optics and PhotonicsGoogle Scholar
  34. 34.
    Huang X et al. (2016) Discriminative extreme learning machine to content-based image retrieval with relevance feedback. Intelligent Control and Automation (WCICA), 2016 12th World Congress on. IEEEGoogle Scholar
  35. 35.
    Javed, A (2016) IoT Patterns: Location Aware. Building Arduino Projects for the Internet of Things. Apress, p 195–211Google Scholar
  36. 36.
    Jurkovic I et al (2016) SU-FT-680: radiobiological analysis of the impact of daily patient deformation and setup variations through the use of the cone beam CT and deformable image registration in lung cancer IMRT. Med Phys 43(6):3620–3620Google Scholar
  37. 37.
    Kim, SR et al. (2016) Anti-reversible dynamic tamper detection scheme using distributed image steganography for IoT applications. J Supercomput: 1–20.Google Scholar
  38. 38.
    Kurugol S et al. (2015) Motion compensated abdominal diffusion weighted MRI by simultaneous image registration and model estimation (SIR-ME). International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International PublishingGoogle Scholar
  39. 39.
    Kwon Y et al (2015) Efficient learning of image super-resolution and compression artifact removal with semi-local Gaussian processes. IEEE Trans Pattern Anal Mach Intell 37(9):1792–1805Google Scholar
  40. 40.
    Labine A et al (2014) SU-EJ-87: lung deformable image registration using surface mesh deformation for dose distribution combination. Med Phys 41(6):175–175Google Scholar
  41. 41.
    Lee J-H et al (1995) Implementation of a passive automatic focusing algorithm for digital still camera. IEEE Trans Consum Electron 41(3):449–454Google Scholar
  42. 42.
    Li W, Zhu J, Li H, Wu Q, Zhang L (2015) A Game Theory Based on Monte Carlo Analysis for Optimizing Evacuation Routing in Complex Scenes. Math Probl Eng 2015Google Scholar
  43. 43.
    Li J et al (2016) A self-learning image super-resolution method via sparse representation and non-local similarity. Neurocomputing 184:196–206Google Scholar
  44. 44.
    Liu YH (2016) Energy-Efficient Phase-Domain RF Receivers for Internet-of-Things (IOT) Applications. Efficient Sensor Interfaces, Advanced Amplifiers and Low Power RF Systems. Springer International Publishing, p 295–311Google Scholar
  45. 45.
    Liu W et al. (2015) An image super resolution reconstruction algorithm based on Undecimated Morphological Wavelet. 2015 I.E. international conference on digital signal processing (DSP). IEEEGoogle Scholar
  46. 46.
    Liu L et al (2016) Learning spatio-temporal representations for action recognition: a genetic programming approach. IEEE transactions on cybernetics 46(1):158–170Google Scholar
  47. 47.
    Mansoor A, Linguraru MG (2016) Generic method for intensity standardization of medical images using multiscale curvelet representation. Biomedical Imaging (ISBI), 2016 I.E. 13th international symposium on. IEEEGoogle Scholar
  48. 48.
    Oquab M et al. (2014) Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE conference on computer vision and pattern recognitionGoogle Scholar
  49. 49.
    Parikh, S, Kalva H, Adzic V (2016) Evaluation of HEVC compression for high bit depth medical images. 2016 I.E. international conference on consumer electronics (ICCE). IEEEGoogle Scholar
  50. 50.
    Peddigari V, Gamadia M, Kehtarnavaz N (2005) Real-time implementation issues in passive automatic focusing for digital still cameras. J Imaging Sci Technol 49(2):114–123Google Scholar
  51. 51.
    Popple R et al (2016) SU-FJ-96: comparison of frame-based and mutual information registration techniques for CT and MR image sets. Med Phys 43(6):3428–3429Google Scholar
  52. 52.
    Radenović F, Tolias G, Chum O (2016) CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples. arXiv preprint arXiv:1604.02426Google Scholar
  53. 53.
    Ramachandra M (2016) Optimization of the data transactions and computations in IoT sensors. Internet of Things and Applications (IOTA), International Conference on. IEEEGoogle Scholar
  54. 54.
    Salih YK, See OH, Ibrahim RW (2016) An intelligent selection method based on game theory in heterogeneous wireless networks. T Emerg Telecommun TGoogle Scholar
  55. 55.
    Samarin A et al (2015) Image registration accuracy of an in-house developed patient transport system for PET/CT+ MR and SPECT+ CT imaging. Nucl Med Commun 36(2):194–200Google Scholar
  56. 56.
    Sandeep P, Jacob T (2016) Single image super-resolution using a joint GMM method. IEEE Trans Image Process 25(9):4233–4244MathSciNetzbMATHGoogle Scholar
  57. 57.
    Schnabel JA et al (2016) Advances and challenges in deformable image registration: from image fusion to complex motion modelling. Med Image Anal 33:145–148Google Scholar
  58. 58.
    Shah AJ, Gupta SB (2016) A technique to preserve edge information in single image super resolution. Procedia Computer Science 85:100–108Google Scholar
  59. 59.
    Shi Y, Du S, Wang W (2016) Local consistent low rank representation for image clustering.” Control and Decision Conference (CCDC), 2016 Chinese. IEEEGoogle Scholar
  60. 60.
    Shih FY, Zhong X (2016) High-capacity multiple regions of interest watermarking for medical images. Inf Sci 367:648–659Google Scholar
  61. 61.
    Song X et al. (2016) Novel Near-Lossless Compression Algorithm for Medical Sequence Images with Adaptive Block-Based Spatial Prediction. J Digit Imaging 1–10Google Scholar
  62. 62.
    Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190Google Scholar
  63. 63.
    Sundaresh R, Rodr JJ (2016) Image super-resolution using graph regularized block sparse representation. 2016 I.E. southwest symposium on image analysis and interpretation (SSIAI). IEEEGoogle Scholar
  64. 64.
    Tandale SB, Chougule SR (2016) Image super resolution in wavelet domain using edge enhancement via a sparse representation. Imp J Interdiscipl Res 2(9)Google Scholar
  65. 65.
    Tang J et al. (2016) Region similarity arrangement for image retrieval. Multimedia and Expo (ICME), 2016 I.E. international conference on. IEEEGoogle Scholar
  66. 66.
    Tolias G, Avrithis Y, Jégou H (2016) Image search with selective match kernels: aggregation across single and multiple images. Int J Comput Vision 116(3):247–261MathSciNetGoogle Scholar
  67. 67.
    Tu NA et al (2016) Topic modeling and improvement of image representation for large-scale image retrieval. Inf Sci 366:99–120MathSciNetGoogle Scholar
  68. 68.
    Vialard F-X et al (2012) Diffeomorphic 3D image registration via geodesic shooting using an efficient adjoint calculation. Int J Comput Vis 97(2):229–241MathSciNetzbMATHGoogle Scholar
  69. 69.
    Wan C (2016) A new texture image retrieval method based on shape and statistical parameters. J Comput Theor Nanosci 13(5):2753–2762Google Scholar
  70. 70.
    Wang H, Wang J (2014) An effective image representation method using kernel classification. 2014 I.E. 26th international conference on tools with artificial intelligence. IEEEGoogle Scholar
  71. 71.
    Wang J et al. (2016) Optimizing top precision performance measure of content-based image retrieval by learning similarity function. arXiv preprint arXiv:1604.06620Google Scholar
  72. 72.
    Wermelinger F et al. (2016) An efficient compressible multicomponent flow solver for heterogeneous CPU/GPU architectures. Proceedings of the Platform for Advanced Scientific Computing Conference. ACMGoogle Scholar
  73. 73.
    Wu W (2016) Paralleled Laplacian of Gaussian (LoG) edge detection algorithm by using GPU. Eighth International Conference on Digital Image Processing (ICDIP 2016). International Society for Optics and PhotonicsGoogle Scholar
  74. 74.
    Xue J, Zhao G, Xiao W (2016) Efficient GPU out-of-core visualization of large-scale CAD models with voxel representations. Adv Eng Software 99:73–80Google Scholar
  75. 75.
    Yan F, Iliyasu AM, Venegas-Andraca SE (2016) A survey of quantum image representations. Quantum Inf Process 15(1):1–35MathSciNetzbMATHGoogle Scholar
  76. 76.
    Yang X et al. (2014) Ultrasound 2D strain estimator based on image registration for ultrasound elastography. SPIE Medical Imaging. International Society for Optics and PhotonicsGoogle Scholar
  77. 77.
    Yang J et al. (2015) A novel regularized K-SVD dictionary learning based medical image super-resolution algorithm. Multimed Tools Appl 1–14Google Scholar
  78. 78.
    Yao Y et al (2016) STEM image simulation with hybrid CPU/GPU programming. Ultramicroscopy 166:1–8Google Scholar
  79. 79.
    Ye S et al. (2015) Coupled fisher discrimination dictionary learning for single image super-resolution.” 2015 I.E. International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEEGoogle Scholar
  80. 80.
    Ye L et al (2016) Saliency detection via similar image retrieval. IEEE Signal Process Lett 23(6):838–842Google Scholar
  81. 81.
    Yue B et al (2016) Joint prior learning for visual sensor network noisy image super-resolution. Sensors 16(3):288Google Scholar
  82. 82.
    Zhang Y, Wu L (2012) A novel method for rigid image registration based on firefly algorithm. Int J Res Rev Soft and Intell Comput (IJRRSIC) 2(2)Google Scholar
  83. 83.
    Zhao N et al. (2016) Fast Single Image Super-resolution using a New Analytical Solution for L2-L2 ProblemsGoogle Scholar
  84. 84.
    Zhong Z et al (2016) TU-AB-202-05: GPU-based 4D deformable image registration using adaptive tetrahedral mesh modeling. Med Phys 43(6):3737–3737Google Scholar
  85. 85.
    Zou HD, Wang HX (2013) A new automatic focusing algorithm and its application on vision measuring machine. Applied Mechanics and Materials, vol 397. Trans Tech PublicationsGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Yu Liu
    • 1
  • Jie Yang
    • 1
  • Jing Mi
    • 1
  • Jingjing Yang
    • 1
  • Xiao Zhang
    • 1
    Email author
  1. 1.Hebei North UniversityZhangjiakouChina

Personalised recommendations