Advertisement

Machine Vision and Applications

, Volume 23, Issue 3, pp 441–459 | Cite as

Super-resolution in practice: the complete pipeline from image capture to super-resolved subimage creation using a novel frame selection method

  • Maria PetrouEmail author
  • Mohamed H. Jaward
  • Shengyong Chen
  • Mark Briers
Original Paper

Abstract

We present a complete super-resolution system using a camera, that is assumed to be on a vibrating platform and continually capturing frames of a static scene, that have to be super-resolved in particular regions of interest. In a practical system the shutter of the camera is not synchronised with the vibrations it is subjected to. So, we propose a novel method for frame selection according to their degree of blurring and we combine a tracker with the sequence of selected frames to identify the subimages containing the region of interest. The extracted subimages are subsequently co-registered using a state of the art sub-pixel registration algorithm. Further selection of the co-registered subimages takes place, according to the confidence in the registration result. Finally, the subimage of interest is super-resolved using a state of the art super-resolution algorithm. The proposed frame selection method is of generic applicability and it is validated with the help of manual frame quality assessment.

Keywords

Super-resolution Image quality assessment Frame selection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baboulaz L., Dragotti P.L.: Exact feature extraction using finite rate of innovation principles with an application to image super-resolution. IEEE Trans. Image Process. 18(2), 281–298 (2009)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bascle, B., Blake, A., Zisserman, A.: Motion deblurring and super-resolution from an image sequence. In: European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 1065, pp 571–582 (1996)Google Scholar
  3. 3.
    Black M., Anandan P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comput. Vis. Image Understand. 63(1), 75–104 (1996)CrossRefGoogle Scholar
  4. 4.
    Blake A., Zisserman A.: Visual Reconstruction. MIT Press, Cambridge (1987)Google Scholar
  5. 5.
    Domke J., Aloimonos Y.: Image transformations and blurring. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 811–823 (2009)CrossRefGoogle Scholar
  6. 6.
    Esparragon, D.R., Gonzalez, L.M.M., Santana, J.G.V., Ortiz, J.C.: Automatic image quality measurement tool. In: 3rd International Conference on Information and Communication Technologies: From Theory to Applications, pp. 1–6, April (2008)Google Scholar
  7. 7.
    Ferzli R., Karam L.J.: A No-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18(4), 717–728 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Gunawan I.P., Ghanbari M.: Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Trans. Circ. Syst. Video Technol. 18(1), 71–83 (2008)CrossRefGoogle Scholar
  9. 9.
    Hardie R., Barnard K., Armstrong E.: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6(2), 1621–1633 (1997)CrossRefGoogle Scholar
  10. 10.
    Katartzis, A., Petrou, M.: Robust Bayesian estimation and normalised convolution for super-resolution image reconstruction. In: Workshop on Image Registration and Fusion, Computer Vision and Pattern Recognition, CVPR’07, Minneapolis, USA, 18–23 June 2007, pp. 1–7 (2007)Google Scholar
  11. 11.
    Katartzis A., Petrou M.: Current trends in super-resolution image reconstruction. In: Stathaki, T. (ed.) Image Fusion: Algorithms and Applications, Academic Press, New York (2008)Google Scholar
  12. 12.
    Knutsson, H., Westin, C.: Normalised and differential convolution. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 93, pp. 515–523 (1993)Google Scholar
  13. 13.
    Lee W.-H., Lai S.-H., Chen C.-L.: Iterative blind image motion deblurring via learning a No-reference image quality measure. IEEE Int. Conf. Image Process. 4, 405–408 (2007)Google Scholar
  14. 14.
    Li, B., Osberger, W.: Measurement of blurring in video sequences, United States Patent 7099518 (2006)Google Scholar
  15. 15.
    Mei T., Hua X.-S., Zhu C.-Z., Zhou H.-Q., Li S.: Home video visual quality assessment with spatiotemporal factors. IEEE Trans. Circ. Syst. Video Technol. 17(6), 699–706 (2007)CrossRefGoogle Scholar
  16. 16.
    Nguyen N., Milanfar P., Golub G.: Efficient generalised cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 308–1299 (2001)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Park S.C., Park M.K., Kang M.G.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 21–36 (2003)CrossRefGoogle Scholar
  18. 18.
    Petrou, M., Petrou, C.: Image Processing, the Fundamentals, J Wiley. ISBN: 9780470745861 (2010)Google Scholar
  19. 19.
    Pham, T., van Vliet, L., Schutte, K.: Robust fusion of irregularly sampled data using adaptive normalised convolution. EURASIP J. Appl. Signal Process. pp. 1–12 (2006)Google Scholar
  20. 20.
    Ponomarenko, N., Battisti, F., Egiazarian, K., Astola, J., Lukin, V.: Metrics performance comparison for colour image databases. In: Fourth International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, Arizona, USA. January 14–16 (2009)Google Scholar
  21. 21.
    Robinson D., Milanfer P.: Fundamental performance limits in image registration. IEEE Trans. Image Process. 13(9), 1185–1199 (2004)CrossRefGoogle Scholar
  22. 22.
    Segall C., Katsaggelos A., Molina R., Mateos J.: Bayesian resolution of enhancement of compressed video. IEEE Trans. Image Process. 13(7), 898–910 (2004)CrossRefGoogle Scholar
  23. 23.
    Sheikh H.R., Bovik A.C., Cormack L.: No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans. Image Process. 14(11), 1918–1927 (2005)CrossRefGoogle Scholar
  24. 24.
    Sheikh H.R., Bovik A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)CrossRefGoogle Scholar
  25. 25.
    Shen, H., Ng, M.K., Li, P., Zhang, L.: Super-resolution reconstruction algorithm to MODIS remote sensing images. Comput J. doi: 10.1093/compjnl/bxm028 (2007)
  26. 26.
    Shen H., Zhang L., Huang B., Li P.: A MAP approach for joint motion estimation, segmentation and super resolution. IEEE Trans. Image Process. 16(2), 479–490 (2007)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Shnayderman A., Gusev A., Eskicioglu A.M.: An SVD-based gray scale image quality measure for local and global assessment. IEEE Trans. Image Process. 15(2), 422–429 (2006)CrossRefGoogle Scholar
  28. 28.
    Thevenaz P., Ruttimann U., Unser M.: A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process. 7(1), 27–41 (1998)CrossRefGoogle Scholar
  29. 29.
    Wang X., Tian B., Liang C., Shi D.: Blind image quality assessment for measuring image blur. Congress Image Sig. Process. CISP’ 08(1), 467–470 (2008)CrossRefGoogle Scholar
  30. 30.
    Wee C.-Y., Paramesran R.: Measure of image sharpness using eigenvalues. Int. J. Inform. Sci. 177(12), 2533–2552 (2007)zbMATHGoogle Scholar
  31. 31.
    Woods N., Galatsanos N., Katsaggelos A.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process. 15(1), 201–213 (2006)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Zamani, A.N., Awang, M.K., Omar, N., Nazeer, S.A.: Image quality assessments and restoration for face detection and recognition system images. In: Second Asia International Conference on Modelling and Simulation, pp. 505–510, May (2008)Google Scholar
  33. 33.
    Zheng H., Hellwich O.: VQ-based Bayesian Estimation for Blur Identification and Image Selection in Video Sequences. Int. J. Innov. Comput. Inform. Control (IJICIC) 2(2), 399–410 (2006)Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Maria Petrou
    • 1
    Email author
  • Mohamed H. Jaward
    • 1
  • Shengyong Chen
    • 1
  • Mark Briers
    • 2
  1. 1.Communications and Signal Processing Group, Electrical and Electronic Engineering DepartmentImperial CollegeLondonUK
  2. 2.AT030, QinetiQMalvernUK

Personalised recommendations