Skip to main content
SpringerLink
Log in

A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always introduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale-invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sodagar I. The MPEG-DASH standard for multimedia Streaming Over the Internet. IEEE Multimedia, 2011, 18(4): 62–67

    Article  Google Scholar 

  2. Schwarz H, Marpe D, Wiegand T. Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(9): 1103–1120

    Article  Google Scholar 

  3. Song B C, Jeong S C, Choi Y. Video super-resolution algorithm using bi-directional overlapped block motion compensation and onthefly dictionary training. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(3): 274–285

    Article  Google Scholar 

  4. Hung E M, de Queiroz R L, Brandi F, de Oliveira K F, Mukherjee D. Video super-resolution using codebooks derived from keyframes. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(9): 1321–1331

    Article  Google Scholar 

  5. Ferreira R U, Hung EM, de Queiroz R L. Video super resolution based on local invariant features matching. In: Proceedings of 19th IEEE International Conference on Image Processing. 2012, 877–880

    Google Scholar 

  6. Lowe D G. Object recognition from local scale-invariant features. In: Proceedings of the 17th IEEE International Conference on Computer Vision. 1999, 1150–1157

    Chapter  Google Scholar 

  7. Freeman WT, Jones T R, Pasztor E C. Example-based superresolution. IEEE Computer Graphics and Applications, 2002, 22(2): 56–65

    Article  Google Scholar 

  8. Brandi F, de Queiroz R, Mukherjee D. Super resolution of video using key-frames. In: Proceedings of the IEEE International Symposium on Circuits Systems. 2008, 1608–1611

    Google Scholar 

  9. Brandi F, de Queiroz R L, Mukherjee D. Super-resolution of video using key-frames and motion estimation. In: Proceedings of the 15th IEEE International Conference on Image Processing. 2008, 321–324

    Google Scholar 

  10. Oliveira K F, Brandi F, Hung E M, de Queiroz R L, Mukherjee D. Bipredictive video super-resolution using key-frames. In: Proceedings of SPIE Symposium on Electronic Image, Visual Information Processing and Communication. 2010, 1–5

    Google Scholar 

  11. Hung E M, de Queiroz R L, Mukherjee D. Inter-frame postprocessing for intra-coded video. Journal of Communication and Information Systems, 2013, 28(1): 1–7

    Article  Google Scholar 

  12. Wen J, Li S, Lu Y, Fang M, Dong X, Chang H, Tao P. Cross segment decoding for improved quality of experience for video applications. In: Proceedings of the 2013 IEEE Data Compression Conference. 2013, 231–240

    Google Scholar 

  13. Wang Q, Tang X, Shum H. Patch based blind image super resolution. In: Proceedings of the 10th IEEE International Conference on Computer Vision. 2005, 709–716

    Google Scholar 

  14. Stephenson T A, Chen T. Adaptive Markov random fields for example-based super-resolution of faces. Journal on Applied Signal Processing, 2006, 2006: 1–11

    Article  Google Scholar 

  15. Qiu G. Interresolution look-up table for improved spatial magnification of image. Journal of Visual Communication and Image Representation. 2000, 11: 360–373

    Article  Google Scholar 

  16. Elad M, Datsenko D. Example-based regularization deployed to superresolution reconstruction of single image. The Computer Journal Advance Access, 2007, 20: 15–30

    Google Scholar 

  17. Besag J. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B, 1974, 36: 192–293

    MathSciNet  MATH  Google Scholar 

  18. Sun D, Roth S, Lewis J, Black M J. Learning optical flow. Lecture Notes in Computer Science, 2008, 5304: 83–97

    Article  Google Scholar 

  19. Liu C, Yuen J, Torralba A, Sivic J, Freeman W T. SIFT flow: dense correspondence across different scenes. Lecture Notes in Computer Science, 2008, 5304: 28–42

    Article  Google Scholar 

  20. Pan F, Lin X, Rahardja S, Lin W, Ong E, Yao S, Lu Z, Yang X. A locally adaptive algorithm for measuring blocking artifacts in images and videos. Signal Processing: Image Communication, 2004, 19(6): 499–506

    Google Scholar 

  21. Brown M, Lowe D G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 2007, 74(1): 59–73

    Article  Google Scholar 

  22. Horn B, Schunck B. Determining optical flow. Artificial Intelligence, 1981, 16: 185–203

    Article  Google Scholar 

  23. Wang S, Uchida S, Liwicki M, Feng Y K. Part-based methods for handwritten digit recognition. Frontiers of Computer Science, 2013, 7(4): 514–525

    Article  MathSciNet  Google Scholar 

  24. Mehrotra H, Majhi B. Local feature based retrieval approach for iris biometrics. Frontiers of Computer Science, 2013, 7(5): 767–781

    Article  MathSciNet  Google Scholar 

  25. PRIYA R, Shanmugama T H. Comprehensive review of significant researches on content based indexing and retrieval of visual information. Frontiers of Computer Science, 2013, 7(5): 782–799

    Article  Google Scholar 

  26. Wang Y W, Zhou Y C, Liu Y, Luo Z, Guo D H, Shao J, Tan F, Wu L, Li J H, Yan B P. A grid-based clustering algorithm for wild bird distribution. Frontiers of Computer Science, 2013, 7(4): 475–485

    Article  MathSciNet  MATH  Google Scholar 

  27. Kang L, Wu L D, Yang Y H. A Novel Unsupervised Approach for Multilevel Image Clustering from Unordered Image Collection. Frontiers of Computer Science, 2013, 7(1): 69–82

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Xuan Dong & Jiangtao Wen

Authors
  1. Xuan Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiangtao Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuan Dong.

Additional information

Xuan Dong received his BS in computer science and technology from Beihang University, China, in 2010. He is a PhD candidate in the Department of Computer Science and Technology, Tsinghua University. His current research interests include computational photography, video processing, video coding, and image segmentation.

Jiangtao Wen received his BS, MS, and PhD all in electrical engineering from Tsinghua University, China in 1992, 1994, and 1996, respectively. From 1996 to 1998, he was a staff research fellow at the University of California, Los Angeles (UCLA). After UCLA, he served as the principal scientist at PacketVideo Corp., chief technical officer at Morphbius Technology Inc., director of Video Codec Technologies at Mobilygen Corp., and as a technology advisor at Ortiva Wireless and Stretch, Inc. Since 2009, he has been a professor in the Department of Computer Science and Technology, Tsinghua University. His research focuses on multimedia communication over challenging networks and computational photography.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, X., Wen, J. A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement. Front. Comput. Sci. 9, 729–740 (2015). https://doi.org/10.1007/s11704-015-4184-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-015-4184-0

Keywords

Search

Navigation