Advertisement

Journal of Computer Science and Technology

, Volume 33, Issue 3, pp 475–486 | Cite as

Geometry of Motion for Video Shakiness Detection

  • Xiao-Qun Wu
  • Hai-Sheng Li
  • Jian Cao
  • Qiang Cai
Regular Paper
  • 35 Downloads

Abstract

This paper presents a novel algorithm for automatically detecting global shakiness in casual videos. Perframe amplitude is computed by the geometry of motion, based on the kinematic model defined by inter-frame geometric transformations. Inspired by motion perception, we investigate the just-noticeable amplitude of shaky motion perceived by the human visual system. Then, we use the thresholding contrast strategy on the statistics of per-frame amplitudes to determine the occurrence of perceived shakiness. For testing the detection accuracy, a dataset of video clips is constructed with manual shakiness label as the ground truth. The experiments demonstrate that our algorithm can obtain good detection accuracy that is in concordance with subjective judgement on the videos in the dataset.

Keywords

video shakiness kinematic model motion perception 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11390_2018_1832_MOESM1_ESM.pdf (307 kb)
ESM 1 (PDF 306 kb)

References

  1. 1.
    Abdollahian G, Taskiran C M, Pizlo Z, Delp E J. Camera motion-based analysis of user generated video. IEEE Trans. Multimedia, 2010, 12(1): 28-41.CrossRefGoogle Scholar
  2. 2.
    Hu S M, Chen T, Xu K, Cheng M M, Martin R R. Internet visual media processing: A survey with graphics and vision applications. The Visual Computer, 2013, 29(5): 393-405.CrossRefGoogle Scholar
  3. 3.
    Zhang L, Zhou L, Huang H. Bundled kernels for nonuniform blind video deblurring. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(9): 1882-1894.CrossRefGoogle Scholar
  4. 4.
    Yan F, Iliyasu A M, Yang H M, Hirota K. Strategy for quantum image stabilization. Science China Information Sciences, 2016, 59(5): 052102.CrossRefGoogle Scholar
  5. 5.
    Kakar P, Sudha N, Ser W. Exposing digital image forgeries by detecting discrepancies in motion blur. IEEE Trans. Multimedia, 2011, 13(3): 443-452.CrossRefGoogle Scholar
  6. 6.
    Su B L, Lu S J, Tan C L. Blurred image region detection and classification. In Proc. the 19th ACM Int. Conf. Multimedia, November 2011, pp.1397-1400.Google Scholar
  7. 7.
    Yu X, Xu F, Zhang S L, Zhang L. Efficient patch-wise non-uniform deblurring for a single image. IEEE Trans. Multimedia, 2014, 16(6): 1510-1524.CrossRefGoogle Scholar
  8. 8.
    Visentini-Scarzanella M, Dragotti P L. Video jitter analysis for automatic bootleg detection. In Proc. the 14th Int. Workshop on Multi-Media Signal Processing, September 2012, pp.101-106.Google Scholar
  9. 9.
    Sibiryakov A. Hand jitter descriptor for mobile video identification. In Proc. Int. Conf. Consumer Electronics, January 2011, pp.77-78.Google Scholar
  10. 10.
    Chen H H, Liang C K, Peng Y C, Chang H A. Integration of digital stabilizer with video codec for digital video cameras. IEEE Trans. Circuits and Systems for Video Technology, 2007, 17(7): 801-813.CrossRefGoogle Scholar
  11. 11.
    Xue Y Y, Erkin B, Wang Y. A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing. IEEE Trans. Multimedia, 2015, 17(1): 134-139.CrossRefGoogle Scholar
  12. 12.
    Yan B, Yuan B H, Yang B. Effective video retargeting with jittery assessment. IEEE Trans. Multimedia, 2014, 16(1): 272-277.CrossRefGoogle Scholar
  13. 13.
    Zhang F L, Wang J, Zhao H, Martin R R, Hu S M. Simultaneous camera path optimization and distraction removal for improving amateur video. IEEE Trans. Image Processing, 2015, 24(12): 5982-5994.MathSciNetCrossRefGoogle Scholar
  14. 14.
    Zhang L, Xu Q K, Huang H. A global approach to fast video stabilization. IEEE Trans. Circuits and Systems for Video Technology, 2017, 27(2): 225-235.CrossRefGoogle Scholar
  15. 15.
    Huang H Z, Fang X N, Ye Y F, Zhang S H, Rosin P L. Practical automatic background substitution for live video. Computational Visual Media, 2017, 3(3): 273-284.CrossRefGoogle Scholar
  16. 16.
    Hasegawa K, Saito H. Synthesis of a stroboscopic image from a hand-held camera sequence for a sports analysis. Computational Visual Media, 2016, 2(3): 277-289.CrossRefGoogle Scholar
  17. 17.
    Joshi N, Kienzle W, Toelle M, Uyttendaele M, Cohen M F. Real-time hyperlapse creation via optimal frame selection. ACM Trans. Graphics, 2015, 34(4): Article No. 63.Google Scholar
  18. 18.
    Wang M, Liang J B, Zhang S H, Lu S P, Shamir A, Hu S M. Hyper-lapse from multiple spatially-overlapping videos. IEEE Trans. Image Processing, 2018, 27(4): 1735-1747.MathSciNetCrossRefGoogle Scholar
  19. 19.
    Tong H H, Li M J, Zhang H J, Zhang C S. Blur detection for digital images using wavelet transform. In Proc. Int. Conf. Multimedia and Expo., June 2004, pp.17-20.Google Scholar
  20. 20.
    Tico M, Trimeche M, Vehvilainen M. Motion blur identification based on differently exposed images. In Proc. Int. Conf. Image Processing, October 2006, pp.2021-2024.Google Scholar
  21. 21.
    Liu R T, Li Z R, Jia J Y. Image partial blur detection and classification. In Proc. Conf. Computer Vision and Pattern Recognition, June 2008.Google Scholar
  22. 22.
    YanW Q, Kankanhalli M S. Detection and removal of lighting & shaking artifacts in home videos. In Proc. ACM Int. Conf. Multimedia, December 2002, pp.107-116.Google Scholar
  23. 23.
    Liu F, Gleicher M, Jin H L, Agarwala A. Content-preserving warps for 3D video stabilization. ACM Trans. Graphics, 2009, 28(3): Article No. 44.Google Scholar
  24. 24.
    Zhang L, Chen X Q, Kong X Y, Huang H. Geodesic video stabilization in transformation space. IEEE Trans. Image Processing, 2017, 26(5): 2219-2229.MathSciNetCrossRefGoogle Scholar
  25. 25.
    Wolpert D M, Ghahramani Z. Computational principles of movement neuroscience. Nature Neuroscience, 2000, 3(Suppl): 1212-1217.Google Scholar
  26. 26.
    Murray R M, Li Z X, Sastry S S. A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994.Google Scholar
  27. 27.
    Zacur E, Bossa M, Olmos S. Left-invariant Riemannian geodesics on spatial transformation groups. SIAM Journal on Imaging Sciences, 2014, 7(3): 1503-1557.MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Duan L Y, Jin J S, Tian Q, Xu C S. Nonparametric motion characterization for robust classification of camera motion patterns. IEEE Trans. Multimedia, 2006, 8(2): 323-340.CrossRefGoogle Scholar
  29. 29.
    Afonso M V, Nascimento J C, Marques J S. Automatic estimation of multiple motion fields from video sequences using a region matching based approach. IEEE Trans. Multimedia, 2013, 16(1): 1-14.CrossRefGoogle Scholar
  30. 30.
    Nishi K, Onda T. Evaluation system for camera shake and image stabilizers. In Proc. Int. Conf. Multimedia and Expo., July 2010, pp.926-931.Google Scholar
  31. 31.
    Albright T D, Stoner G R. Visual motion perception. Proceedings the National Academy of Sciences of the United States of America, 1995, 92(7): 2433-2440.CrossRefGoogle Scholar
  32. 32.
    Peli E, García-Pérez M A. Motion perception during involuntary eye vibration. Experimental Brain Research, 2003, 149(4): 431-438.CrossRefGoogle Scholar
  33. 33.
    Healey C G, Sawant A P. On the limits of resolution and visual angle in visualization. ACM Trans. Applied Perception, 2012, 9(4): Article No. 20.Google Scholar
  34. 34.
    Martins A J, Kowler E, Palmer C. Smooth pursuit of small-amplitude sinusoidal motion. Journal of the Optical Society of America A, 1985, 2(2): 234-242.CrossRefGoogle Scholar
  35. 35.
    He K M, Chang H W, Sun J. Content-aware rotation. In Proc. Int. Conf. Computer Vision, December 2013, pp.553-560.Google Scholar
  36. 36.
    Shi J B, Tomasi C. Good features to track. In Proc. Computer Society Conf. Computer Vision and Pattern Recognition, June 1994, pp.593-600.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Xiao-Qun Wu
    • 1
    • 2
  • Hai-Sheng Li
    • 1
    • 2
  • Jian Cao
    • 1
    • 2
  • Qiang Cai
    • 1
    • 2
  1. 1.School of Computer and Information EngineeringBeijing Technology and Business UniversityBeijingChina
  2. 2.Beijing Key Laboratory of Big Data Technology for Food SafetyBeijing Technology and Business UniversityBeijingChina

Personalised recommendations