Neural Processing Letters

, Volume 43, Issue 2, pp 459–477 | Cite as

Real-Time Model-Based Video Stabilization for Microaerial Vehicles

Article

Abstract

The emerging branch of micro aerial vehicles (MAVs) has attracted a great interest for their indoor navigation capabilities, but they require a high quality video for tele-operated or autonomous tasks. A common problem of on-board video quality is the effect of undesired movements, so different approaches solve it with both mechanical stabilizers or video stabilizer software. Very few video stabilizer algorithms in the literature can be applied in real-time but they do not discriminate at all between intentional movements of the tele-operator and undesired ones. In this paper, a novel technique is introduced for real-time video stabilization with low computational cost, without generating false movements or decreasing the performance of the stabilized video sequence. Our proposal uses a combination of geometric transformations and outliers rejection to obtain a robust inter-frame motion estimation, and a Kalman filter based on an ANN learned model of the MAV that includes the control action for motion intention estimation.

Keywords

Video stabilization Micro aerial vehicles Real-time RANSAC Modelling Motion intention Kalman filter 

References

  1. 1.
    Aguilar WG, Angulo C (2012a) Compensación y aprendizaje de efectos generados en la imagen durante el desplazamiento de un robot. In: Proceedings of the 10th symposium CEA-Spanish Committee of Automatic Control, February 2012Google Scholar
  2. 2.
    Aguilar WG, Angulo C (2012b) Compensación de los efectos generados en la imagen por el control de navegaciń del robot aibo ers 7. In: Proceedings of the 7th congress of science and technology ESPE 2012, June 2012Google Scholar
  3. 3.
    Aguilar WG, Angulo C (2013) Estabilización robusta de vídeo basada en diferencia de nivel de gris. In: Proceedings of the 8th congress of science and technology ESPE 2013, June 2013Google Scholar
  4. 4.
    Aguilar WG, Angulo C (2014a) Real-time video stabilization without phantom movements for micro aerial vehicles. EURASIP J Image Video Process 2014(1):46Google Scholar
  5. 5.
    Aguilar WG, Angulo C (2014b) Optimization of robust video stabilization based on motion intention for micro aerial vehicles. In: 2014 International multi-conference on systems signals and devices (SSD), February 2014Google Scholar
  6. 6.
    Aguilar WG, Angulo C (2014c) Estabilización de vídeo en micro vehículos aéreos y su aplicación en la detección de caras. In: Proceedings of the 9th congress of science and technology ESPE 2014, May 2014Google Scholar
  7. 7.
    Aguilar WG, Angulo C, Costa R, Molina L (2014) Control autónomo de cuadricopteros para seguimiento de trayectorias. In: Proceedings of the 9th congress of science and technology ESPE 2014, May 2014Google Scholar
  8. 8.
    Alahi A, Ortiz R, Vandergheynst P (2012) FREAK: fast retina keypoint. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR), 2012, pp 510–517Google Scholar
  9. 9.
    Bailey SW, Bodenheimer B (2012) A comparison of motion capture data recorded from a vicon system and a Microsoft kinect sensor. In: Proceedings of the ACM symposium on applied perception, SAP ’12, 2012. ACM, New York, p 121Google Scholar
  10. 10.
    Battiato S, Gallo G, Puglisi G, Scellato S (2007) SIFT features tracking for video stabilization. In: 14th International conference on image analysis and processing, 2007, ICIAP 2007, pp 825–830Google Scholar
  11. 11.
    Bay H, Tuytelaars T, Gool L (2006) SURF: speeded up robust features. In: Leonardis A, Bischof H, Pinz A (eds) Computer vision—ECCV 2006, lecture notes in computer science, vol 3951. Springer Berlin Heidelberg, Berlin, pp 404–417Google Scholar
  12. 12.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI–8(6):679–698CrossRefGoogle Scholar
  13. 13.
    Chang HC, Lai SH, Lu KR (2004) A robust and efficient video stabilization algorithm. In: 2004 IEEE international conference on multimedia and expo, 2004, ICME ’04, vol 1, pp 29–32Google Scholar
  14. 14.
    Derpanis KG (2010) Overview of the RANSAC algorithm. Computer Science, York UniversityGoogle Scholar
  15. 15.
    Dimov D, Nikolov A (2014) Real time video stabilization for handheld devices. In: Proceedings of the 15th international conference on computer systems and technologies. ACM, pp 124–133Google Scholar
  16. 16.
    Fang CL, Tsai TH, Chang CH (2012) Video stabilization with local rotational motion model. In: 2012 IEEE Asia Pacific conference on circuits and systems (APCCAS), pp 551–554Google Scholar
  17. 17.
    Faugeras O, Luong QT, Papadopoulou T (2001) The geometry of multiple images: the laws that govern the formation of images of a scene and some of their applications. MIT Press, CambridgeMATHGoogle Scholar
  18. 18.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395MathSciNetCrossRefGoogle Scholar
  19. 19.
    Forsyth DA, Ponce J (2002) Computer vision: a modern approach. Prentice Hall Professional Technical Reference, Upper Saddle RiverGoogle Scholar
  20. 20.
    Grundmann M, Kwatra V, Essa I (2011) Auto-directed video stabilization with robust l1 optimal camera paths. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 225–232Google Scholar
  21. 21.
    Harris C, Stephens M (1988) A combined corner and edge detection. In: Proceedings of the fourth Alvey vision conference, pp 147–151Google Scholar
  22. 22.
    Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New YorkMATHGoogle Scholar
  23. 23.
    Hsu YF, Chou CC, Shih MY (2012) Moving camera video stabilization using homography consistency. In: 19th IEEE international conference on image processing (ICIP), 2012, pp 2761–2764Google Scholar
  24. 24.
    Kang SJ, Wang TS, Kim DH, Morales A, Ko SJ (2012) Video stabilization based on motion segmentation. In: 2012 IEEE international conference on consumer electronics (ICCE), pp 416–417Google Scholar
  25. 25.
    Kendoul F (2012) Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems. J Field Robot 29(2):315–378CrossRefGoogle Scholar
  26. 26.
    Lee KY, Chuang YY, Chen BY, Ouhyoung M (2009) Video stabilization using robust feature trajectories. In: IEEE 12th international conference on computer vision, 2009, pp 1397–1404Google Scholar
  27. 27.
    Leutenegger S, Chli M, Siegwart RY (2011) BRISK: binary robust invariant scalable keypoints. In: 2011 IEEE international conference on computer vision (ICCV), pp 2548–2555Google Scholar
  28. 28.
    Liu F, Gleicher M, Wang J, Jin H, Agarwala A (2011) Subspace video stabilization. ACM Trans Graph (TOG) 30(1):4CrossRefGoogle Scholar
  29. 29.
    Lowe D (1999) Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on computer vision, 1999, vol 2, pp 1150–1157Google Scholar
  30. 30.
    Luo J, Oubong G (2009) A comparison of SIFT, PCA-SIFT and SURF. Int J Image Process (IJIP) 3(4):143–152Google Scholar
  31. 31.
    Matsushita Y, Ofek E, Ge W, Tang X, Shum HY (2006) Full-frame video stabilization with motion inpainting. IEEE Trans Pattern Anal Mach Intell 28(7):1150–1163CrossRefGoogle Scholar
  32. 32.
    Mikolajczyk K, Schmid C (2004) Scale & affine invariant interest point detectors. Int J Comput Vis 60(1):63–86Google Scholar
  33. 33.
    Niskanen M, Silven O, Tico M (2006) Video stabilization performance assessment. In: 2006 IEEE international conference on multimedia and expo, pp 405–408Google Scholar
  34. 34.
    Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE international conference on computer vision (ICCV), pp 2564–2571Google Scholar
  35. 35.
    Song C, Zhao H, Jing W, Zhu H (2012) Robust video stabilization based on particle filtering with weighted feature points. IEEE Trans Consum Electron 58(2):570–577CrossRefGoogle Scholar
  36. 36.
    Tordoff B, Murray DW (2002) Guided sampling and consensus for motion estimation. In: Heyden A, Sparr G, Nielsen M, Johansen P (eds) Computer vision—ECCV 2002, lecture notes in computer science, vol 2350. Springer Berlin Heidelberg, Berlin, pp 82–96Google Scholar
  37. 37.
    Wang C, Kim JH, Byun KY, Ni J, Ko SJ (2009) Robust digital image stabilization using the Kalman filter. IEEE Trans Consum Electron 55(1):6–14CrossRefGoogle Scholar
  38. 38.
    Xu J, Chang HW, Yang S, Wang M (2012) Fast feature-based video stabilization without accumulative global motion estimation. IEEE Trans Consum Electron 58(3):993–999CrossRefGoogle Scholar
  39. 39.
    Yang J, Schonfeld D, Mohamed M (2009) Robust video stabilization based on particle filter tracking of projected camera motion. IEEE Trans Circuits Syst Video Technol 19(7):945–954CrossRefGoogle Scholar
  40. 40.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45CrossRefGoogle Scholar
  41. 41.
    Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Automatic Control Department (ESAII)Universitat Politècnica de CatalunyaBarcelonaSpain

Personalised recommendations