Machine Vision and Applications

, Volume 25, Issue 5, pp 1283–1308 | Cite as

HMPMR strategy for real-time tracking in aerial images, using direct methods

  • Carol MartínezEmail author
  • Pascual Campoy
  • Iván F. Mondragón
  • José Luis Sánchez-Lopez
  • Miguel A. Olivares-Méndez
Original Paper


The vast majority of approaches make use of features to track objects. In this paper, we address the tracking problem with a tracking-by-registration strategy based on direct methods. We propose a hierarchical strategy in terms of image resolution and number of parameters estimated in each resolution, that allows direct methods to be applied in demanding real-time visual-tracking applications. We have called this strategy the Hierarchical Multi-Parametric and Multi-Resolution strategy (HMPMR). The Inverse Composition Image Alignment Algorithm (ICIA) is used as an image registration technique and is extended to an HMPMR-ICIA. The proposed strategy is tested with different datasets and also with image data from real flight tests using an Unmanned Aerial Vehicle, where the requirements of direct methods are easily unsatisfied (e.g. vehicle vibrations). Results show that using an HMPMR approach, it is possible to cope with the efficiency problem and with the small motion constraint of direct methods, conducting the tracking task at real-time frame rates and obtaining a performance that is comparable to, or even better than, the one obtained with the other algorithms that were analyzed.


Visual tracking Image registration  Hierarchical methods UAVs Parametric motion 



The work described in this paper is the result of several research stages conducted at the Computer Vision Group of the Universidad Politécnica de Madrid. The authors would like to thank the Universidad Politécnica de Madrid, the Consejería de Educación de la Comunidad de Madrid, and the Fondo Social Europeo (FSE) for the Ph.D. Scholarships of some of the Authors. This work has been supported by the Spanish Ministry of Science under grant MICYT DPI2010-20751-C02-01.

Supplementary material

Supplementary material 1 (mp4 48222 KB)


  1. 1.
    Anderson, C.H., Bergen, J.R., Burt, P.J., Ogden, J.M.: Pyramid methods in image processing. RCA Eng. 29(6), 33–41 (1984)Google Scholar
  2. 2.
    Baker, S., Datta, A., Kanade, T.: Parameterizing Homographies. Tech. Rep. CMU-RI-TR-06-11, Robotics Institute, Pittsburgh (2006)Google Scholar
  3. 3.
    Baker, S., Matthews, I.: Equivalence and efficiency of image alignment algorithms. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition 1, pp. 1090–1097 (2001)Google Scholar
  4. 4.
    Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(1), 221–255 (2004)CrossRefGoogle Scholar
  5. 5.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: ECCV ’92: Proceedings of the Second European Conference on Computer Vision, pp. 237–252 (1992)Google Scholar
  6. 6.
    Bouguet, J.Y.: Pyramidal implementation of the Lucas Kanade feature tracker: description of the algorithm. Technical report, OpenCV Document, Intel Microprocessor Research Labs (2002)Google Scholar
  7. 7.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly (2008)Google Scholar
  8. 8.
    Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983). doi: 10.1109/TCOM.1983.1095851 CrossRefGoogle Scholar
  9. 9.
    Can, A., Stewart, C., Roysam, B., Tanenbaum, H.: A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina. IEEE Transactions on Pattern Analysis and Machine Intelligence (2002)Google Scholar
  10. 10.
    Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23, 921–935 (2012)CrossRefGoogle Scholar
  11. 11.
    Corral, E.M.: Efficient model-based 3d tracking by using direct image registration. Ph.D. thesis, Facultad de Informática. Universidad Politécnica de Madrid, Spain (2012)Google Scholar
  12. 12.
    Dufaux, F., Konrad, J.: Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9(3), 497–501 (2000)CrossRefGoogle Scholar
  13. 13.
    Dupac, J., Matas, J., Naiser, F.: Ultra-fast tracking based on zero-shift points. Image Vis. Comput. 30(12), 1016–1031 (2012)CrossRefGoogle Scholar
  14. 14.
    García Carrillo, L., Rondon, E., Sanchez, A., Dzul, A., Lozano, R.: Stabilization and trajectory tracking of a quad-rotor using vision. J. Intell. Robot. Syst. 61, 103–118 (2011)CrossRefGoogle Scholar
  15. 15.
    Hager, G., Belhumeur, P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998). doi: 10.1109/34.722606 CrossRefGoogle Scholar
  16. 16.
    Hanna, K., Okamoto, N.: Combining stereo and motion analysis for direct estimation of scene structure. In: Proceedings of Fourth International Conference on Computer Vision, pp. 357–365 (1993). doi: 10.1109/ICCV.1993.378192
  17. 17.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, New York (2003)Google Scholar
  18. 18.
    Hess, R.: SIFT feature detector implementation in C. (2007)
  19. 19.
    Hess, R.: An open-source SIFT library. In: Proceedings of the International Conference on Multimedia, pp. 1493–1496 (2010).
  20. 20.
    Holzer, S., Ilic, S., Navab, N.: Multilayer adaptive linear predictors for real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 105–117 (2013). doi: 10.1109/TPAMI.2012.86
  21. 21.
    Hwangbo, M., Kim, J.S., Kanade, T.: Inertial-aided klt feature tracking for a moving camera. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, pp. 1909–1916 (2009). doi: 10.1109/IROS.2009.5354093
  22. 22.
    Irani, M., Anandan, P.: About direct methods. In: Vision Algorithms: Theory and Practice, Lecture Notes in Computer Science, vol. 1883, pp. 267–277. Springer, Berlin (2000)Google Scholar
  23. 23.
    Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994). doi: 10.1007/BF01420982 CrossRefGoogle Scholar
  24. 24.
    Jurie, F., Dhome, M.: Real time robust template matching. In: Rosin, P.L., Marshall, A.D. (eds.) British Machine Vision Conference, BMVC 2002, September, 2002, pp. 123–132. British Machine Vision Association, Cardiff (2002)Google Scholar
  25. 25.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012). doi: 10.1109/TPAMI.2011.239 CrossRefGoogle Scholar
  26. 26.
    Kumar, R., Sawhney, H., Samarasekera, S., Hsu, S., Tao, H., Guo, Y., Hanna, K., Pope, A., Wildes, R., Hirvonen, D., Hansen, M., Burt, P.: Aerial video surveillance and exploitation. Proc. IEEE 89(10), 1518–1539 (2001)CrossRefGoogle Scholar
  27. 27.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  28. 28.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  29. 29.
    Malis, E., Benhimane, S.: A unified approach to visual tracking and servoing. Robot. Auton. Syst. 52(1), 39–52 (2005). doi: 10.1016/j.robot.2005.03.014. (Advances in Robot Vision)CrossRefGoogle Scholar
  30. 30.
    Martinez, C., Mejias, L., Campoy, P.: A multi-resolution image alignment technique based on direct methods for pose estimation of aerial vehicles. In: Proceedings of the International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 542–548 (2011). doi: 10.1109/DICTA.2011.97
  31. 31.
    Martínez, C., Richardson, T., Thomas, P., du Bois, J.L., Campoy, P.: A vision-based strategy for autonomous aerial refueling tasks. Robot. Auton. Syst. 61(8), 876–895 (2013). doi: 10.1016/j.robot.2013.02.006 CrossRefGoogle Scholar
  32. 32.
    Mejias, L., Saripalli, S., Campoy, P., Sukhatme, G.: Visual servoing approach for tracking features in urban areas using an autonomous helicopter. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 2503–2508, Orlando (2006)Google Scholar
  33. 33.
    Mondragon, I.F., Campoy, P., Correa, J., Mejias, L.: Visual model feature tracking for UAV control. In: IEEE International Symposium on Intelligent Signal Processing, 2007. WISP 2007, pp. 1–6 (2007). doi: 10.1109/WISP.2007.4447629
  34. 34.
    Mondragón, I.F., Campoy, P., Martinez, C., Olivares-Mendez, M.: 3D pose estimation based on planar object tracking for UAVs control. In: Proceedings of IEEE International Conference on Robotics and Automation 2010 ICRA2010, Anchorage (2010)Google Scholar
  35. 35.
    Rao, C., Guo, Y., Sawhney, H., Kumar, R.: A heterogeneous feature-based image alignment method. In: 18th International Conference on Pattern Recognition, 2006. ICPR 2006 (2006)Google Scholar
  36. 36.
    Sawhney, H., Kumar, R.: True multi-image alignment and its application to mosaicing and lens distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 21(3), 235–243 (1999). doi: 10.1109/34.754589 CrossRefGoogle Scholar
  37. 37.
    Sawhney, H.S., Hsu, S., Kumar, R.: Robust video mosaicing through topology inference and local to global alignment. In: Proc. European Conference on Computer Vision, pp. 103–119 (1998)Google Scholar
  38. 38.
    Sheikh, Y., Khan, S., Shah, M.: Feature-based georegistration of aerial images. In: International Conference on Geosensor Networks (2004)Google Scholar
  39. 39.
    Shi, J., Tomasi, C.: Good features to track. In: 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94), pp. 593–600 (1994)Google Scholar
  40. 40.
    Shum, H.Y., Szeliski, R.: Panoramic vision. chap. Construction of Panoramic Image Mosaics with Global and Local Alignment, pp. 227–268. Springer, Secaucus (2001).
  41. 41.
    Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vis. 2(1), 1–104 (2006). doi: 10.1561/0600000009 CrossRefGoogle Scholar
  42. 42.
    Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’97, pp. 251–258. ACM Press/Addison-Wesley Publishing Co., New York (1997). doi: 10.1145/258734.258861
  43. 43.
    Teuliere, C., Eck, L., Marchand, E.: Chasing a moving target from a flying uav. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4929–4934 (2011). doi: 10.1109/IROS.2011.6094404
  44. 44.
    Torr, P.H.S., Zisserman, A.: Feature based methods for structure and motion estimation. In: Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, ICCV ’99, pp. 278–294. Springer, London (2000)Google Scholar
  45. 45.
    Tsaig, Y., Averbuch, A.: Automatic segmentation of moving objects in video sequences: a region labeling approach. IEEE Trans. Circuits Syst. Video Techol. 12(7), 597–612 (2002)CrossRefGoogle Scholar
  46. 46.
    Turcajova, R., Kautsky, J.: A hierarchical multiresolution technique for image registration. In: Proceedings of SPIE Mathematical Imaging: Wavelet Applications in Signal and Image Processing (1996)Google Scholar
  47. 47.
    Ye, G.: Image Registration and Super-resolution Mosaicing. Ph.D. thesis, The University of New South Wales (2005)Google Scholar
  48. 48.
    Zhang, H., Yuan, F.: Vehicle tracking based on image alignment in aerial videos. In: EMMCVPR’07: Proceedings of the 6th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 295–302. Springer, Berlin (2007)Google Scholar
  49. 49.
    Zimmermann, K., Matas, J., Svoboda, T.: Tracking by an optimal sequence of linear predictors. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 677–692 (2009)CrossRefGoogle Scholar
  50. 50.
    Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Carol Martínez
    • 1
    Email author
  • Pascual Campoy
    • 1
  • Iván F. Mondragón
    • 2
  • José Luis Sánchez-Lopez
    • 1
  • Miguel A. Olivares-Méndez
    • 1
    • 3
  1. 1.Computer Vision GroupUniversidad Politécnica de Madrid, Centro de Automática y Robótica (CAR) UPM-CSICMadridSpain
  2. 2.Department of Industrial Engineering, Faculty of EngineeringPontificia Universidad JaverianaBogotá D.C.Colombia
  3. 3.Automation Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT)University of LuxembourgLuxembourgLuxembourg

Personalised recommendations