Journal of Intelligent & Robotic Systems

, Volume 61, Issue 1–4, pp 119–134 | Cite as

A Catadioptric and Pan-Tilt-Zoom Camera Pair Object Tracking System for UAVs

  • Metin Tarhan
  • Erdinç AltuğEmail author


Unmanned aerial vehicles (UAVs) are seeing widespread use in military, scientific, and civilian sectors in recent years. As the mission demands increase, these systems are becoming more complicated. Omnidirectional camera is a vision sensor that can captures 360° view in a single frame. In recent years omnidirectional camera usage has experienced a remarkable increase in many fields, where many innovative research has been done. Although, it is very promising, employment of omnidirectional cameras in UAVs is quite new. In this paper, an innovative sensory system is proposed, that has an omnidirectional imaging device and a pan tilt zoom (PTZ) camera. Such a system combines the advantages of both of the camera systems. The system can track any moving object within its 360° field of view and provide detailed images of it. The detection of the moving object has been accomplished by an adaptive background subtraction method implemented on the lowered resolution images of the catadioptric camera. A novel algorithm has also been developed to estimate the relative distance of the object with respect to the UAV, using tracking information of both of the cameras. The algorithms are implemented on an experimental system to validate the approach.


Unmanned aerial vehicles Catadioptric vision Tracking Surveillance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Castillo, P., Lozano, R., Dzul, A.E.: Modelling and Control of Mini-Flying Machines. Springer, London (2005)Google Scholar
  2. 2.
    Winters, N., Gaspar, J., Lacey, G., Santos-Victor, J.: Omni-directional vision for robot navigation. In: Proc. of IEEE Workshop on Omnidirectional Vision. Hilton Head Island, SC (2000)Google Scholar
  3. 3.
    Fiala, M., Basu A.: Robot navigation using panoramic tracking. Pattern Recogn. 37, 2195–2215 (2004)CrossRefGoogle Scholar
  4. 4.
    Gava, C.C., Vassallo, R.F., Roberti, F., Carelli, C., Bastos-Filho, T.F.: Nonlinear control techniques and omnidirectional vision for team formation on cooperative robotics. In: IEEE International Conference on Robotics and Automation, pp. 2409–2414. Roma, Italy (2007)Google Scholar
  5. 5.
    Sun, Y., Cao, Q., Chen, W.: An object tracking and global localization method using omnidirectional vision system. In: Proceedings of the 5th World Congress on Intelligent Control and Automation, pp. 4730–4735. Hangzhou, Peoples Republic of China (2004)Google Scholar
  6. 6.
    Jamzad, M., Hadjkhodabakhshi, A.R., Mirrokni, V.S.: Object detection and localization using omnidirectional vision in the robocup environment. Sci. Iran. 14(6), 599–611 (2007)Google Scholar
  7. 7.
    Chen, Z., Chen, Y., Wang, Y.: Image-based control of an autonomous robot with omnidirectional vision. In: IEEE International Symposium on Industrial Embedded Systems. Lisbon (2007)Google Scholar
  8. 8.
    J. Gluckman, S. K. Nayar: Ego-motion and omnidirectional cameras. In: Proc. International Conference on Computer Vision (ICCV98), pp. 999–1005 (1998)Google Scholar
  9. 9.
    Bruss, A., Horn, B.K.P.: Passive navigation. Comput. Vis. Graph. Image Process. 21, 3–20 (1983)CrossRefGoogle Scholar
  10. 10.
    Hrabar, S., Sukhatme, G.S.: Omnidirectional vision for an autonomous helicopter. IEEE Int. Conf. Robot. Autom. 1, 558–563 (2003)Google Scholar
  11. 11.
    Hrabar, S., Sukhatme, G.S.: A comparison of two camera configurations for optic-flow based navigation of a UAV through urban canyons. In: Proceedings of the International Conference on Intelligent Robots and Systems, pp. 2673–2680 (2004)Google Scholar
  12. 12.
    Demonceaux C., Vasseur P., Pégard C.: Robust attitude estimation with catadioptric vision. In: Proceedings of the International Conference on Intelligent Robots and Systems. Beijing, China (2006)Google Scholar
  13. 13.
    Demonceaux C., Vasseur P., Pégard C.: Omnidirectional vision on UAV for attitude computation. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, pp. 2842–2847 (2006)Google Scholar
  14. 14.
    Bazin, J.C., Kweon, I.S., Demonceaux, C., Vasseur, P.: UAV attitude estimation by combining horizon-based and homography-based approaches for catadioptric image. In: 6th IFAC/EURON Intelligent Autonomous Vehicles. Toulouse, France (2007)Google Scholar
  15. 15.
    Demonceaux, C., Vasseur, P., Pégard, C.: UAV attitude computation by omnidirectional vision in urban environment. IEEE Int. Conf. Robot. Autom. (2007). doi: 10.1109/ROBOT.2007.363618 Google Scholar
  16. 16.
    Bazin, J.C., Kweon, I., Demonceaux, C., Vasseur, P.: UAV attitude estimation by vanishing points in catadioptric images. In: IEEE International Conference on Robotics and Automation, pp. 2743–2749. Pasadena, CA (2008)CrossRefGoogle Scholar
  17. 17.
    Tarhan, M., Altuğ, E.: Control of a quadrotor air vehicle by vanishing points in catadioptric images. In: International Symposium on Optomechatronic Technologies (ISOT). Istanbul Turkey (2009)Google Scholar
  18. 18.
    Tarhan, M., Altuğ, E.: EKF based attitude estimation and stabilization of a quadrotor UAV using vanishing points in catadioptric images. J. Intell. Robot. Syst. (2010). doi: 10.1007/s10846-010-9459-y Google Scholar
  19. 19.
    Sato, Y., Hashimoto, K., Shibata, Y.: A new networked surveillance video system by combination of omni-directional and network controlled cameras. In: Network-Based Information Systems, vol. 5186. Springer, Berlin (2008)Google Scholar
  20. 20.
    Scotti, G., Marcenaro, L., Coelho, C., Selvaggi, F., Regazzoni, C.S.: Dual camera intelligent sensor for high definition 360 degrees surveillance. IEE Proc., Vis. Image Signal Process. 152, 250–257 (2005)CrossRefGoogle Scholar
  21. 21.
    Yao, Y., Abidi, B., Abidi M.: Fusion of omnidirectional and PTZ cameras for accurate cooperative tracking. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS’06) (2006)Google Scholar
  22. 22.
    Chen, C.H., Yao, Y., Page, D., Abidi, B., Koschan, A., Abidi, M.: Heterogeneous fusion of omnidirectional and PTZ cameras for multiple object tracking. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1052–1063 (2008)CrossRefGoogle Scholar
  23. 23.
    Elhabian, Y., El-Sayed, K.M., Ahmed, S.H.: Moving object detection in spatial domain using background removal techniques—state-of-art. Recent Pat. Comput. Sci. 1, 32–54 (2008)CrossRefGoogle Scholar
  24. 24.
    Sugandi, B., Kim, H., Tan, J.K., Ishikawa, S.: Tracking of moving objects by using a low resolution image. In: Second International Conference on Innovative Computing, Information and Control, ICICIC (2007)Google Scholar
  25. 25.
    Scotti, G., Cuocolo, A., Coelho, C., Marchesotti, L.: A novel pedestrian classification algorithm for a high definition dual camera 360 degrees surveillance system. In: IEEE International Conference on Image Processing, ICIP (2005)Google Scholar
  26. 26.
  27. 27.
  28. 28.
    Barreto, J.P., Araujo, H.: Geometric properties of central catadioptric line images and their application in calibration. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1327–1333 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.System Dynamics and Control Graduate ProgramIstanbul Technical UniversityIstanbulTurkey
  2. 2.Department of Mechanical EngineeringIstanbul Technical UniversityIstanbulTurkey

Personalised recommendations