Skip to main content
SpringerLink
Log in

Wide-Area Persistent Airborne Video: Architecture and Challenges

  • Chapter
Distributed Video Sensor Networks

Abstract

The need for persistent video covering large geospatial areas using embedded camera networks and stand-off sensors has increased over the past decade. The availability of inexpensive, compact, light-weight, energy-efficient, high resolution optical sensors and associated digital image processing hardware has led to a new class of airborne surveillance platforms. Traditional tradeoffs posed between lens size and resolution, that is the numerical aperture of the system, can now be mitigated using an array of cameras mounted in a specific geometry. This fundamental advancement enables new imaging systems to cover very large fields of view at high resolution, albeit with spatially varying point spread functions. Airborne imaging systems capable of acquiring 88 megapixels per frame, over a wide field-of-view of 160 degrees or more at low frame rates of several hertz along with color sampling have been built using an optical array with up to eight cameras. These platforms fitted with accurate orientation sensors circle above an area of interest at constant altitude, adjusting steadily the orientation of the camera array fixed around a narrow area of interest, ideally locked to a point on the ground. The resulting image sequence maintains a persistent observation of an extended geographical area depending on the altitude of the platform and the configuration of the camera array. Suitably geo-registering and stabilizing these very large format videos provide a virtual nadir view of the region being monitored enabling a new class of urban scale activity analysis applications. The sensor geometry, processing challenges and scene interpretation complexities are highlighted.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adelson, E.H., Bergen, J.R.: The plenoptic function and elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing, pp. 3–20. MIT Press, Cambridge (1991)

    Google Scholar 

  2. Andrienko, G., Roberts, J.C., Weaver, C. (eds.): 5th Int. Conf. Coordinated & Multiple Views in Exploratory Visualization (2007)

    Google Scholar 

  3. Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Fux tensor constrained geodesic active contours with sensor fusion for persistent object tracking. J. Multimed. 2(4), 20–33 (2007)

    Google Scholar 

  4. Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Geodesic active contour based fusion of visible and infrared video for persistent object tracking. In: 8th IEEE Workshop Applications of Computer Vision (WACV 2007), Austin, TX (2007)

    Google Scholar 

  5. Chou, E.C., Iyengar, S.S., Seetharaman, G., Holyer, J., Lybanon, M.: Velocity vectors for features of sequential oceanographic images. IEEE Trans. Geosci. Remote Sens. 36(3), 985–998 (1998)

    Article  Google Scholar 

  6. Cornbleet, S.: Geometrical optics reviewed: A new light on an old subject. Proc. IEEE 71(4), 471–502 (1983)

    Article  Google Scholar 

  7. Easson, G., DeLozier, S., Momm, H.G.: Estimating speed and direction of small dynamic targets through optical satellite imaging. Remote Sens. 2, 1331–1347 (2010)

    Article  Google Scholar 

  8. Ertl, T.: Guest editor’s introduction: Special section on the IEEE symposium on visual analytics science and technology (vast). IEEE Trans. Vis. Comput. Graph. 16(2), 177 (2010)

    Article  Google Scholar 

  9. Fennell, M.T., Wishner, R.P.: Battlefield awareness via synergistic sar and mti exploitation. IEEE AES Syst. Mag. 39–45 (1998)

    Google Scholar 

  10. Hafiane, A., Palaniappan, K., Seetharaman, G.: UAV-video registration using block-based features. In: IEEE Int. Geoscience and Remote Sensing Symposium, vol. II, pp. 1104–1107 (2008)

    Google Scholar 

  11. Hafiane, A., Seetharaman, G., Palaniappan, K., Zavidovique, B.: Rotationally invariant hashing of median patterns for texture classification. In: Lecture Notes in Computer Science (ICIAR), vol. 5112, p. 619 (2008)

    Google Scholar 

  12. Hasler, A.F., Palaniappan, K., Manyin, M., Dodge, J.: A high performance interactive image spreadsheet (IISS). Comput. Phys. 8(4), 325–342 (1994)

    Article  Google Scholar 

  13. Hinz, S., Lenhart, D., Leitloff, J.: Detection and tracking of vehicles in low framerate aerial image sequences. In: Proc. Workshop on High-Resolution Earth Imaging for Geo-Spatial Information, Hannover, Germany (2007)

    Google Scholar 

  14. Kuthirummal, S., Nayar, S.K.: Multiview radial catadioptric imaging for scene capture. ACM Trans. Graph. (SIGGRAPH) 25(3), 916–923 (2006)

    Article  Google Scholar 

  15. Levoy, M.: Light fields and computational imaging. IEEE Comput. 46–55 (2006)

    Google Scholar 

  16. Massaro, J., Rao, R.M.: Ordering random object poses. In: IEEE Int. Conf. Acoustics, Speech and Signal Proc., pp. 1365–1368 (2009)

    Google Scholar 

  17. Nath, S.K., Palaniappan, K.: Adaptive robust structure tensors for orientation estimation and image segmentation. In: Lecture Notes in Computer Science (ISVC), vol. 3804, pp. 445–453 (2005)

    Google Scholar 

  18. Nayar, S.K.: Computational cameras: Redefining the image. IEEE Comput. Mag., Special Issue on Computational Photography, 30–38 (2006)

    Google Scholar 

  19. Nayar, S.K., Branzoi, V., Boult, T.E.: Programmable imaging: Towards a flexible camera. Int. J. Comput. Vis. 70(1), 7–22 (2006)

    Article  Google Scholar 

  20. Ni, K., Bresson, X., Chan, T., Esedoglu, S.: Local histogram based segmentation using the Wasserstein distance. Int. J. Comput. Vis. 84, 97–111 (2009)

    Article  Google Scholar 

  21. Palaniappan, K., Fraser, J.: Multiresolution tiling for interactive viewing of large datasets. In: Int. Conf. on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography and Hydrology, pp. 318–323. American Meteorological Society, Boston (2001)

    Google Scholar 

  22. Palaniappan, K., Uhlmann, J., Li, D.: Extensor based image interpolation. In: IEEE Int. Conf. Image Processing, vol. 2, pp. 945–948 (2003)

    Google Scholar 

  23. Palaniappan, K., Jiang, H.S., Baskin, T.I.: Non-rigid motion estimation using the robust tensor method. In: IEEE CVPR Workshop on Articulated and Nonrigid Motion, vol. 1, pp. 25–33, Washington DC, USA (2004)

    Google Scholar 

  24. Palaniappan, K., Ersoy, I., Nath, S.K.: Moving object segmentation using the flux tensor for biological video microscopy. In: Lecture Notes in Computer Science (PCM), vol. 4810, pp. 483–493 (2007)

    Google Scholar 

  25. Palaniappan, K., Bunyak, F., Kumar, P., Ersoy, I., Jaeger, S., Ganguli, K., Haridas, A., Fraser, J., Rao, R., Seetharaman, G.: Efficient feature extraction and likelihood fusion for vehicle tracking in low frame rate airborne video. In: 13th Int. Conf. Information Fusion (2010)

    Google Scholar 

  26. Ramachandran, M., Veeraraghavan, A., Chellappa, R.: Video stabilization and mosaicing. In: Bovik, A. (ed.) The Essential Guide to Video Processing, 2nd edn., pp. 109–138. Academic Press, Elsevier, New York (2008)

    Google Scholar 

  27. Regazzoni, C.S., Cavallaro, A., Porikli, F.: Video tracking in complex scenes for surveillance applications. EURASIP J. Image Video Process. (Special Issue), 1–2 (2008)

    Google Scholar 

  28. Rosenbaum, D., Kurz, F., Thomas, U., Suri, S., Reinartz, P.: Towards automatic near real-time traffic monitoring with an airborne wide angle camera system. Eur. Transp. Res. Rev. 1, 11–21 (2009)

    Article  Google Scholar 

  29. Sankaranarayanan, A.C., Veeraraghavan, A., Chellappa, R.: Object detection, tracking and recognition for multiple smart cameras. Proc. IEEE 96(10), 1606–1624 (2008)

    Article  Google Scholar 

  30. Seetharaman, G.: Three dimensional perception of image sequences. In: Young, T.Y. (ed.) Handbook of Computer Vision. Academic Press, San Diego (1994)

    Google Scholar 

  31. Seetharaman, G., Bao, H., Shivaram, G.: Calibration of camera parameters using vanishing points. J. Franklin Inst. 331(5), 555–585 (1994)

    Article  Google Scholar 

  32. Seetharaman, G., Gasperas, G., Palaniappan, K.: A piecewise affine model for image registration in 3-D motion analysis. In: IEEE Int. Conf. Image Processing, pp. 561–564 (2000)

    Google Scholar 

  33. Szeliski, R.: Image alignment and stitching. In: Paragios, N. (ed.) Handbook of Mathematical Models in Computer Vision, pp. 273–292. Springer, Berlin (2005)

    Google Scholar 

  34. Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Berlin (2010)

    Google Scholar 

  35. Weng, J., Huang, T.S., Ahuja, N.: Motion and Structure from Image Sequences. Springer, Berlin (1991)

    Google Scholar 

  36. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4), A13 (2006)

    Article  Google Scholar 

  37. Yue, Z., Guarino, D., Chellappa, R.: Moving object verification in airborne video sequences. IEEE Trans. Circuits Syst. Video Technol. 19(1), 77–89 (2009)

    Article  Google Scholar 

  38. Zhou, L., Kambhamettu, C., Goldgof, D., Palaniappan, K., Hasler, A.F.: Tracking non-rigid motion and structure from 2D satellite cloud images without correspondences. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1330–1336 (2001)

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Dr. Ross McNutt of PSS for providing the wide-area imagery used in this paper, Dr. Filiz Bunyak for various discussions and producing the figures related to spatio-temporal reflectance variations, and Joshua Fraser for creating the Maya-based rendering of the airborne imaging platform flight path and geometry. A new version of the Kolam software tool to support visualization of wide-area airborne video was developed by Joshua Fraser and Anoop Haridas and used for preparing the figures showing imagery in the paper. Matlab mex files to access PSS imagery especially for tracking was contributed by Ilker Ersoy as well as managing the collection of WAMI data sets. This research was partially supported by grants from the Leonard Wood Institute (LWI 181223) in cooperation with the U.S. Army Research Laboratory (ARL) under Cooperative Agreement Number W911NF-07-2-0062, and the U.S. Air Force Research Laboratory (AFRL) under agreements FA8750-09-2-0198, FA8750-10-1-0182. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied of LWI, ARL, AFRL or the U.S. Government. This document has been cleared for public release under case number 88ABW-2010-2725. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon.

Author information

Authors and Affiliations

  1. University of Missouri, Columbia, MO, 65211, USA

    Kannappan Palaniappan

  2. Army Research Laboratory, Adelphi, MD, 20783, USA

    Raghuveer M. Rao

  3. Air Force Research Laboratory, Rome, NY, 13441, USA

    Guna Seetharaman

Authors
  1. Kannappan Palaniappan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raghuveer M. Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guna Seetharaman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kannappan Palaniappan .

Editor information

Editors and Affiliations

  1. Center for Research, Intelligent Systems, University of California, Riverside, Riverside, 92521, California, USA

    Bir Bhanu

  2. Dept. Computer Science & Engineering, University of California, Riverside, Riverside, 92521, California, USA

    Chinya V. Ravishankar

  3. Dept. Electrical Engineering, University of California, Riverside, Riverside, 92521, California, USA

    Amit K. Roy-Chowdhury

  4. Dept. Electrical Engineering, Packard Bldg., Stanford University, Serra Mall 350, Stanford, 94305-9505, California, USA

    Hamid Aghajan

  5. Dept. Computer Science, University of California, Los Angeles, Boelter Hall 4731, Los Angeles, 90095-1596, California, USA

    Demetri Terzopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Palaniappan, K., Rao, R.M., Seetharaman, G. (2011). Wide-Area Persistent Airborne Video: Architecture and Challenges. In: Bhanu, B., Ravishankar, C., Roy-Chowdhury, A., Aghajan, H., Terzopoulos, D. (eds) Distributed Video Sensor Networks. Springer, London. https://doi.org/10.1007/978-0-85729-127-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-127-1_24

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-126-4

  • Online ISBN: 978-0-85729-127-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation