Wide-Area Persistent Airborne Video: Architecture and Challenges

  • Kannappan PalaniappanEmail author
  • Raghuveer M. Rao
  • Guna Seetharaman


The need for persistent video covering large geospatial areas using embedded camera networks and stand-off sensors has increased over the past decade. The availability of inexpensive, compact, light-weight, energy-efficient, high resolution optical sensors and associated digital image processing hardware has led to a new class of airborne surveillance platforms. Traditional tradeoffs posed between lens size and resolution, that is the numerical aperture of the system, can now be mitigated using an array of cameras mounted in a specific geometry. This fundamental advancement enables new imaging systems to cover very large fields of view at high resolution, albeit with spatially varying point spread functions. Airborne imaging systems capable of acquiring 88 megapixels per frame, over a wide field-of-view of 160 degrees or more at low frame rates of several hertz along with color sampling have been built using an optical array with up to eight cameras. These platforms fitted with accurate orientation sensors circle above an area of interest at constant altitude, adjusting steadily the orientation of the camera array fixed around a narrow area of interest, ideally locked to a point on the ground. The resulting image sequence maintains a persistent observation of an extended geographical area depending on the altitude of the platform and the configuration of the camera array. Suitably geo-registering and stabilizing these very large format videos provide a virtual nadir view of the region being monitored enabling a new class of urban scale activity analysis applications. The sensor geometry, processing challenges and scene interpretation complexities are highlighted.


Wide-area motion imagery Wide field-of-view sensors Very large format video Persistent surveillance Camera sensor arrays High numerical aperture optics Airborne imaging 



The authors wish to thank Dr. Ross McNutt of PSS for providing the wide-area imagery used in this paper, Dr. Filiz Bunyak for various discussions and producing the figures related to spatio-temporal reflectance variations, and Joshua Fraser for creating the Maya-based rendering of the airborne imaging platform flight path and geometry. A new version of the Kolam software tool to support visualization of wide-area airborne video was developed by Joshua Fraser and Anoop Haridas and used for preparing the figures showing imagery in the paper. Matlab mex files to access PSS imagery especially for tracking was contributed by Ilker Ersoy as well as managing the collection of WAMI data sets. This research was partially supported by grants from the Leonard Wood Institute (LWI 181223) in cooperation with the U.S. Army Research Laboratory (ARL) under Cooperative Agreement Number W911NF-07-2-0062, and the U.S. Air Force Research Laboratory (AFRL) under agreements FA8750-09-2-0198, FA8750-10-1-0182. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied of LWI, ARL, AFRL or the U.S. Government. This document has been cleared for public release under case number 88ABW-2010-2725. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon.


  1. 1.
    Adelson, E.H., Bergen, J.R.: The plenoptic function and elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing, pp. 3–20. MIT Press, Cambridge (1991) Google Scholar
  2. 2.
    Andrienko, G., Roberts, J.C., Weaver, C. (eds.): 5th Int. Conf. Coordinated & Multiple Views in Exploratory Visualization (2007) Google Scholar
  3. 3.
    Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Fux tensor constrained geodesic active contours with sensor fusion for persistent object tracking. J. Multimed. 2(4), 20–33 (2007) Google Scholar
  4. 4.
    Bunyak, F., Palaniappan, K., Nath, S.K., Seetharaman, G.: Geodesic active contour based fusion of visible and infrared video for persistent object tracking. In: 8th IEEE Workshop Applications of Computer Vision (WACV 2007), Austin, TX (2007) Google Scholar
  5. 5.
    Chou, E.C., Iyengar, S.S., Seetharaman, G., Holyer, J., Lybanon, M.: Velocity vectors for features of sequential oceanographic images. IEEE Trans. Geosci. Remote Sens. 36(3), 985–998 (1998) CrossRefGoogle Scholar
  6. 6.
    Cornbleet, S.: Geometrical optics reviewed: A new light on an old subject. Proc. IEEE 71(4), 471–502 (1983) CrossRefGoogle Scholar
  7. 7.
    Easson, G., DeLozier, S., Momm, H.G.: Estimating speed and direction of small dynamic targets through optical satellite imaging. Remote Sens. 2, 1331–1347 (2010) CrossRefGoogle Scholar
  8. 8.
    Ertl, T.: Guest editor’s introduction: Special section on the IEEE symposium on visual analytics science and technology (vast). IEEE Trans. Vis. Comput. Graph. 16(2), 177 (2010) CrossRefGoogle Scholar
  9. 9.
    Fennell, M.T., Wishner, R.P.: Battlefield awareness via synergistic sar and mti exploitation. IEEE AES Syst. Mag. 39–45 (1998) Google Scholar
  10. 10.
    Hafiane, A., Palaniappan, K., Seetharaman, G.: UAV-video registration using block-based features. In: IEEE Int. Geoscience and Remote Sensing Symposium, vol. II, pp. 1104–1107 (2008) Google Scholar
  11. 11.
    Hafiane, A., Seetharaman, G., Palaniappan, K., Zavidovique, B.: Rotationally invariant hashing of median patterns for texture classification. In: Lecture Notes in Computer Science (ICIAR), vol. 5112, p. 619 (2008) Google Scholar
  12. 12.
    Hasler, A.F., Palaniappan, K., Manyin, M., Dodge, J.: A high performance interactive image spreadsheet (IISS). Comput. Phys. 8(4), 325–342 (1994) CrossRefGoogle Scholar
  13. 13.
    Hinz, S., Lenhart, D., Leitloff, J.: Detection and tracking of vehicles in low framerate aerial image sequences. In: Proc. Workshop on High-Resolution Earth Imaging for Geo-Spatial Information, Hannover, Germany (2007) Google Scholar
  14. 14.
    Kuthirummal, S., Nayar, S.K.: Multiview radial catadioptric imaging for scene capture. ACM Trans. Graph. (SIGGRAPH) 25(3), 916–923 (2006) CrossRefGoogle Scholar
  15. 15.
    Levoy, M.: Light fields and computational imaging. IEEE Comput. 46–55 (2006) Google Scholar
  16. 16.
    Massaro, J., Rao, R.M.: Ordering random object poses. In: IEEE Int. Conf. Acoustics, Speech and Signal Proc., pp. 1365–1368 (2009) Google Scholar
  17. 17.
    Nath, S.K., Palaniappan, K.: Adaptive robust structure tensors for orientation estimation and image segmentation. In: Lecture Notes in Computer Science (ISVC), vol. 3804, pp. 445–453 (2005) Google Scholar
  18. 18.
    Nayar, S.K.: Computational cameras: Redefining the image. IEEE Comput. Mag., Special Issue on Computational Photography, 30–38 (2006) Google Scholar
  19. 19.
    Nayar, S.K., Branzoi, V., Boult, T.E.: Programmable imaging: Towards a flexible camera. Int. J. Comput. Vis. 70(1), 7–22 (2006) CrossRefGoogle Scholar
  20. 20.
    Ni, K., Bresson, X., Chan, T., Esedoglu, S.: Local histogram based segmentation using the Wasserstein distance. Int. J. Comput. Vis. 84, 97–111 (2009) CrossRefGoogle Scholar
  21. 21.
    Palaniappan, K., Fraser, J.: Multiresolution tiling for interactive viewing of large datasets. In: Int. Conf. on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography and Hydrology, pp. 318–323. American Meteorological Society, Boston (2001) Google Scholar
  22. 22.
    Palaniappan, K., Uhlmann, J., Li, D.: Extensor based image interpolation. In: IEEE Int. Conf. Image Processing, vol. 2, pp. 945–948 (2003) Google Scholar
  23. 23.
    Palaniappan, K., Jiang, H.S., Baskin, T.I.: Non-rigid motion estimation using the robust tensor method. In: IEEE CVPR Workshop on Articulated and Nonrigid Motion, vol. 1, pp. 25–33, Washington DC, USA (2004) Google Scholar
  24. 24.
    Palaniappan, K., Ersoy, I., Nath, S.K.: Moving object segmentation using the flux tensor for biological video microscopy. In: Lecture Notes in Computer Science (PCM), vol. 4810, pp. 483–493 (2007) Google Scholar
  25. 25.
    Palaniappan, K., Bunyak, F., Kumar, P., Ersoy, I., Jaeger, S., Ganguli, K., Haridas, A., Fraser, J., Rao, R., Seetharaman, G.: Efficient feature extraction and likelihood fusion for vehicle tracking in low frame rate airborne video. In: 13th Int. Conf. Information Fusion (2010) Google Scholar
  26. 26.
    Ramachandran, M., Veeraraghavan, A., Chellappa, R.: Video stabilization and mosaicing. In: Bovik, A. (ed.) The Essential Guide to Video Processing, 2nd edn., pp. 109–138. Academic Press, Elsevier, New York (2008) Google Scholar
  27. 27.
    Regazzoni, C.S., Cavallaro, A., Porikli, F.: Video tracking in complex scenes for surveillance applications. EURASIP J. Image Video Process. (Special Issue), 1–2 (2008) Google Scholar
  28. 28.
    Rosenbaum, D., Kurz, F., Thomas, U., Suri, S., Reinartz, P.: Towards automatic near real-time traffic monitoring with an airborne wide angle camera system. Eur. Transp. Res. Rev. 1, 11–21 (2009) CrossRefGoogle Scholar
  29. 29.
    Sankaranarayanan, A.C., Veeraraghavan, A., Chellappa, R.: Object detection, tracking and recognition for multiple smart cameras. Proc. IEEE 96(10), 1606–1624 (2008) CrossRefGoogle Scholar
  30. 30.
    Seetharaman, G.: Three dimensional perception of image sequences. In: Young, T.Y. (ed.) Handbook of Computer Vision. Academic Press, San Diego (1994) Google Scholar
  31. 31.
    Seetharaman, G., Bao, H., Shivaram, G.: Calibration of camera parameters using vanishing points. J. Franklin Inst. 331(5), 555–585 (1994) CrossRefGoogle Scholar
  32. 32.
    Seetharaman, G., Gasperas, G., Palaniappan, K.: A piecewise affine model for image registration in 3-D motion analysis. In: IEEE Int. Conf. Image Processing, pp. 561–564 (2000) Google Scholar
  33. 33.
    Szeliski, R.: Image alignment and stitching. In: Paragios, N. (ed.) Handbook of Mathematical Models in Computer Vision, pp. 273–292. Springer, Berlin (2005) Google Scholar
  34. 34.
    Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, Berlin (2010) Google Scholar
  35. 35.
    Weng, J., Huang, T.S., Ahuja, N.: Motion and Structure from Image Sequences. Springer, Berlin (1991) Google Scholar
  36. 36.
    Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4), A13 (2006) CrossRefGoogle Scholar
  37. 37.
    Yue, Z., Guarino, D., Chellappa, R.: Moving object verification in airborne video sequences. IEEE Trans. Circuits Syst. Video Technol. 19(1), 77–89 (2009) CrossRefGoogle Scholar
  38. 38.
    Zhou, L., Kambhamettu, C., Goldgof, D., Palaniappan, K., Hasler, A.F.: Tracking non-rigid motion and structure from 2D satellite cloud images without correspondences. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1330–1336 (2001) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Kannappan Palaniappan
    • 1
    Email author
  • Raghuveer M. Rao
    • 2
  • Guna Seetharaman
    • 3
  1. 1.University of MissouriColumbiaUSA
  2. 2.Army Research LaboratoryAdelphiUSA
  3. 3.Air Force Research LaboratoryRomeUSA

Personalised recommendations