Journal of Signal Processing Systems

, Volume 61, Issue 3, pp 293–315 | Cite as

Improving the Performance of Hyperspectral Image and Signal Processing Algorithms Using Parallel, Distributed and Specialized Hardware-Based Systems

  • Antonio Plaza
  • Javier Plaza
  • Hugo Vegas


Advances in sensor technology are revolutionizing the way remotely sensed data is collected, managed and analyzed. The incorporation of latest-generation sensors to airborne and satellite platforms is currently producing a nearly continual stream of high-dimensional data, and this explosion in the amount of collected information has rapidly created new processing challenges. For instance, hyperspectral signal processing is a new technique in remote sensing that generates hundreds of spectral bands at different wavelength channels for the same area on the surface of the Earth. Many current and future applications of remote sensing in Earth science, space science, and soon in exploration science will require (near) real-time processing capabilities. In recent years, several efforts have been directed towards the incorporation of high-performance computing (HPC) systems and architectures in remote sensing missions. With the aim of providing an overview of current and new trends in parallel and distributed systems for remote sensing applications, this paper explores three HPC-based paradigms for efficient implementation of the Pixel Purity Index (PPI) algorithm, available from the popular Kodak’s Research Systems ENVI software package, as a representative case study for demonstration purposes. Several different parallel programming techniques are used to improve the performance of the PPI on a variety of parallel platforms, including a set of message passing interface (MPI)-based implementations on a massively parallel Beowulf cluster at NASA’s Goddard Space Flight Center in Maryland and on a variety of heterogeneous networks of workstations at University of Maryland; a Handel-C implementation of the algorithm on a Virtex-II field programmable gate array (FPGA); and a compute unified device architecture (CUDA)-based implementation on graphical processing units (GPUs) of NVidia. Combined, these parts deliver an excellent snapshot of the state-of-the-art in those areas, and offer a thoughtful perspective on the potential and emerging challenges of adapting HPC systems to remote sensing problems.


Parallel systems Hyperspectral imaging Cluster computer systems Heterogeneous parallel systems FPGAs GPUs 



This work has been supported by the European Community’s Marie Curie Research Training Networks Programme under reference MRTN-CT-2006-035927, Hyperspectral Imaging Network (HYPER-I-NET). This work has also been supported by the Spanish Ministry of Science and Innovation (HYPERCOMP/EODIX project, reference AYA2008-05965-C04-02). Funding from Junta de Extremadura (local government) under project PRI09A110 is also gratefully acknowledged. The authors gratefully thank Profs. Manuel Prieto and Franciscto Tirado and their group at ComplutenseUniversity of Madrid for many fruitful discussions and for his collaboration in the generation of some of the figures presented in the paper.


  1. 1.
    Goetz, A.F.H., Vane, G., Solomon, J.E., & Rock, B.N. (1985). Imaging spectrometry for Earth remote sensing. Science, 228, 1147–1153.CrossRefGoogle Scholar
  2. 2.
    Schowengerdt, R.A. (2007). Remote sensing, 3rd edn. New York: Academic.Google Scholar
  3. 3.
    Green, R.O., et al. (1998). Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS). Remote Sensing of Environment, 65, 227–248.CrossRefGoogle Scholar
  4. 4.
    Chang, C.-I. (2003). Hyperspectral imaging: Techniques for spectral detection and classification. New York: Kluwer Academic.Google Scholar
  5. 5.
    Crookes, D. (1999). Architectures for high performance image processing: The future. Journal of Systems Architecture, 45, 739–748.CrossRefGoogle Scholar
  6. 6.
    Jeon, J., Kim, H.-S., Choi, G., & Park, H. (2000). KAIST image computing system (KICS): A parallel architecture for real-time multimedia data processing. Journal of Systems Architecture, 46, 1403–1418.CrossRefGoogle Scholar
  7. 7.
    Chen, L., Fujishiro, I., & Nakajima, K. (2003). Optimizing parallel performance of unstructured volume rendering for the Earth simulator. Parallel Computing, 29, 355–371.CrossRefGoogle Scholar
  8. 8.
    Aloisio, G., & Cafaro, M. (2003). A dynamic earth observation system. Parallel Computing, 29, 1357–1362.CrossRefGoogle Scholar
  9. 9.
    Hawick, K.A., Coddington, P.D., & James, H.A. (2003). Distributed frameworks and parallel algorithms for processing large-scale geographic data. Parallel Computing, 29, 1297–1333.CrossRefGoogle Scholar
  10. 10.
    Wang, P., Liu, K.Y., Cwik, T., & Green, R.O. (2002). MODTRAN on supercomputers and parallel computers. Parallel Computing, 28, 53–64.zbMATHCrossRefGoogle Scholar
  11. 11.
    Tehranian, S., Zhao, Y., Harvey, T., Swaroop, A., & McKenzie, K. (2006). A robust framework for real-time distributed processing of satellite data. Journal of Parallel and Distributed Computing, 66, 403–418.CrossRefGoogle Scholar
  12. 12.
    Plaza, A., Valencia, D., Plaza, J., & Martinez, P. (2006). Commodity cluster-based parallel processing of hyperspectral imagery. Journal of Parallel and Distributed Computing, 66(3), 345–358.zbMATHCrossRefGoogle Scholar
  13. 13.
    Landgrebe, D.A. (2003). Signal theory methods in multispectral remote sensing. Hoboken: Wiley.CrossRefGoogle Scholar
  14. 14.
    Dorband, J., Palencia, J., Ranawake, U. (2003). Commodity computing clusters at Goddard Space Flight Center. Journal of Space Communication, 1(3), 113–123. Available online: Scholar
  15. 15.
    Brightwell, R., Fisk, L.A., Greenberg, D.S., Hudson, T., Levenhagen, M., Maccabe, A.B., et al. (2000). Massively parallel computing using commodity components. Parallel Computing, 26, 243–266.zbMATHCrossRefGoogle Scholar
  16. 16.
    Kalluri, S., Zhang, Z., JaJa, J., Liang, S., & Townshend, J. (2001). Characterizing land surface anisotropy from AVHRR data at a global scale using high performance computing. International Journal of Remote Sensing, 22, 2171–2191.CrossRefGoogle Scholar
  17. 17.
    Tilton, J.C. (2005). Method for implementation of recursive hierarchical segmentation on parallel computers. U.S. Patent Office, Washington, DC, U.S. Pending Published Application 09/839147, 2005. Available online:
  18. 18.
    Le Moigne, J., Campbell, W.J., & Cromp, R.F. (2002). An automated parallel image registration technique based on the correlation of wavelet features. IEEE Transactions on Geoscience and Remote Sensing, 40, 1849–1864.CrossRefGoogle Scholar
  19. 19.
    Achalakul, T., & Taylor, S. (2003). A distributed spectral-screening PCT algorithm. Journal of Parallel and Distributed Computing, 63, 373–384.zbMATHCrossRefGoogle Scholar
  20. 20.
    Lastovetsky, A. (2003). Parallel computing on heterogeneous networks. Hoboken: Wiley-Interscience.zbMATHCrossRefGoogle Scholar
  21. 21.
    Dhodhi, M.K., Saghri, J.A., Ahmad, I., & Ul-Mustafa, R. (1999). D-ISODATA: A distributed algorithm for unsupervised classification of remotely sensed data on network of workstations. Journal of Parallel and Distributed Computing, 59, 280–301.CrossRefGoogle Scholar
  22. 22.
    Hawick, K., James, H., Silis, A., Grove, D., Pattern, C., Mathew, J., et al. (1999). DISCWorld: An environment for service-based meta-computing. Future Generation Computer Systems, 15, 623–635.CrossRefGoogle Scholar
  23. 23.
    Foster, I., & Kesselman, C. (1999). The grid: Blueprint for a new computing infrastructure. San Francisco: Morgan Kaufman.Google Scholar
  24. 24.
    Vladimirova, T., & Wu, X. (2006). On-board partial run-time reconfiguration for pico-satellite constellations. First NASA/ESA Conference on Adaptive Hardware and Systems, AHS.Google Scholar
  25. 25.
    El-Araby, E., El-Ghazawi, T., & Le Moigne, J. (2004). Wavelet spectral dimension reduction of hyperspectral imagery on a reconfigurable computer. In Proceedings of the 4th IEEE international conference on field-programmable technology.Google Scholar
  26. 26.
    Valencia, D., & Plaza, A. (2006). FPGA-based compression of hyperspectral imagery using spectral unmixing and the pixel purity index algorithm. Lecture Notes in Computer Science, 3993, 24–31.CrossRefGoogle Scholar
  27. 27.
    Setoain, J., Prieto, M., Tenllado, C., Plaza, A., & Tirado, F. (2007). Parallel morphological endmember extraction using commodity grahics hardware. IEEE Geoscience and Remote Sensing Letters, 43, 441–445.CrossRefGoogle Scholar
  28. 28.
    Boardman, J., Kruse, F.A., & Green, R.O. (1995). Mapping target signatures via partial unmixing of AVIRIS data. Summaries of the NASA/JPL Airborne Earth Science Workshop, Pasadena, CA.Google Scholar
  29. 29.
    ITT Visual Information Solutions. (2008). ENVI users guide. Boulder: ITTVIS. Online:
  30. 30.
    Plaza, A., & Chang, C.-I. (2007). High performance computing in remote sensing. Boca Raton: CRC.CrossRefGoogle Scholar
  31. 31.
    Plaza, A., Martinez, P., Perez, R., & Plaza, J. (2002). Spatial-spectral endmember extraction by multidimensional morphological operations. IEEE Transactions on Geoscience and Remote Sensing, 40, 2025–2041.CrossRefGoogle Scholar
  32. 32.
    Plaza, A., Plaza, J., & Valencia, D. (2007). Impact of platform heterogeneity on the design of parallel algorithms for morphological processing of high-dimensional image data. Journal of Supercomputing, 40, 81–107.CrossRefGoogle Scholar
  33. 33.
    Plaza, A., Plaza, J., & Valencia, D. (2006). AMEEPAR: Parallel morphological algorithm for hyperspectral image classification in heterogeneous networks of workstations. Lecture Notes in Computer Science, 3391, 888–891.Google Scholar
  34. 34.
    Plaza, A., Martinez, P., Perez, R., & Plaza, J. (2004). A quantitative and comparative analysis of endmember extraction algorithms from hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 42, 650–663.CrossRefGoogle Scholar
  35. 35.
    Chang, C.-I., & Plaza, A. (2006). A fast iterative implementation of the pixel purity index algorithm. IEEE Geoscience and Remote Sensing Letters, 3, 63–67.CrossRefGoogle Scholar
  36. 36.
    Plaza, A., & Chang, C.-I. (2006). Impact of initialization on design of endmember extraction algorithms. IEEE Transactions on Geoscience and Remote Sensing, 44, 3397–3407.CrossRefGoogle Scholar
  37. 37.
    Keshava, N., & Mustard, J.F. (2002). Spectral unmixing. IEEE Signal Processing Magazine, 19, 44–57.CrossRefGoogle Scholar
  38. 38.
    Seinstra, F.J., Koelma, D., & Geusebroek, J.M. (2002). A software architecture for user transparent parallel image processing. Parallel Computing, 28, 967–993.zbMATHCrossRefGoogle Scholar
  39. 39.
    Veeravalli, B., & Ranganath, S. (2003). Theoretical and experimental study on large size image processing applications using divisible load paradigm on distributed bus networks. Image and Vision Computing, 20, 917–935.CrossRefGoogle Scholar
  40. 40.
    Gropp, W., Huss-Lederman, S., Lumsdaine, A., & Lusk, E. (1999). In MPI-—the complete reference, the MPI extensions (Vol. 2). Cambridge: MIT.Google Scholar
  41. 41.
    Lastovetsky, A., & Reddy, R. (2006). HeteroMPI: Towards a message-passing library for heterogeneous networks of computers. Journal of Parallel and Distributed Computing, 66, 197–220.zbMATHCrossRefGoogle Scholar
  42. 42.
    MPICH Team. (2005). MPICH: A portable implementation of MPI. Available online:
  43. 43.
    Kung, S.Y. (1988). VLSI array processors. Upper Saddle River: Prentice-Hall.Google Scholar
  44. 44.
    Petkov, N. (1992). Systolic parallel processing. The Netherlands: North Holland.Google Scholar
  45. 45.
    Lavernier, D., Fabiani, E., Derrien, S., & Wagner, C. (1999). Systolic array for computing the pixel purity index algorithm on hyperspectral images. In Proceedings of SPIE (Vol. 4480, pp. 130–138).Google Scholar
  46. 46.
    Celoxica Ltd., (2003). Handel-C language reference manual.Google Scholar
  47. 47.
    Celoxica Ltd. (2003). DK design suite user manual. Available online:
  48. 48.
    Xilinx Inc. (2010). Available online: (last check: January 2010).
  49. 49.
    Owens, J.D., Luebke, D., Govindaraju, N., Harris, M., Kruger, J., Lefohn, A.E., et al. (2005). A survey of general purpose computation on graphics hardware. In Proceedings of Eurographics: State of the art reports (pp. 21–51).Google Scholar
  50. 50.
    Montrym, J., Moreton, H. (2005). The GeForce 68000. IEEE Micro, 25, 41–51.CrossRefGoogle Scholar
  51. 51.
    Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M., et al. (2004). Brook for GPUs: Stream computing on graphics hardware. ACM Transactions on Graphics, 23, 777–786.CrossRefGoogle Scholar
  52. 52.
    Lastovetsky, A., & Reddy, R. (2004). On performance analysis of heterogeneous parallel algorithms. Parallel Computing, 30, 1195–1216.CrossRefMathSciNetGoogle Scholar
  53. 53.
    Hennessy, J.L., & Patterson, D.A. (2002). Computer architecture: A quantitative approach, 3rd ed. San Mateo: Morgan Kaufmann.zbMATHGoogle Scholar
  54. 54.
    Jyotheswar, J., & Mahapatra, S. (2007). Efficient FPGA implementation of DWT and modified SPIHT for lossless image compression. Journal of Systems Architecture, 53, 369–378.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Department of Technology of Computers and CommunicationsUniversity of ExtremaduraCaceresSpain
  2. 2.ArTeCS Group, Department of Computer ArchitectureComplutense UniversityMadridSpain

Personalised recommendations