Skip to main content

A two-stage method for spectral–spatial classification of hyperspectral images

Journal of Mathematical Imaging and Vision volume 62, pages 790–807 (2020)Cite this article

Abstract

We propose a novel two-stage method for the classification of hyperspectral images. Pixel-wise classifiers, such as the classical support vector machine (SVM), consider spectral information only. As spatial information is not utilized, the classification results are not optimal and the classified image may appear noisy. Many existing methods, such as morphological profiles, superpixel segmentation, and composite kernels, exploit the spatial information. In this paper, we propose a two-stage approach inspired by image denoising and segmentation to incorporate the spatial information. In the first stage, SVMs are used to estimate the class probability for each pixel. In the second stage, a convex variant of the Mumford–Shah model is applied to each probability map to denoise and segment the image into different classes. Our proposed method effectively utilizes both spectral and spatial information of the data sets and is fast as only convex minimization is needed in addition to the SVMs. Experimental results on three widely utilized real hyperspectral data sets indicate that our method is very competitive in accuracy, timing, and the number of parameters when compared with current state-of-the-art methods, especially when the inter-class spectra are similar or the percentage of training pixels is reasonably high.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Patel, N., Patnaik, C., Dutta, S., Shekh, A., Dave, A.: Study of crop growth parameters using airborne imaging spectrometer data. Int. J. Remote Sens. 22(12), 2401–2411 (2001)

    Google Scholar 

  2. Datt, B., McVicar, T.R., Van Niel, T.G., Jupp, D.L., Pearlman, J.S.: Preprocessing EO-1 hyperion hyperspectral data to support the application of agricultural indexes. IEEE Trans. Geosci. Remote Sens. 41(6), 1246–1259 (2003)

    Google Scholar 

  3. Trierscheid, M., Pellenz, J., Paulus, D., Balthasar, D.: Hyperspectral imaging or victim detection with rescue robots. In: IEEE International Workshop on Safety, Security and Rescue Robotics, 2008. SSRR 2008, pp. 7–12. IEEE (2008)

  4. Eismann, M.T., Stocker, A.D., Nasrabadi, N.M.: Automated hyperspectral cueing for civilian search and rescue. Proc. IEEE 97(6), 1031–1055 (2009)

    Google Scholar 

  5. Lu, R., Chen, Y.-R.: Hyperspectral imaging for safety inspection of food and agricultural products. In: Pathogen Detection and Remediation for Safe Eating, vol. 3544, pp. 121–134. International Society for Optics and Photonics (1999)

  6. Gowen, A., O’Donnell, C., Cullen, P., Downey, G., Frias, J.: Hyperspectral imaging-an emerging process analytical tool for food quality and safety control. Trends Food Sci. Technol. 18(12), 590–598 (2007)

    Google Scholar 

  7. Manolakis, D., Shaw, G.: Detection algorithms for hyperspectral imaging applications. IEEE Signal Process. Mag. 19(1), 29–43 (2002)

    Google Scholar 

  8. Stein, D.W., Beaven, S.G., Hoff, L.E., Winter, E.M., Schaum, A.P., Stocker, A.D.: Anomaly detection from hyperspectral imagery. IEEE Signal Process. Mag. 19(1), 58–69 (2002)

    Google Scholar 

  9. Hörig, B., Kühn, F., Oschütz, F., Lehmann, F.: Hymap hyperspectral remote sensing to detect hydrocarbons. Int. J. Remote Sens. 22(8), 1413–1422 (2001)

    Google Scholar 

  10. Mountrakis, G., Im, J., Ogole, C.: Support vector machines in remote sensing: a review. ISPRS J. Photogramm. Remote Sens. 66(3), 247–259 (2011)

    Google Scholar 

  11. Fauvel, M., Tarabalka, Y., Benediktsson, J.A., Chanussot, J., Tilton, J.C.: Advances in spectral-spatial classification of hyperspectral images. Proc. IEEE 101(3), 652–675 (2013)

    Google Scholar 

  12. Li, J., Bioucas-Dias, J.M., Plaza, A.: Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Trans. Geosci. Remote Sens. 48(11), 4085–4098 (2010)

    Google Scholar 

  13. Li, J., Bioucas-Dias, J.M., Plaza, A.: Spectral-spatial hyperspectral image segmentation using subspace multinomial logistic regression and markov random fields. IEEE Trans. Geosci. Remote Sens. 50(3), 809–823 (2012)

    Google Scholar 

  14. Li, J., Bioucas-Dias, J.M., Plaza, A.: Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression. IEEE Trans. Geosci. Remote Sens. 10(2), 318–322 (2013)

    Google Scholar 

  15. Benediktsson, J.A., Palmason, J.A., Sveinsson, J.R.: Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans. Geosci. Remote Sens. 43(3), 480–491 (2005)

    Google Scholar 

  16. Yue, J., Zhao, W., Mao, S., Liu, H.: Spectral-spatial classification of hyperspectral images using deep convolutional neural networks. Remote Sens. Lett. 6(6), 468–477 (2015)

    Google Scholar 

  17. Makantasis, K., Karantzalos, K., Doulamis, A., Doulamis, N.: Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: 2015 IEEE International on Geoscience and Remote Sensing Symposium (IGARSS), pp. 4959–4962. IEEE (2015)

  18. Morchhale, S., Pauca, V.P., Plemmons, R.J., Torgersen, T.C.: Classification of pixel-level fused hyperspectral and LiDAR data using deep convolutional neural networks. In: 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), pp. 1–5 (2016)

  19. Pan, B., Shi, Z., Xu, X.: R-vcanet: a new deep-learning-based hyperspectral image classification method. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(5), 1975–1986 (2017)

    Google Scholar 

  20. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM (1992)

  21. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  22. Schölkopf, B., Smola, A.J., Williamson, R.C., Bartlett, P.L.: New support vector algorithms. Neural Comput. 12(5), 1207–1245 (2000)

    Google Scholar 

  23. Melgani, F., Bruzzone, L.: Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans. Geosci. Remote Sens. 42(8), 1778–1790 (2004)

    Google Scholar 

  24. Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 43(6), 1351–1362 (2005)

    Google Scholar 

  25. Camps-Valls, G., Gomez-Chova, L., Muñoz-Marí, J., Vila-Francés, J., Calpe-Maravilla, J.: Composite kernels for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 3(1), 93–97 (2006)

    Google Scholar 

  26. Fang, L., Li, S., Duan, W., Ren, J., Benediktsson, J.A.: Classification of hyperspectral images by exploiting spectral-spatial information of superpixel via multiple kernels. IEEE Trans. Geosci. Remote Sens. 53(12), 6663–6674 (2015)

    Google Scholar 

  27. Tarabalka, Y., Benediktsson, J.A., Chanussot, J.: Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques. IEEE Trans. Geosci. Remote Sens. 47(8), 2973–2987 (2009)

    Google Scholar 

  28. Kang, X., Li, S., Benediktsson, J.A.: Spectral-spatial hyperspectral image classification with edge-preserving filtering. IEEE Trans. Geosci. Remote Sens. 52(5), 2666–2677 (2014)

    Google Scholar 

  29. Tarabalka, Y., Fauvel, M., Chanussot, J., Benediktsson, J.A.: Svm-and mrf-based method for accurate classification of hyperspectral images. IEEE Geosci. Remote Sens. Lett. 7(4), 736–740 (2010)

    Google Scholar 

  30. Ghamisi, P., Benediktsson, J.A., Ulfarsson, M.O.: Spectral-spatial classification of hyperspectral images based on hidden markov random fields. IEEE Trans. Geosci. Remote Sens. 52(5), 2565–2574 (2014)

    Google Scholar 

  31. Liu, T., Gu, Y., Chanussot, J., Mura, M Dalla: Multimorphological superpixel model for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 55(12), 6950–6963 (2017)

    Google Scholar 

  32. Fauvel, M., Benediktsson, J.A., Chanussot, J., Sveinsson, J.R.: Spectral and spatial classification of hyperspectral data using svms and morphological profiles. IEEE Trans. Geosci. Remote Sens. 46(11), 3804–3814 (2008)

    Google Scholar 

  33. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)

    MathSciNet  MATH  Google Scholar 

  34. Chen, Y., Nasrabadi, N.M., Tran, T.D.: Hyperspectral image classification using dictionary-based sparse representation. IEEE Trans. Geosci. Remote Sens. 49(10), 3973–3985 (2011)

    Google Scholar 

  35. Chen, Y., Nasrabadi, N.M., Tran, T.D.: Hyperspectral image classification via kernel sparse representation. IEEE Trans. Geosci. Remote Sens. 51(1), 217–231 (2013)

    Google Scholar 

  36. Fang, L., Li, S., Kang, X., Benediktsson, J.A.: Spectral-spatial hyperspectral image classification via multiscale adaptive sparse representation. IEEE Trans. Geosci. Remote Sens. 52(12), 7738–7749 (2014)

    Google Scholar 

  37. Fang, L., Li, S., Kang, X., Benediktsson, J.A.: Spectral-spatial classification of hyperspectral images with a superpixel-based discriminative sparse model. IEEE Trans. Geosci. Remote Sens. 53(8), 4186–4201 (2015)

    Google Scholar 

  38. Fang, L., Wang, C., Li, S., Benediktsson, J.A.: Hyperspectral image classification via multiple-feature-based adaptive sparse representation. IEEE Trans. Instrum. Meas. 66(7), 1646–1657 (2017)

    Google Scholar 

  39. Li, S., Lu, T., Fang, L., Jia, X., Benediktsson, J.A.: Probabilistic fusion of pixel-level and superpixel-level hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 54(12), 7416–7430 (2016)

    Google Scholar 

  40. Mumford, D., Shah, J.: Boundary detection by minimizing functionals. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 17, pp. 137–154, San Francisco (1985)

  41. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)

    MathSciNet  MATH  Google Scholar 

  42. Morar, A., Moldoveanu, F., Gröller, E.: Image segmentation based on active contours without edges. In: 2012 IEEE 8th International Conference on Intelligent Computer Communication and Processing, pp. 213–220. IEEE (2012)

  43. Cai, X., Chan, R., Zeng, T.: A two-stage image segmentation method using a convex variant of the mumford-shah model and thresholding. SIAM J. Imaging Sci. 6(1), 368–390 (2013)

    MathSciNet  MATH  Google Scholar 

  44. Chan, R., Yang, H., Zeng, T.: A two-stage image segmentation method for blurry images with poisson or multiplicative gamma noise. SIAM J. Imaging Sci. 7(1), 98–127 (2014)

    MathSciNet  MATH  Google Scholar 

  45. Cai, X., Chan, R., Nikolova, M., Zeng, T.: A three-stage approach for segmenting degraded color images: smoothing, lifting and thresholding (SLaT). J. Sci. Comput. 72(3), 1313–1332 (2017)

    MathSciNet  MATH  Google Scholar 

  46. Pontil, M., Verri, A.: Support vector machines for 3d object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 20(6), 637–646 (1998)

    Google Scholar 

  47. El-Naqa, I., Yang, Y., Wernick, M.N., Galatsanos, N.P., Nishikawa, R.M.: A support vector machine approach for detection of microcalcifications. IEEE Trans. Med. Imaging 21(12), 1552–1563 (2002)

    Google Scholar 

  48. Osuna, E., Freund, R., Girosit, F.: Training support vector machines: an application to face detection. In: 1997 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130–136. IEEE (1997)

  49. Tay, F.E., Cao, L.: Application of support vector machines in financial time series forecasting. Omega 29(4), 309–317 (2001)

    Google Scholar 

  50. Kim, K.-J.: Financial time series forecasting using support vector machines. Neurocomputing 55(1), 307–319 (2003)

    Google Scholar 

  51. Vapnik, V.N., Vapnik, V.: Statistical Learning Theory, vol. 1. Wiley, New York (1998)

    MATH  Google Scholar 

  52. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenomena 60(1–4), 259–268 (1992)

    MathSciNet  MATH  Google Scholar 

  53. Mumford, D.: Elastica and computer vision. In: Algebraic Geometry and Its Applications, pp. 491–506. Springer (1994)

  54. Chan, T., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput. 22(2), 503–516 (2000)

    MathSciNet  MATH  Google Scholar 

  55. Shen, J., Kang, S.H., Chan, T.F.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2003)

    MathSciNet  MATH  Google Scholar 

  56. Hintermüller, M., Stadler, G.: An infeasible primal-dual algorithm for total bounded variation-based inf-convolution-type image restoration. SIAM J. Sci. Comput. 28(1), 1–23 (2006)

    MathSciNet  MATH  Google Scholar 

  57. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)

    MathSciNet  MATH  Google Scholar 

  58. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    MATH  Google Scholar 

  59. Cai, X., Steidl, G.: Multiclass segmentation by iterated rof thresholding. In: International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 237–250. Springer (2013)

  60. Chan, R.H., Ho, C.-W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)

    Google Scholar 

  61. Hwang, H., Haddad, R.A.: Adaptive median filters: new algorithms and results. IEEE Trans. Image Process. 4(4), 499–502 (1995)

    Google Scholar 

  62. Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004)

    MathSciNet  MATH  Google Scholar 

  63. Hsu, C.-W., Lin, C.-J.: A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Netw. 13(2), 415–425 (2002)

    Google Scholar 

  64. Lin, H.-T., Lin, C.-J., Weng, R.C.: A note on platt’s probabilistic outputs for support vector machines. Mach. Learn. 68(3), 267–276 (2007)

    Google Scholar 

  65. Wu, T.-F., Lin, C.-J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5(Aug), 975–1005 (2004)

    MathSciNet  MATH  Google Scholar 

  66. Chang, C.-C., Lin, C.-J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  67. Zhao, X.-L., Wang, F., Huang, T.-Z., Ng, M.K., Plemmons, R.J.: Deblurring and sparse unmixing for hyperspectral images. IEEE Trans. Geosci. Remote Sens. 51(7), 4045–4058 (2013)

    Google Scholar 

  68. Gonzales, R.C., Woods, R.E.: Digital Image Processing. Addison-Wesley, Reading, MA (1992)

    Google Scholar 

  69. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)

    MATH  Google Scholar 

  70. Chan, R.H.-F., Jin, X.-Q.: An Introduction to Iterative Toeplitz Solvers, vol. 5. SIAM, Philadelphia (2007)

    MATH  Google Scholar 

  71. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward–backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    MathSciNet  MATH  Google Scholar 

  72. Kohavi, R., et al.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Ijcai, Montreal, vol. 14, pp. 1137–1145 (1995)

  73. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)

    Google Scholar 

  74. Liao, H., Li, F., Ng, M.K.: Selection of regularization parameter in total variation image restoration. JOSA A 26(11), 2311–2320 (2009)

    MathSciNet  Google Scholar 

  75. Dong, Y., Hintermüller, M., Rincon-Camacho, M.M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011)

    MathSciNet  MATH  Google Scholar 

  76. Wen, Y.-W., Chan, R.H.: Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Trans. Image Process. 21(4), 1770–1781 (2012)

    MathSciNet  MATH  Google Scholar 

  77. Bredies, K., Dong, Y., Hintermüller, M.: Spatially dependent regularization parameter selection in total generalized variation models for image restoration. Int. J. Comput. Math. 90(1), 109–123 (2013)

    MathSciNet  MATH  Google Scholar 

  78. Gader, P., Zare, A., Close, R., Aitken, J., Tuell, G.: Muufl gulfport hyperspectral and LiDAR airborne data set. Univ. Florida, Gainesville, FL, Tech. Rep. REP-2013-570 (2013)

  79. Debes, C., Merentitis, A., Heremans, R., Hahn, J., Frangiadakis, N., van Kasteren, T., Liao, W., Bellens, R., Pižurica, A., Gautama, S., et al.: Hyperspectral and LiDAR data fusion: outcome of the 2013 grss data fusion contest. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 7(6), 2405–2418 (2014)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Computational Intelligence Group from the Basque University for sharing the hyperspectral data sets in their website (http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes), Prof. Leyuan Fang from College of Electrical and Information Engineering at Hunan University for providing the programs of the SC-MK and MFASR methods in his homepage (http://www.escience.cn/people/LeyuanFang) and Prof. Xudong Kang from College of Electrical and Information Engineering at Hunan University for providing the program of the EPF method in his homepage (http://xudongkang.weebly.com/). Raymond H. Chan’s research is supported by HKRGC Grants No. CUHK14306316, CityU Grant: 9380101, CRF Grant C1007-15G, AoE/M-05/12. Kelvin K. Kan’s research is supported by US Air Force Office of Scientific Research under grant FA9550-15-1-0286. Mila Nikolova’s research is supported by the French Research Agency (ANR) under grant No ANR-14-CE27-001 (MIRIAM) and by the Isaac Newton Institute for Mathematical Sciences for support and hospitality during the programme Variational Methods and Effective Algorithms for Imaging and Vision, EPSRC grant no EP/K032208/1. Robert J. Plemmons’ research is supported by HKRGC Grant No. CUHK14306316 and US Air Force Office of Scientific Research under grant FA9550-15-1-0286.

Author information

Authors and Affiliations

  1. Department of Mathematics, City University of Hong Kong, Tat Chee Avenue, Kowloon Tong, Hong Kong

    Raymond H. Chan

  2. Department of Mathematics, Emory University, Atlanta, GA, 30322, USA

    Kelvin K. Kan

  3. CMLA, ENS Cachan, CNRS, Université Paris-Saclay, 94235, Cachan, France

    Mila Nikolova

  4. Department of Computer Science and Department of Mathematics, Wake Forest University, Winston-Salem, NC, 27106, USA

    Robert J. Plemmons

Authors
  1. Raymond H. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelvin K. Kan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mila Nikolova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert J. Plemmons
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raymond H. Chan.

Additional information

In memory of Mila Nikolova.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chan, R.H., Kan, K.K., Nikolova, M. et al. A two-stage method for spectral–spatial classification of hyperspectral images. J Math Imaging Vis 62, 790–807 (2020). https://doi.org/10.1007/s10851-019-00925-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-019-00925-9

Keywords

  • Hyperspectral image classification
  • Image segmentation
  • Image denoising
  • Mumford–Shah model
  • Support vector machine
  • Alternating direction method of multipliers