Abstract
Forward radiative transfer (RT) models are essential for atmospheric applications such as remote sensing and weather and climate models, where computational efficiency becomes equally as important as accuracy for high-resolution hyperspectral measurements that need rigorous RT simulations for thousands of channels. This study introduces a fast and accurate RT model for the hyperspectral infrared (HIR) sounder based on principal component analysis (PCA) or machine learning (i.e., neural network, NN). The Geosynchronous Interferometric Infrared Sounder (GIIRS), the first HIR sounder onboard the geostationary Fengyun-4 satellites, is considered to be a candidate example for model development and validation. Our method uses either PCA or NN (PCA/NN) twice for the atmospheric transmittance and radiance, respectively, to reduce the number of independent but similar simulations to accelerate RT simulations; thereby, it is referred to as a multi-domain compression model. The first PCA/NN gives monochromatic gas transmittance in both spectral and atmospheric pressure domains for each gas independently. The second PCA/NN is performed in the traditional spectral radiance domain. Meanwhile, a new method is introduced to choose representative variables for the PCA/NN scheme developments. The model is three orders of magnitude faster than the standard line-by-line-based simulations with averaged brightness temperature difference (BTD) less than 0.1 K, and the compressions based on PCA or NN methods result in comparable efficiency and accuracy. Our fast model not only avoids an excessively complicated transmittance scheme by using PCA/NN but is also highly flexible for hyperspectral instruments with similar spectral ranges simply by updating the corresponding spectral response functions.
摘 要
由于吸收系数的剧烈变化,高光谱大气吸收的计算和存储量不容忽视。因此,本文基于多维度的信息冗余,研发了一种快速且准确的基于主成分分析 (Principal Component Analysis,PCA)和机器学习(即神经网络,Neural Network,NN) 方法的晴空高光谱快速辐射传输计算模型。地球同步干涉红外探测仪 (Geosynchronous Interferometric Infrared Sounder,GIIRS)是搭载于风云四号静止卫星的第一个红外高光谱探测仪,被用于验证本文所研发模型的计算精度和效率。本文所研发的模型分别在大气透过率维度和辐射维度应用了PCA和NN算法,进一步压缩信息,减少了单色的精确模拟数量,从而加速模型计算。本模型在GIIRS上的验证结果显示:与精确的逐线模型相比,本模型的平均亮温差小于 0.1 K,且计算效率提升了约三个数量级。同时这两种方法应用在不同大气条件时虽各有优势,但是二者计算精度和效率均较为接近,可以视具体情况选择更为适合的方法。本论文所研发的快速模型还避免了额外的复杂透过率方案的使用,对于具有类似光谱范围的高光谱仪器,仅需改变模型中使用的仪器对应光谱响应函数即可,具有高度的灵活性。除了最终计算得到的仪器通道辐射量和亮温,本模型还可提供中间输出的高光谱大气透过率和辐射量数据集,可直接或作为其他模型子模块使用,适用的科研和业务场景也更为丰富。
Similar content being viewed by others
References
Antonelli, P., and Coauthors, 2004: A principal component noise filter for high spectral resolution infrared measurements. J. Geophys. Res.: Atmos., 109, D23102, https://doi.org/10.1029/2004JD004862.
Bai, W. G., P. Zhang, W. J. Zhang, J. Li, 2016: An efficient method for hyper-spectral infrared atmospheric radiation transfer calculation. J. Infrared. Millim. W., 35(1), 99–108, https://doi.org/10.11972/j.ssn.1001-9014.2016.01.017.
Borbas, E. E., S. W. Seemann, H. L. Huang, J. Li, and W. P. Menze, 2005: Global profile training database for satellite regression retrievals with estimates of skin temperature and emissivity. Proc. 14th Int. ATOVS Study Conf, 763–770.
Burrows, C., T. McNally, and D. Coppens, 2021: Progress in the assimilation of GIIRS data at ECMWF. International TOVS Conference (ITSC-23).
Chen, Y., F. Z. Weng, Y. Han, and Q. H. Liu, 2008: Validation of the Community Radiative Transfer Model by using CloudSat data. J. Geophys. Res., 113(D8), https://doi.org/10.1029/2007JD009561. https://doi.org/10.1029/2007JD009561.
Chevallier, F., S. Di Michele, and A. P. McNally, 2006: Diverse profile datasets from the ECMWF 91-level short-range forecasts. ECMWF, Rep. No. NWPSAF-EC-TR-010, 14 pp.
Chevallier, F., F. Chéruy, N. A. Scott, and A. Chédin, 1998: A neural network approach for a fast and accurate computation of a longwave radiative budget. J. Appl. Meteorol., 37(11), 1385–1397, https://doi.org/10.1175/1520-0450(1998)037<1385:ANNAFA>2.0.CO;2.
Clough, S. A., M. W. Shephard, E. J. Mlawer, J. S. Delamere, M. J. Iacono, K. Cady-Pereira, S. Boukabara, and P. D. Brown, 2005: Atmospheric radiative transfer modeling: A summary of the AER codes. Journal of Quantitative Spectroscopy and Radiative Transfer, 91, 233–244, https://doi.org/10.1016/j.jqsrt.2004.05.058.
Crevoisier, C., S. Heilliette, A. Chédin, S. Serrar, R. Armante, and N. A. Scott, 2004: Midtropospheric CO2 concentration retrieval from AIRS observations in the tropics. Geophys. Res. Lett., 31(17), https://doi.org/10.1029/2004GL020141.
Di, D., J. Li, W. Han, W. G. Bai, C. Q. Wu, and W. P. Menzel, 2018: Enhancing the fast radiative transfer model for FengYun-4 GIIRS by using local training profiles. J. Geophys. Res.: Atmos., 123(22), 12,583–12,596, https://doi.org/10.1029/2018JD029089.
Dorvlo, A. S. S., J. A. Jervase, and A. Al-Lawati, 2002: Solar radiation estimation using artificial neural networks. Applied Energy, 71(4), 307–319, https://doi.org/10.1016/S0306-2619(02)00016-8.
Duan, M., Q. Min, and D. Lu, 2010: A polarized radiative transfer model based on successive order of scattering. Adv. Atmos. Sci., 27, 891–900, https://doi.org/10.1007/s00376-009-9049-8.
Efremenko, D., A. Doicu, D. Loyola, T. Trautmann, 2014: Optical property dimensionality reduction techniques for accelerated radiative transfer performance: application to remote sensing total ozone retrievals. J. Quant. Spectrosc. Radiat. Trans., 133, 128–135, https://doi.org/10.1016/j.jqsrt.2013.07.023.
Glorot, X., A. Bordes, and Y. Bengio, 2011: Deep sparse rectifier neural networks. Proc. 14th Int. Conf. on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 315–323.
Han, Y., P. van Deist, Q. H. Liu, F. Z. Weng, B. H. Yan, R. Treadon, and J. Derber, 2006: JCSDA Community Radiative Transfer Model (CRTM)-Version 1. NOAA Technical Rep. NESDIS 122, 40 pp.
Jordan, M. I., and T. M. Mitchell, 2015: Machine learning: Trends, perspectives, and prospects. Science, 349, 255–260, https://doi.org/10.1126/science.aaa8415.
Kan, W. L., P. M. Dong, Z. Q. Zhang, and S. G. Ding, 2020: Development and application of ARMS fast transmittance model for GIIRS data. Journal of Quantitative Spectroscopy and Radiative Transfer, 251, 107025, https://doi.org/10.1016/j.jqsrt.2020.107025.
Karpowicz, B. M., P. G. Stegmann, B. T. Johnson, H. W. Christophersen, E. J. Hyer, A. Lambert, and E. Simon, 2022: pyCRTM: A python interface for the community radiative transfer model. Journal of Quantitative Spectroscopy and Radiative Transfer, 288, 108263, https://doi.org/10.1016/j.jqsrt.2022.108263.
Le, T. H., C. Liu, B. Yao, V. Natraj, and Y. L. Yung, 2020: Application of machine learning to hyperspectral radiative transfer simulations. Journal of Quantitative Spectroscopy and Radiative Transfer, 246, 106928, https://doi.org/10.1016/j.jqsrt.2020.106928.
Li, J., W. P. Menzel, T. J. Schmit, and J. Schmetz, 2022a: Applications of geostationary hyperspectral infrared sounder observations: Progress, challenges, and future perspectives. Bull. Amer. Meteor. Soc., 103(12), E2733–E2755, https://doi.org/10.1175/BAMS-D-21-0328.1.
Li, J., W. W. Wolf, W. P. Menzel, W. J. Zhang, H.-L. Huang, and T. H. Achtor, 2000: Global soundings of the atmosphere from ATOVS measurements: The algorithm and validation. J. Appl. Meteorol. Climatol., 39(8), 1248–1268, https://doi.org/10.1175/1520-0450(2000)039<1248:GSOTAF>2.0.CO;2.
Li, J., Z. L. Li, P. Wang, T. J. Schmit, W. G. Bai, and R. Atlas, 2017: An efficient radiative transfer model for hyperspectral IR radiance simulation and applications under cloudy-sky conditions. J. Geophys. Res.: Atmos., 122(14), 7600–7613, https://doi.org/10.1002/2016JD026273.
Li, L., Z. Y. Ni, C. L. Qi, L. Yang, and C. P. Han, 2022b: Pre-Launch radiometric calibration of geostationary interferometric infrared sounder on FengYun-4B satellite. Acta Optica Sinica, 42(6), 0630001, https://doi.org/10.3788/AOS202242.0630001. (in Chinese with English abstract)
Li, Z. L., and Coauthors, 2018: Value-added impact of geostationary hyperspectral infrared sounders on local severe storm forecasts—Via a quick regional OSSE. Adv. Atmos. Sci., 35(10), 1217–1230, https://doi.org/10.1007/s00376-018-8036-3.
Liang, X. M., and Q. H. Liu, 2020: Applying deep learning to clear-sky radiance simulation for VIIRS with community radiative transfer model–Part 2: Model architecture and assessment. Remote Sensing, 12(22), 3825, https://doi.org/10.3390/rs12223825.
Liou, K. N., 1992: Radiation and Cloud Processes in the Atmosphere. Theory, Observation, and Modeling. Oxford University Press.
Liou, K. N., 2002: An Introduction to Atmospheric Radiation (Vol. 84). Elsevier.
Liu, C., B. Yao, V. Natraj, F. Z. Weng, T. H. Le, R. L. Shia, and Y. L. Yung, 2020: A spectral data compression (SDCOMP) radiative transfer model for high-spectral-resolution radiation simulations. J. Atmos. Sci., 77(6), 2055–2066, https://doi.org/10.1175/JAS-D-19-0238.1.
Liu, X., W. L. Smith, D. K. Zhou, and A. Larar, 2006: Principal component-based radiative transfer model for hyperspectral sensors: Theoretical concept. Appl. Opt., 45(1), 201–209, https://doi.org/10.1364/AO.45.000201.
Liu, X., Q. G. Yang, H. Li, Z. H. Jin, W. Wu, S. Kizer, D. K. Zhou, and P. Yang, 2016: Development of a fast and accurate PCRTM radiative transfer model in the solar spectral region. Appl. Opt., 55(29), 8236–8247, https://doi.org/10.1364/AO.55.008236.
Matricardi, M., 2008: The generation of RTTOV regression coefficients for IASI and AIRS using a new profile training set and a new line-by-line database. ECMWF Technical Memoranda, Reading, https://doi.org/10.21957/59u3oc9es. https://doi.org/10.21957/59u3oc9es.
Matricardi, M., 2010: A principal component based version of the RTTOV fast radiative transfer model. Quart. J. Roy. Meteor. Soc., 136(652), 1823–1835, https://doi.org/10.1002/qj.680.
McMillin, L. M., L. J. Crone, and T. J. Kleespies, 1995: Atmospheric transmittance of an absorbing gas. 5. Improvements to the OPTRAN approach. Appl. Opt., 34(36), 8396–8399, https://doi.org/10.1364/AO.34.008396.
Menzel, W. P., T. J. Schmit, P. Zhang, and J. Li, 2018: Satellite-based atmospheric infrared sounder development and applications. Bull. Amer. Meteor. Soc., 99(3), 583–603, https://doi.org/10.1175/BAMS-D-16-0293.1.
Moncet, J. L., G. Uymin, A. E. Lipton, and H. E. Snell, 2008: Infrared radiance modeling by optimal spectral sampling. J. Atmos. Sci., 65(12), 3917–3934, https://doi.org/10.1175/2008JAS2711.1.
Natraj, V., X. Jiang, R.-L. Shia, X. L. Huang, J. S. Margolis, and Y. L. Yung, 2005: Application of principal component analysis to high spectral resolution radiative transfer: A case study of the O2A band. Journal of Quantitative Spectroscopy and Radiative Transfer, 95(4), 539–556, https://doi.org/10.1016/j.jqsrt.2004.12.024.
Poostchi, M., K. Silamut, R. J. Maude, S. Jaeger, and G. Thoma, 2018: Image analysis and machine learning for detecting malaria. Translational Research, 194, 36–55, https://doi.org/10.1016/j.trsl.2017.12.004.
Rothman, L. S., and Coauthors, 2013: The HITRAN2012 molecular spectroscopic database. Journal of Quantitative Spectroscopy and Radiative Transfer, 130, 4–50, https://doi.org/10.1016/j.jqsrt.2013.07.002.
Saunders, R., and Coauthors, 2018: An update on the RTTOV fast radiative transfer model (currently at version 12). Geoscientific Model Development, 11(7), 2717–2737, https://doi.org/10.5194/gmd-11-2717-2018.
Schmetz, J., 2021: Good things need time: Progress with the first hyperspectral sounder in geostationary orbit. Geophy. Res. Lett., 48(21), e2021GL096207, https://doi.org/10.1029/2021GL096207.
Shi, G. Y., 1998: On the k-distribution and correlated k-distribution models in the atmospheric radiation calculations. Scientia Atmospherica Sinica, 22(4), 659–676, https://doi.org/10.3878/j.issn.1006-9895.1998.04.25. (in Chinese with English abstract)
Stegmann, P. G., B. Johnson, I. Moradi, B. Karpowicz, and W. McCarty, 2022: A deep learning approach to fast radiative transfer. Journal of Quantitative Spectroscopy and Radiative Transfer, 280, 108088, https://doi.org/10.1016/j.jqsrt.2022.108088.
Takenaka, H., T. Y. Nakajima, A. Higurashi, A. Higuchi, T. Takamura, R. T. Pinker, and T. Nakajima, 2011: Estimation of solar radiation using a neural network based on radiative transfer. J. Geophys. Res.: Atmos., 116(D8), D08215, https://doi.org/10.1029/2009JD013337.
Taylor, M., P. G. Kosmopoulos, S. Kazadzis, I. Keramitsoglou, and C. T. Kiranoudis, 2016: Neural network radiative transfer solvers for the generation of high resolution solar irradiance spectra parameterized by cloud and aerosol parameters. Journal of Quantitative Spectroscopy and Radiative Transfer, 168, 176–192, https://doi.org/10.1016/j.jqsrt.2015.08.018.
Ukkonen, P., 2022: Exploring pathways to more accurate machine learning emulation of atmospheric radiative transfer. Journal of Advances in Modeling Earth Systems, 14(4), e2021MS002875, https://doi.org/10.1029/2021MS002875.
Wang, C. J., B. S. He, and M. F. Modest, 2019: Full-spectrum correlated-k-distribution look-up table for radiative transfer in nonhomogeneous participating media with gas-particle mixtures. International Journal of Heat and Mass Transfer, 137, 1053–1063, https://doi.org/10.1016/j.ijheatmasstransfer.2019.03.149.
Wang, C. X., P. Yang, and X. Liu, 2015: A high-spectral-resolution radiative transfer model for simulating multilayered clouds and aerosols in the infrared spectral region. J. Atmos. Sci., 72(2), 926–942, https://doi.org/10.1175/JAS-D-14-0046.1.
Wang, C. X., P. Yang, S. Platnick, A. K. Heidinger, B. A. Baum, T. Greenwald, Z. B. Zhang, and R. E. Holz, 2013: Retrieval of ice cloud properties from AIRS and MODIS observations based on a fast high-spectral-resolution radiative transfer model. J. Appl. Meteorol. Climatol., 52(3), 710–726, https://doi.org/10.1175/JAMC-D-12-020.1.
Yang, J., Z. Q. Zhang, C. Y. Wei, F. Lu, and Q. Guo, 2017: Introducing the new generation of Chinese geostationary weather satellites, Fengyun-4. Bull. Amer. Meteor. Soc., 98(8), 1637–1658, https://doi.org/10.1175/BAMS-D-16-0065.1.
Zeng, Z. C., L. Lee, and C. L. Qi, 2022: Diurnal carbon monoxide observed from a geostationary infrared hyperspectral sounder: First result from GIIRS onboard FY-4B. Atmospheric Measurement Techniques Discussions, https://doi.org/10.5194/amt-2022-305.
Zeng, Z. C., L. Lee, and C. L. Qi, 2023: Optimal estimation retrieval of tropospheric ammonia from the Geostationary Interferometric Infrared Sounder onboard Feng Yun-4B. Atmospheric Measurement Techniques Discussions, https://doi.org/10.5194/amt-2023-12.
Zhang, H., and G. Y. Shi, 2000: A fast and efficient Line-By-Line calculation method for atmospheric absorption. Chinese Journal of Atmospheric Sciences, 24(1), 111–121, https://doi.org/10.3878/j.issn.1006-9895.2000.01.12. (in Chinese with English abstract)
Zhang, H., G. Y. Shi, and Y. Liu, 2005: A comparison between the two line-by-line integration algorithms. Chinese Journal of Atmospheric Sciences, 29(4), 581–593, https://doi.org/10.3878/j.issn.1006-9895.2005.04.09. (in Chinese with English abstract)
Zhang, J., Z. L. Li, J. Li, and J. L. Li, 2014: Ensemble retrieval of atmospheric temperature profiles from AIRS. Adv. Atmos. Sci., 31, 559–569, https://doi.org/10.1007/s00376-013-3094-z.
Zhang, K., C. Q. Wu, and J. Li, 2016: Retrieval of atmospheric temperature and moisture vertical profiles from satellite advanced infrared sounder radiances with a new regularization parameter selecting method. J. Meteor. Res., 30(3), 356–370, https://doi.org/10.1007/s13351-016-6025-y.
Zhou, Y., C. J. Wang, and T. Ren, 2020: A machine learning based efficient and compact full-spectrum correlated k-distribution model. Journal of Quantitative Spectroscopy and Radiative Transfer, 254, 107199, https://doi.org/10.1016/j.jqsrt.2020.107199.
Acknowledgements
We thank the Atomic and Molecular Physics Division, the Harvard-Smithsonian Center for the HITRAN dataset, and the Atmospheric and Environmental Research (AER) Inc for the LBLRTM model. This research is supported by the National Natural Science Foundation of China (Grant No. 42122038). The model simulations are conducted in the High Performance Computing Center of Nanjing University of Information Science & Technology.
Author information
Authors and Affiliations
Corresponding author
Additional information
Article Highlights
• A fast high-spectral radiative transfer model based on PCA or NN techniques for multi-domain compressions is developed.
• The fast model gives GIIRS channel brightness temperature differences of less than 0.5 K compared with LBL results.
• The fast model is approximately three orders of magnitude faster than the LBL model
• The PCA- and NN-based RT models achieve similar accuracy and efficiency.
Rights and permissions
About this article
Cite this article
Su, M., Liu, C., Di, D. et al. A Multi-Domain Compression Radiative Transfer Model for the Fengyun-4 Geosynchronous Interferometric Infrared Sounder (GIIRS). Adv. Atmos. Sci. 40, 1844–1858 (2023). https://doi.org/10.1007/s00376-023-2293-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00376-023-2293-5