Radiative transfer simulations and remote sensing studies fundamentally require accurate and efficient computation of the optical properties of non-spherical particles. This paper proposes a deep learning (DL) scheme in conjunction with an optical property database to achieve this goal. Deep neural network (DNN) architectures were obtained from a dataset of the optical properties of super-spheroids with extensive shape parameters, size parameters, and refractive indices. The dataset was computed through the invariant imbedding T-matrix method. Four separate DNN architectures were created to compute the extinction efficiency factor, single-scattering albedo, asymmetry factor, and phase matrix. The criterion for designing these neural networks was the achievement of the highest prediction accuracy with minimal DNN parameters. The numerical results demonstrate that the determination coefficients are greater than 0.999 between the prediction values from the neural networks and the truth values from the database, which indicates that the DNN can reproduce the optical properties in the dataset with high accuracy. In addition, the DNN model can robustly predict the optical properties of particles with high accuracy for shape parameters or refractive indices that are unavailable in the database. Importantly, the ratio of the database size (∼127 GB) to that of the DNN parameters (∼20 MB) is approximately 6810, implying that the DNN model can be treated as a highly compressed database that can be used as an alternative to the original database for real-time computing of the optical properties of non-spherical particles in radiative transfer and atmospheric models.
辐射传输模拟和遥感反演需要准确和快速地计算非球形粒子的光学特性. 传统上一般采用查找表方法来解决电磁散射计算效率低的问题. 但随着粒子参数增加, 查找表数据体量变大, 不便于模式使用. 本文提出了一种深度学习方法用于存储和计算非球形粒子光学特性. 我们将基于不变嵌入T-矩阵方法计算的超椭球粒子光学特性作为训练数据库, 选取长宽比、 圆滑度、 粒径和复折射指数作为训练参数, 设计了四种最优的神经网络架构分别计算或预测消光效率因子、 单次散射消光比、 不对称因子以及相矩阵元素. 结果表明: 神经网络预测值与数据库真值之间的决定系数大于0.999, 可以准确再现数据库的光学特性信息. 另外, 神经网络模型还能够可靠预测出未知参数(尺寸和折射指数)的粒子光学特性值. 通过将大型数据库近乎无损压缩为四个神经网络后, 可将小巧的网络模型代替原始查找表接入辐射传输算法中, 从而实现非球形粒子光学特性的高效计算.
Abadi, M., and Coauthors, 2016: TensorFlow: A system for large-scale machine learning. Proc. 12th USENIX Symp. on Operating Systems Design and Implementation, Savannah, USENIX, 265–283.
Barr, A. H., 1981: Superquadrics and angle-preserving transformations. IEEE Computer Graphics and Applications, 1, 11–23, https://doi.org/10.1109/MCG.1981.1673799.
Bengio, Y., 2009: Learning deep architectures for AI. Foundations and Trends® in Machine Learning, 2, 1–127, https://doi.org/10.1561/2200000006.
Bi, L., and P. Yang, 2014: Accurate simulation of the optical properties of atmospheric ice crystals with the invariant imbedding T-matrix method. Journal of Quantitative Spectroscopy and Radiative Transfer, 138, 17–35, https://doi.org/10.1016/j.jqsrt.2014.01.013.
Bi, L., and P. Yang, 2017: Improved ice particle optical property simulations in the ultraviolet to far-infrared regime. Journal of Quantitative Spectroscopy and Radiative Transfer, 189, 228–237, https://doi.org/10.1016/j.jqsrt.2016.12.007.
Bi, L., P. Yang, G. W. Kattawar, and R. Kahn, 2009: Single-scattering properties of triaxial ellipsoidal particles for a size parameter range from the Rayleigh to geometric-optics regimes. Appl. Opt, 48, 114–126, https://doi.org/10.1364/AO.48.000114.
Bi, L., P. Yang, G. W. Kattawar, and M. I. Mishchenko, 2013a: A numerical combination of extended boundary condition method and invariant imbedding method applied to light scattering by large spheroids and cylinders. Journal of Quantitative Spectroscopy and Radiative Transfer, 123, 17–22, https://doi.org/10.1016/j.jqsrt.2012.11.033.
Bi, L., P. Yang, G. W. Kattawar, and M. I. Mishchenko, 2013b: Efficient implementation of the invariant imbedding T-matrix method and the separation of variables method applied to large non-spherical inhomogeneous particles. Journal of Quantitative Spectroscopy and Radiative Transfer, 116, 169–183, https://doi.org/10.1016/j.jqsrt.2012.11.014.
Bi, L., W. S. Lin, Z. Wang, X. Y. Tang, X. Y. Zhang, and B. Q. Yi, 2018: Optical modeling of sea salt aerosols: The effects of nonsphericity and inhomogeneity. J. Geophys. Res. Atmos., 123, 543–558, https://doi.org/10.1002/2017JD027869.
Bohren, C. F., and D. R. Huffman, 1983: Absorption and Scattering of Light by Small Particles. Wiley, 530 pp.
Charlson, R. J., S. E. Schwartz, J. M. Hales, R. D. Cess, J. A. CoakleyJr., J. E. Hansen, and D. J. Hofmann, 1992: Climate forcing by anthropogenic aerosols. Science, 255, 423–430, https://doi.org/10.1126/science.255.5043.423.
Chen, D. H., and Coauthors, 2008: New generation of multi-scale NWP system (GRAPES): General scientific design. Chinese Science Bulletin, 53, 3433–3445, https://doi.org/10.1007/s11434-008-0494-z.
Chen, Y. S., H. L. Jiang, C. Y. Li, X. P. Jia, and P. Ghamisi, 2016: Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens., 54, 6232–6251, https://doi.org/10.1109/TGRS.2016.2584107.
Di Noia, A., and O. P. Hasekamp, 2018: Neural networks and support vector machines and their application to aerosol and cloud remote sensing: A review. Springer Series in Light Scattering: Volume 1: Multiple Light Scattering, Radiative Transfer and Remote Sensing, A. Kokhanovsky, Ed., Springer, 279–329, https://doi.org/10.1007/978-3-319-70796-9_4.
Draine, B. T., and P. J. Flatau, 1994: Discrete-dipole approximation for scattering calculations. Journal of the Optical Society of America A, 11, 1491–1499, https://doi.org/10.1364/JOSAA.11.001491.
Dubovik, O., B. Holben, T. F. Eck, A. Smirnov, Y. J. Kaufman, M. D. King, D. Tanré, and I. Slutsker, 2002: Variability of absorption and optical properties of key aerosol types observed in worldwide locations. J. Atmos. Sci., 59, 590–608, https://doi.org/10.1175/1520-0469(2002)059<0590:VOAAOP>2.0.CO;2.
Dubovik, O., and Coauthors, 2006: Application of spheroid models to account for aerosol particle nonsphericity in remote sensing of desert dust. J. Geophys. Res. Atmos., 111, D11208, https://doi.org/10.1029/2005JD006619.
Dubovik, O., T. Lapyonok, Y. J. Kaufman, M. Chin, P. Ginoux, R. A. Kahn, and A. Sinyuk, 2008: Retrieving global aerosol sources from satellites using inverse modeling. Atmospheric Chemistry and Physics, 8, 209–250, https://doi.org/10.5194/acp-8-209-2008.
Dubovik, O., and Coauthors, 2019: Polarimetric remote sensing of atmospheric aerosols: Instruments, methodologies, results, and perspectives. Journal of Quantitative Spectroscopy and Radiative Transfer, 224, 474–511, https://doi.org/10.1016/j.jqsrt.2018.11.024.
Glorot, X., and Y. Bengio, 2010: Understanding the difficulty of training deep feedforward neural networks. Proc. 13th Int. Conf. on Artificial Intelligence and Statistics (AISTATS) 2010, Sardinia, JMLR, 249–256.
Glorot, X., A. Bordes, and Y. Bengio, 2011: Deep sparse rectifier neural networks. Proc. Fourteenth Int. Conf. on Artificial Intelligence and Statistics, Fort Lauderdale, PMLR, 315–323.
Gong, S. L., and X. Y. Zhang, 2008: CUACE/Dust-an integrated system of observation and modeling systems for operational dust forecasting in Asia. Atmospheric Chemistry and Physics, 8, 2333–2340, https://doi.org/10.5194/acp-8-2333-2008.
Groth, S. P., A. J. Baran, T. Betcke, S. Havemann, and W. Śmigaj, 2015: The boundary element method for light scattering by ice crystals and its implementation in BEM++. Journal of Quantitative Spectroscopy and Radiative Transfer, 167, 40–52, https://doi.org/10.1016/j.jqsrt.2015.08.001.
Ham, Y. G., J. H. Kim, and J. J. Luo, 2019: Deep learning for multi-year ENSO forecasts. Nature, 573, 568–572, https://doi.org/10.1038/s41586-019-1559-7.
Heintzenberg, J., and Coauthors, 1997: Measurements and modelling of aerosol single-scattering albedo: Progress, problems and prospects. Contrib. Atmos. Phys., 70, 249–263.
Hinton, G., and Coauthors, 2012: Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29, 82–97, https://doi.org/10.1109/MSP.2012.2205597.
Hinton, G. E., and R. R. Salakhutdinov, 2006: Reducing the dimensionality of data with neural networks. Science, 313, 504–507, https://doi.org/10.1126/science.1127647.
IPCC, 2013: Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate. Cambridge University Press, 1535 pp.
Johnson, B. R., 1988: Invariant imbedding T matrix approach to electromagnetic scattering. Appl. Opt., 27, 4861–4873, https://doi.org/10.1364/AO.27.004861.
Kahnert, F. M., J. J. Stamnes, and K. Stamnes, 2002: Using simple particle shapes to model the Stokes scattering matrix of ensembles of wavelength-sized particles with complex shapes: Possibilities and limitations. Journal of Quantitative Spectroscopy and Radiative Transfer, 74, 167–182, https://doi.org/10.1016/S0022-4073(01)00194-7.
Kahnert, M., A. Kylling, 2004: Radiance and flux simulations for mineral dust aerosols: Assessing the error due to using spherical or spheroidal model particles. J. Geophys. Res. Atmos., 109, D09203, https://doi.org/10.1029/2003JD004318.
Kahnert, M., T. Nousiainen, and P. Räisänen, 2007: Mie simulations as an error source in mineral aerosol radiative forcing calculations. Quart. J. Roy. Meteor. Soc., 133, 299–307, https://doi.org/10.1002/qj.40.
Kahnert, M., T. Nousiainen, and H. Lindqvist, 2014: Review: Model particles in atmospheric optics. Journal of Quantitative Spectroscopy and Radiative Transfer, 146, 41–58, https://doi.org/10.1016/j.jqsrt.2014.02.014.
King, M. D., Y. J. Kaufman, D. Tanré, and T. Nakajima, 1999: Remote sensing of tropospheric aerosols from space: Past, present, and future. Bull. Amer. Meteor Soc., 80, 2229–2260, https://doi.org/10.1175/1520-0477(1999)080<2229:RSOTAF>2.0.CO;2.
Kingma, D. P., and L. J. Ba, 2015: Adam: A method for stochastic optimization. Proc. 3rd International Conf. on Learning Representations, San Diego, ICLR.
Kok, J. F., and Coauthors, 2017: Integrative analysis of desert dust size and abundance suggests less dust climate cooling. Nature Geoscience, 10, 274–278, https://doi.org/10.1038/ngeo2912.
Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2012: ImageNet classification with deep convolutional neural networks. Proc. 25th Int. Conf. on Neural Information Processing Systems, Lake Tahoe, ACM, 1097–1105, https://doi.org/10.5555/2999134.2999257.
LeCun, Y., Y. Bengio, and G. Hinton, 2015: Deep learning. Nature, 521, 436–444, https://doi.org/10.1038/nature14539.
Li, Z., F. Niu, J. W. Fan, Y. J. Liu, D. Rosenfeld, and Y. N. Ding, 2011: Long-term impacts of aerosols on the vertical development of clouds and precipitation. Nature Geoscience, 888–894, https://doi.org/10.1038/ngeo1313.
Lin, W. S., L. Bi, and O. Dubovik, 2018: Assessing superspheroids in modeling the scattering matrices of dust aerosols. J. Geophys. Res. Atmos., 123, 13 917–13 943, https://doi.org/10.1029/2018JD029464.
Liu, C., R. Lee Panetta, and P. Yang, 2012: Application of the pseudo-spectral time domain method to compute particle single-scattering properties for size parameters up to 200. Journal of Quantitative Spectroscopy and Radiative Transfer, 113, 1728–1740, https://doi.org/10.1016/j.jqsrt.2012.04.021.
Liu, Q. H., 1997: The PSTD algorithm: A time-domain method requiring only two cells per wavelength. Microwave and Optical Technology Letters, 15, 158–165, https://doi.org/10.1002/(SICI)1098-2760(19970620)15:3<158::AID-MOP11>3.0.CO;2-3.
Mishchenko, M. I., and M. A. Yurkin, 2017: On the concept of random orientation in far-field electromagnetic scattering by non-spherical particles. Opt. Lett., 2, 494–497, https://doi.org/10.1364/OL.42.000494.
Mishchenko, M. I., A. A. Lacis, B. E. Carlson, and L. D. Travis, 1995: Nonsphericity of dust-like tropospheric aerosols: Implications for aerosol remote sensing and climate modeling. Geophys. Res. Lett., 22, 1077–1080, https://doi.org/10.1029/95GL00798.
Mishchenko, M. I., and Coauthors, 2003: Aerosol retrievals from AVHRR radiances: Effects of particle nonsphericity and absorption and an updated long-term global climatology of aerosol properties. Journal of Quantitative Spectroscopy and Radiative Transfer, 79-80, 953–972, https://doi.org/10.1016/S0022-4073(02)00331-X.
Morman, S. A., and G. S. Plumlee, 2013: The role of airborne mineral dusts in human disease. Aeolian Research, 9, 203–212, https://doi.org/10.1016/j.aeolia.2012.12.001.
Myhre, G., and Coauthors, 2013: Anthropogenic and Natural Radiative Forcing. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, 659–740.
Nousiainen, T., M. Kahnert, and H. Lindqvist, 2011: Can particle shape information be retrieved from light-scattering observations using spheroidal model particles. Journal of Quantitative Spectroscopy and Radiative Transfer, 112, 2213–2225, https://doi.org/10.1016/j.jqsrt.2011.05.008.
Oquab, M., L. Bottou, I. Laptev, and J. Sivic, 2014: Learning and transferring mid-level image representations using convolutional neural networks. Proc. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, IEEE, 1717–1724, https://doi.org/10.1109/CVPR.2014.222.
Rosenfeld, D., 2000: Suppression of rain and snow by urban and industrial air pollution. Science, 287, 1793–1796, https://doi.org/10.1126/science.287.5459.1793.
Rosenfeld, D., U. Lohmann, G. B. Raga, C. D. O’dowd, M. Kulmala, S. Fuzzi, A. Reissell, and M. O. Andreae, 2008: Flood or drought: How do aerosols affect precipitation. Science, 321, 1309–1313, https://doi.org/10.1126/science.1160606.
Saito, M., P. Yang, J. C. Ding, and X. Liu, 2021: A comprehensive database of the optical properties of irregular aerosol particles for radiative transfer simulations. J. Atmos. Sci., 78, 2089–2111, https://doi.org/10.1175/JAS-D-20-0338.1.
Satheesh, S. K., and K. K. Moorthy, 2005: Radiative effects of natural aerosols: A review. Atmos. Environ., 39, 2089–2110, https://doi.org/10.1016/j.atmosenv.2004.12.029.
Schmidhuber, J., 2017: Deep Learning. Encyclopedia of Machine Learning and Data Mining, C. Sammut and G. I. Webb, Eds., Springer, 338–348, https://doi.org/10.1007/978-1-4899-7502-7_909-1.
Shrivastava, A., A. Kundu, C. Dhir, D. Naik, and O. Tuzel, 2021: Optimize what matters: Training DNN-Hmm keyword spotting model using end metric. Preprints, ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, IEEE, 4000–4004, https://doi.org/10.1109/ICASSP39728.2021.9414797.
Silver, D., and Coauthors, 2016: Mastering the game of Go with deep neural networks and tree search. Nature, 529, 484–489, https://doi.org/10.1038/nature16961.
Sokolik, I., A. Andronova, and T. C. Johnson, 1993: Complex refractive index of atmospheric dust aerosols. Atmos. Environ. Part A Gene. Top., 27, 2495–2502, https://doi.org/10.1016/0960-1686(93)90021-P.
Sun, L.-H., L. Bi, and B. Q. Yi, 2021: The use of superspheroids as surrogates for modeling electromagnetic wave scattering by ice crystals. Remote Sensing, 13, 1733, https://doi.org/10.3390/rs13091733.
Tang, X. Y., L. Bi, W. S. Lin, D. Liu, K. J. Zhang, and W. J. Li, 2019: Backscattering ratios of soot-contaminated dusts at triple LiDAR wavelengths: T-matrix results. Optics Express, 27, A92–A116, https://doi.org/10.1364/OE.27.000A92.
Tegen, I., A. A. Lacis, and I. Fung, 1996: The influence on climate forcing of mineral aerosols from disturbed soils. Nature, 380, 419–422, https://doi.org/10.1038/380419a0.
Van De Hulst, H. C., 1981: Light Scattering by Small Particles. Dover, 485 pp.
Wang, G. H., and Coauthors, 2016: Persistent sulfate formation from London Fog to Chinese haze. Proceedings of the National Academy of Sciences of the United States of America, 133, 13 630–13 635, https://doi.org/10.1733/pnas.1616540113.
Wang, H., G. Y. Shi, X. Y. Zhang, S. L. Gong, S. C. Tan, B. Chen, H. Z. Che, and T. Li, 2015: Mesoscale modelling study of the interactions between aerosols and PBL meteorology during a haze episode in China Jing-Jin-Ji and its near surrounding region-Part 2: Aerosols’ radiative feedback effects. Atmospheric Chemistry and Physics, 15, 3277–3287, https://doi.org/10.5194/acp-15-3277-2015.
Wang, H., Y. Peng, X. Y. Zhang, H. L. Liu, M. Zhang, H. Z. Che, Y. L. Cheng, and Y. Zheng, 2018: Contributions to the explosive growth of PM2.5 mass due to aerosol-radiation feedback and decrease in turbulent diffusion during a red alert heavy haze in Beijing-Tianjin-Hebei, China. Atmospheric Chemistry and Physics, 18, 17 717–17 733, https://doi.org/10.5194/acp-18-17717-2018.
Wang, Z., L. Bi, B. Q. Yi, and X. Y. Zhang, 2019: How the inhomogeneity of wet sea salt aerosols affects direct radiative forcing. Geophys. Res. Lett., 46, 1805–1813, https://doi.org/10.1029/2018GL081193.
Wriedt, T., 2002: Using the T-matrix method for light scattering computations by non-axisymmetric particles: Superellipsoids and realistically shaped particles. Particle & Particle Systems Characterization, 19, 256–268, https://doi.org/10.1002/1521-4117(200208)19:4<256::AID-PPSC256>3.0.CO;2-8.
Xue, J. S., S. Y. Zhuang, G. F. Zhu, H. Zhang, Z. Q. Liu, Y. Liu, and Z. R. Zhuang, 2008: Scientific design and preliminary results of three-dimensional variational data assimilation system of GRAPES. Chinese Science Bulletin, 53, 3446–3457, https://doi.org/10.1007/s11434-008-0416-0.
Yang, P., and K. N. Liou, 1996a: Geometric-optics-integral-equation method for light scattering by non-spherical ice crystals. Appl. Opt., 35, 6568–6584, https://doi.org/10.1364/AO.35.006568.
Yang, P., and K. N. Liou, 1996b: Finite-difference time domain method for light scattering by small ice crystals in three-dimensional space. Journal of the Optical Society of America A, 13, 2072–2085, https://doi.org/10.1364/JOSAA.13.002072.
Yang, P., and Coauthors, 2007: Modeling of the scattering and radiative properties of non-spherical dust-like aerosols. Journal of Aerosol Science, 38, 995–1014, https://doi.org/10.1016/j.jaerosci.2007.07.001.
Yang, P., L. Bi, B. A. Baum, K.-N. Liou, G. W. Kattawar, M. I. Mishchenko, and B. Cole, 2013: Spectrally consistent scattering, absorption, and polarization properties of atmospheric ice crystals at wavelengths from 0.2 to 100 µm. J. Atmos. Sci., 70, 330–347, https://doi.org/10.1175/JAS-D-12-039.1.
Yee, K., 1966: Numerical solution of initial boundary value problems involving maxwell’s equations in isotropic media. IEEE Trans. Antennas Propag., 14, 302–307, https://doi.org/10.1109/TAP.1966.1138693.
Yurkin, M. A., and A. G. Hoekstra, 2011: The discrete-dipole-approximation code ADDA: Capabilities and known limitations. Journal of Quantitative Spectroscopy and Radiative Transfer, 112, 2234–2247, https://doi.org/10.1016/j.jqsrt.2011.01.031.
Zhang, R. Y., 2010: Getting to the critical nucleus of aerosol formation. Science, 328, 1366–1367, https://doi.org/10.1126/science.1189732.
Zhang, X. Y., J. Z. Wang, Y. Q. Wang, H. L. Liu, J. Y. Sun, and Y. M. Zhang, 2015: Changes in chemical components of aerosol particles in different haze regions in China from 2006 to 2013 and contribution of meteorological factors. Atmospheric Chemistry and Physics, 15, 12 935–12 952, https://doi.org/10.5194/acp-15-12935-2015.
Zhao, T. X.-P., I. Laszlo, O. Dubovik, B. N. Holben, J. Sapper, D. Tanré, and C. Pietras, 2003: A study of the effect of non-spherical dust particles on the AVHRR aerosol optical thickness retrievals. Geophys. Res. Lett., 30, 1317, https://doi.org/10.1029/2002GL016379.
Zhou, C., X. Zhang, S. Gong, Y. Wang, and M. Xue, 2016: Improving aerosol interaction with clouds and precipitation in a regional chemical weather modeling system. Atmospheric Chemistry and Physics, 11, 145–160, https://doi.org/10.5194/acp-16-145-2016.
Zhou, C. H., and Coauthors, 2008: Development and evaluation of an operational SDS forecasting system for East Asia: CUACE/Dust. Atmospheric Chemistry and Physics, 8, 787–798, https://doi.org/10.5194/acp-8-787-2008.
Zhou, C.-H., and Coauthors, 2012: Towards the improvements of simulating the chemical and optical properties of Chinese aerosols using an online coupled model-CUACE/Aero. Tellus B: Chemical and Physical Meteorology, 64, 18965, https://doi.org/10.3402/tellusb.v64i0.18965.
We acknowledge Ms. Rui LIU from the Training Center of Atmospheric Sciences of Zhejiang University for her efforts managing computing resources and Dr. Wushao LIN for organizing the II-TM optical properties. A portion of the computations was performed in the National Supercomputer Center in Guangzhou (NSCC-GZ), Tianjin (NSCC-TJ), and Wuxi (NSCC-WX), as well as the cluster at State Key Lab of CAD&CG at Zhejiang University. This research was supported by the NSFC Major Project (Grant Nos. 42090030, and 42090032), the National Natural Science Foundation of China (Grant Nos. 42022038, and 42075155), and the National Key Research and Development Program (2019YFC1510400).
• An optical property dataset computed through the T-matrix method can be highly compressed using an optimized deep neural network.
• The optical properties of super-spheroid models can be accurately and efficiently computed through a neural network.
• The neural network can be used instead of a conventional look-up table in atmospheric radiative transfer and related atmospheric models.
This paper is a contribution to the special issue on Cloud—Aerosol—Radiation—Precipitation Interaction: Progress and Challenges.
About this article
Cite this article
Yu, J., Bi, L., Han, W. et al. Application of a Neural Network to Store and Compute the Optical Properties of Non-Spherical Particles. Adv. Atmos. Sci. 39, 2024–2039 (2022). https://doi.org/10.1007/s00376-021-1375-5
- non-spherical particles
- light scattering
- super-spheroid model
- deep learning
- neural network