Abstract
The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument on the polar orbiter Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) is an elastic backscatter lidar that produces a global uniformly-calibrated aerosol data set. Several Calibration/Validation (Cal/Val) studies for CALIOP conducted with ground-based lidars and CALIOP data showed large aerosol profile disagreements, both random and systematic. In an attempt to better understand these problems, we undertook a series of ground-based lidar measurements in Atlanta, Georgia, which did not provide better agreement with CALIOP data than the earlier efforts, but rather prompted us to investigate the statistical limitations of such comparisons. Meaningful Cal/Val requires intercomparison data sets with small enough uncertainties to provide a check on the maximum expected calibration error. For CALIOP total attenuated backscatter, reducing the noise to the required level requires averaging profiles along the ground track for distances of at least 1,500 km. Representative comparison profiles often cannot be acquired with ground-based lidars because spatial aerosol inhomogeneities introduce systematic error into the averages. These conclusions have implications for future satellite lidar Cal/Val efforts, because planned satellite lidars measuring aerosol backscatter, wind vector, and CO2 concentration profiles may all produce data requiring considerable along-track averaging for meaningful Cal/Val.
Similar content being viewed by others
Introduction
When new spaceborne remote sensing instruments are launched, calibration-validation campaigns (commonly known as Cal/Val) are typically mounted. The purpose of Cal/Val is to characterize an instrument’s performance to ensure that it meets science needs. Calibration involves both pre-launch and post-launch measurements; here, we are concerned with the on-orbit phase of Cal/Val1. Validation often includes the process of assessing the overall confidence in a reported measurement result. This confidence is expressed as the estimated standard deviation assigned to a reported result, and it is often derived from intercomparisons between satellite measurements and independent reference measurements. It is this statistical definition that we address in this paper.
Lidar (light detection and ranging) instruments employ pulsed lasers with co-located receiver telescopes and fast electronics to record range-resolved profiles of backscattered light from atmospheric aerosols and molecules. The CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) instrument on the polar orbiter CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation) is an elastic backscatter lidar, launched in April 2006, which produces a global uniformly-calibrated data sets of atmospheric backscatter profiles for several purposes, including improved predictive capability for air quality2. CALIOP initially produced Level 1 and Level 2 data products that included vertical profiles of attenuated backscatter in which the signal from any given altitude is attenuated by the atmosphere above that altitude, because both transmitted and scattered light must pass through that portion of the atmosphere. In several Cal/Val studies, the CALIOP Level 1 data were compared with data from ground-based lidars and estimates of calibration biases were reported with standard deviations3,4,5. The comparisons showed large random uncertainties as well as systematic errors in the boundary layer.
Since November 2010, an expedited CALIOP Level 1.5 near-real-time product, usually provided between 6 and 30 h after downlink, has been made available by NASA for purposes of operational forecasting6. Under the MACC (Monitoring Atmospheric Composition and Climate) project, the Level 1.5 total attenuated backscatter profile was assessed for assimilation into the ECMWF (European Centre for Medium-Range Weather Forecasts) global forecasting model IFS-MOZART (Integrated Forecast System coupled to the Model for OZone And Related chemical Tracers). That assessment used a statistical appraisal of the agreement between CALIOP data and archived EARLINET (European Aerosol Research Lidar Network7) profiles using three measures of comparison: the mean bias, the factor of exceedance, and the correlation coefficient, along with scatterplots. The scatterplots showed ground and satellite aerosol profile differences as large as ±50%8.
In an attempt to better understand both the large random differences and the systematic differences between satellite and ground-based lidar measurements in previous studies, we undertook a series of ground-based lidar measurements at Agnes Scott College (ASC) in Atlanta, Georgia, which is only 12 km from the nighttime CALIOP ground track at nearest approach. For each episode, we acquired the corresponding CALIOP Level 1.5 total attenuated backscatter profile and compared it with our ground-based lidar data. Our measurements did not provide better agreement with CALIOP data than our previous statistical study using Level 1.5 data or the earlier ground-based efforts using Level 1 data. The significance of the Atlanta study is that it prompted us to re-examine the particular difficulties and challenges presented by Cal/Val for satellite lidars, and how those challenges were addressed by the methodologies of other research teams. Through this process, we arrived at the scientific question that we addressing:
What are the statistical considerations for planning a successful Cal/Val campaign for a spaceborne lidar, and what characteristics of the atmosphere must be considered?
The answers to this question may have significant implications for the planning of future satellite lidar Cal/Val campaigns.
Results
Atlanta Measurements
Because the ASC lidar project led us to formulate the scientific question but did not directly shed new light on it, the project is described under Methods and in the Supplementary Information, with only a few comments on the two types of errors presented here: (1) Random errors. The random noise in the CALIOP profiles is much higher than the ASC profiles due to their much longer measurement range and shorter integration times, and the daytime noise level is higher than nighttime due to the statistical fluctuations associated with solar background light. The subjective agreement between the CALIOP and ASC profiles in the aerosol-free atmosphere above 10 km is good in all cases in the sense that the CALIOP data excursions are roughly centered around the ASC profiles. This finding is not surprising, because the ASC profile conversion requires a retrieval algorithm that normalizes the total backscatter profile to the molecular backscatter profile, calculated from radiosonde air density, at altitudes above the aerosols. Good agreement only means that radiosonde data recorded within a few hours at a nearby site are representative of air density observed by CALIOP. (2) Systematic errors. In the boundary layer and a few kilometers above it, the disagreement between the ASC lidar and CALIOP is often systematic. The two profiles have differences well outside of CALIOP random noise excursions in about half of the episodes. Occasionally, the top of the boundary layer is not even apparent in the CALIOP data. No seasonal or day versus night trends in the disagreement were observed in this limited data set, despite a much closer CALIOP-to-ASC offset distance at night and the lack of solar background noise for night episodes. The best agreement occurs when aerosol backscatter is lowest, but once again this is not surprising because those profiles are dominated by molecular backscatter. Unfortunately, all of our daytime summer overpasses were dominated by clouds, so we could not compare profiles with high aerosol optical depth on a warm summer day with CALIOP data. Our comparisons in Fig. S2 do not show appreciably better agreement than in the previous statistical study8 or in Level 1 data comparisons at EARLINET ground lidar stations. This finding suggested to us that there are some underlying problems in ground lidar Cal/Val methods, and prompted us to look into statistical limitations due to the CALIOP random noise.
Statistical considerations
To address the large random errors, it is instructive to calculate the amount of averaging that would be required to reduce the rms noise of the CALIOP profiles to the 2% level reported for an airborne Cal/Val campaign9. The estimated maximum bias in the CALIOP calibration is 5%10, so 2% uncertainty is small enough to enable a check on that value. In our study, we used Level 1.5 total attenuated backscatter profiles, which have an along-track resolution of 20 km. Level 1.5 mean and uncertainty profiles for CALIOP’s closest approach to Atlanta on 13 November 2013 (nighttime) and 14 January 2014 (daytime) are shown in Fig. 1. Nighttime uncertainties are primarily due to signal noise (statistical fluctuations in the arrival rate of photons) and are a function of the signal level and hence, altitude. Daytime uncertainties are much larger and are nearly constant with altitude because they are dominated by solar background noise, which does not change with altitude. For the purpose of illustration, we will use an altitude of 5 km, which is a mid-range altitude in our study as well as in other reported CALIOP Cal/Val studies. The uncertainties as percentages of the mean at 5 km altitude are shown in Table 1 for all of our episodes, along with their average values. These values were averaged over a 1-km altitude region centered on 5 km. Notably, a 20-km along-track average has 17% uncertainty at night and 49% during day. These values are larger than the maximum 5% error that CALIOP Cal/Val is intended to check, necessitating more averaging in order to reduce the uncertainty.
To achieve 2% uncertainty for CALIOP Cal/Val for our case study, the noise level in the final averaged CALIOP profile must decrease by a factor of 8.6 on average, for nighttime. If this decrease in noise is proportional to the square root of the number of samples, 74 of the 20-km Level 1.5 nighttime profiles must be averaged together. This number of profiles corresponds to 1,480 km of along-track averaging. The daytime requirements are 590 profiles and 11,800 km. These calculations are for the best case scenario: a noise decrease proportional to the square root of the sample number is only expected for independent Gaussian samples with stationary statistics. Unless CALIOP and the comparison instrument measure the atmospheric aerosol profiles on the same optical paths, the statistics may not be stationary. The percentage uncertainties calculated here strictly pertain only to our Atlanta overpasses, but in general terms, this case study indicates that achieving 2% uncertainty at 5 km altitude in CALIOP Cal/Val studies requires along-track averaging of thousands of km during nighttime and tens of thousands of km during daytime. The required along-track averaging distances are shown in Fig. 2 for a range of desired uncertainties.
In our experimantal study, we used 100-km averages of CALIOP profiles, as did Grigas et al.8. The protocol employed by previous Cal/Val studies with Level 1 data used 5-km averaged profiles, which would require the acquisition of such profiles on nearly 300 nighttime overpasses in order to achieve 2% uncertainty. Acquiring 300 Cal/Val profiles in the same location is probably beyond the means of any one lidar station. In addition, the same aerosol backscatter profile would almost certainly not be observed by CALIOP and the ground lidar over the course of 300 profiles due to aerosol inhomogeneity. Averaging over long along-track distances reduces random error, but it may well introduce systematic error. This problem was demonstrated by our Atlanta study and by other CALIOP Cal/Val studies: the typical protocol of using ground track offsets <100 km does not guarantee that the CALIOP and ground lidar will see the same aerosol backscatter profile, especially in the boundary layer. Even with our relatively small 12-km offset, we frequently saw obvious differences in the boundary layer, as shown in Fig. S2. These results suggest that ground lidars often cannot provide Cal/Val intercomparison data that is useful for quantifying the CALIOP calibration error at its expected value of a few percent.
Discussion
In our experimental study, we recorded and analysed ground lidar data for comparison with CALIOP profiles, and found large random errors as well as frequent systematic disagreements below the free troposphere. These phenomena had been reported in several previous studies and studied in terms of the representativeness of ground lidar data from a given site for comparison with CALIOP. We then investigated the statistics of Cal/Val schemes to elucidate the circumstances under which quantitative Cal/Val of a satellite lidar is possible with ground lidars. We found that about 1,500 km of along-track averaging of CALIOP profiles is needed to achieve 2% uncertainty, for nighttime comparisons. None of the reported ground lidar Cal/Val efforts achieved such a small uncertainty.
The measurement methodologies of CALIOP, ground, and airborne lidars are distinctly different, as illustrated in Fig. 3. CALIOP averages quick measurements (moving at ~7 km/s along the ground track) of vertical atmospheric sheets, looking downward. In contrast, ground lidars average over longer time periods at a single location (usually urban), looking upward, using Raman or elastic backscatter lidars that require retrieval algorithms to calculate downward-looking total attenuated backscatter. The airborne measurements used a nadir-viewing HSRL (High Spectral Resolution Lidar) and flew under the CALIPSO satellite. The airborne instrument probed the same vertical atmospheric plane as CALIOP, allowing both instruments to average the same aerosol backscatter profiles. Therefore, the long averaging tracks did not introduce systematic errors, despite aerosol inhomogeneities, and distortions caused by converting zenith-viewing lidar signals to nadir-viewing were avoided. The CALIOP and HSRL measurements were not simultaneous, but Rogers et al.9 showed that HSRL profiles displaced in time by no more than 45 minutes from CALIOP did not show significant differences. At the airplane’s nominal ground speed of 110 m/s, 45 minutes corresponds to about 300 km of along-track averaging, which is significantly more than any of the ground-based efforts. More importantly, the lack of artefacts allows for multiple-flight averaging. Combined, these similarities between the CALIOP and airborne HSRL measurement methodologies allowed the airborne researchers to average CALIOP noise down to a few percent and conduct quantitative Cal/Val.
Our findings have implications for future satellite lidar Cal/Val efforts. Lidar data from satellites will always have greater noise than ground lidars, due to their great speed and orbital distance. To plan successful Cal/Val campaigns, the lidar community will require: (1) an estimate of the noise level in profiles generated by the satellite sensor; and (2) an estimate of the calibration uncertainty that the Cal/Val effort is intended to check. Researchers can then calculate the number of satellite profiles that must be averaged to provide quantitative Cal/Val, as well as the practicality of proposed Cal/Val campaigns in terms of the data acquisition time. For ground lidar measurements, researchers must also consider what is known about the homogeneity of the parameter of interest (aerosol backscatter, wind vector, CO2 concentration, etc.) at the lidar site to assess whether averaging multiple profiles can be accomplished without introducing systematic errors.
Methods
Instrumentation
The ground lidar data were acquired with a micropulse lidar developed by the Georgia Tech Research Institute (GTRI) and Agnes Scott College (ASC) under a grant from the National Science Foundation to develop an educational lidar11. The ASC lidar is located in Atlanta, Georgia. The lidar employs a 2500 Hz doubled Nd:YLF laser transmitting 14 μJ per pulse at 523.5 nm. To accommodate a wide range of altitudes, the lidar receiver has short-range and long-range channels that receive 9% and 91% of the light, respectively. The short range channel achieves complete crossover at 500 m. The ASC lidar is a simple elastic backscatter lidar, which records an uncalibrated signal in two receiver channels, digitized at 15-m altitude intervals and time-averaged as desired. More detail is given in Forrister et al.12. For the work reported here, each data file was the average from 80,000 laser pulses and the files were recorded continuously for two hours.
The CALIOP lidar is a nadir-viewing 532-nm elastic backscatter lidar. Its pulse repetition frequency (~20 Hz) determines the laser pulse spacing of 333 m along the ground track, or 3 pulses/km. The CALIPSO orbit is inclined such that the ground tracks repeat every sixteen days. In Atlanta, the closest ground tracks occur first during daytime, with an average offset distance of 80 km, and two days later during nighttime, with an average offset of 12 km.
The Level 1.5 data product used in our comparisons is a near-real-time product that was developed by NASA for potential assimilation into an aerosol forecast model. The Level 1.5 total attenuated backscatter profile is used in our comparisons. The term “total” indicates the sum of the parallel and cross polarization signals for molecular and aerosol backscatter at each range interval. CALIOP total attenuated backscatter data is calibrated and reported in absolute units (km−1 sr−1). The Level 1.5 data files also include the predominant aerosol type, which is typically Urban or Clear Continental near Atlanta, Georgia.
Measurement Procedure
The lidar ASC lidar is 8.3 km due east of the AERONET sun photometer at GTRI; both are located in Atlanta, Georgia. The National Weather Service (NWS) balloon launch site for upper air data is 48.5 km SSW of the sun photometer. A map designating instrument locations as well as nominal CALIOP ground tracks is shown in Fig. 4; details are listed in Forrister et al.12.
We initially believed that the small offset distance between ASC and the nighttime ground track made the ASC lidar a good choice for this comparison study. Also, the ASC lidar, with its low crossover altitude, was designed for studies in the boundary layer as well as higher altitudes. Our data acquisition episodes included a variety of wind directions, causing CALIOP and the ASC lidar to not always measure the same air mass. However, measurement biases did not appear to be correlated with wind direction or speed.
Additional details on the ASC lidar data acquisition and data analysis procedures are presented in the Supplementary Information.
Additional Information
How to cite this article: Gimmestad, G. et al. Comparisons of aerosol backscatter using satellite and ground lidars: implications for calibrating and validating spaceborne lidar. Sci. Rep. 7, 42337; doi: 10.1038/srep42337 (2017).
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
The Commissioning Phase and the Calibration/Validation Activities, Levrini, G. & Attema, E. Available at: http://www.esa.int/esapub/bulletin/bullet106/bul106_10.pdf. (Accessed: December 2016).
CALIPSO Mission Overview, Lorentz and Dunbar. Available at: https://www.nasa.gov/mission_pages/calipso/mission/index.html (Accessed: December 2016).
Pappalardo, G. et al. EARLINET correlative measurements for CALIPSO: first intercomparison results. J. Geophys. Res. Atmos 115, D00H19, doi: 10.1029/2009JD012147 (2002).
Mona, L. et al. One year of CNR-IMAA multi-wavelength Raman lidar measurements in correspondence of CALIPSO overpass: Level 1 products comparison. Atmospheric Chemistry and Physics Discuss. 9(2), 8429–8468, doi: 10.5194/acpd-9-8429-2009 (2009).
Mamouri, R. E. et al. Validation of CALIPSO spaceborne-derived attenuated backscatter coefficient profiles using a ground-based lidar in Athens, Greece. Atmos. Meas. Tech. 2, 513–522, doi: 10.5194/amt-2-513-2009 (2009).
Adapting CALIPSO Climate Measurements for Near Real Time Analyses and Forecasting, M., Vaughan et al. (2011). Available at: http://www-calipso.larc.nasa.gov/resources/pdfs/VaughanM_211104015final00251.pdf. (Accessed: May 2016).
Pappalardo, G. et al. EARLINET: towards an advanced sustainable European aerosol lidar network. Atmos. Meas. Tech. 7, 2389–2409, doi: 10.5194/amt-7-23892014 (2014).
Grigas, T. et al. On the performance of CALIOP near-real-time backscatter products compared to EARLINET. Atmospheric Chemistry and Physics 15, 12179–12191 (2015).
Rogers, R. et al. Assessment of the CALIOP lidar 532 nm attenuated backscatter calibration using the NASA LaRC airborne high spectral resolution lidar, Atmospheric Chemistry and Physics 11, 1295–1311, doi: 10.5194/acp-11-1295 (2011).
Powell, K. et al. CALIPSO lidar calibration algorithms: Part 1 – nighttime 532 nm parallel channel and 532 nm perpendicular channel. J. Atmos. Oceanic Technol. 26, 2015–2033, doi: 10.1175/2009JTECHA1242.1 (2009).
West, L. Atmospheric laser radar as an undergraduate educational experience, American Journal of Physics 74, 665–669 (2006).
Forrister, H., Roberts, D., Mercer, A. & Gimmestad, G. Infrared lidar observations of stratospheric aerosols. Applied Optics 53, D40–D48 (2014).
NASA A-Train figure. Available at: http://disc.sci.gsfc.nasa.gov/googleearth/agu_images/a-train_arch_new.jpg (accessed: July 2016).
Acknowledgements
The authors acknowledge the CALIPSO scientific team for granting access to the CALIOP Level 1.5 data. Development of the Agnes Scott College lidar was funded by the National Science Foundation under Grant No. 0116039. This material is partly based upon work supported by a National Science Foundation Graduate Research Fellowship under Grant No. DGE-1148903. We would also like to acknowledge NUI Galway support from SFI under Contract No. 12/RC/2302 (MaREI) and the European Commission under MACC-II, and the Irish Environmental Protection Agency.
Author information
Authors and Affiliations
Contributions
The paper is an outgrowth of reference 9, resulting from a collaboration between the National University of Ireland Galway (NUIG) and the Georgia Institute of Technology in Atlanta, Georgia, USA G.G. was the primary author of the paper, and he acquired the nighttime ASC lidar data when he was with Georgia Tech; H.F. acquired the rest of the ASC data and the supporting data, analyzed all of it, prepared the figures, and edited the manuscript; T.G. assisted with the Atlanta experiment plan, retrieved the CALIOP data, and assisted with revisions to the manuscript; and C.O. supervised the overall project at NUIG and contributed to writing the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Gimmestad, G., Forrister, H., Grigas, T. et al. Comparisons of aerosol backscatter using satellite and ground lidars: implications for calibrating and validating spaceborne lidar. Sci Rep 7, 42337 (2017). https://doi.org/10.1038/srep42337
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/srep42337
- Springer Nature Limited