Advertisement

Monotonicity Prior for Cloud Tomography

Conference paper
  • 455 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12363)

Abstract

We introduce a differentiable monotonicity prior, useful to express signals of monotonic tendency. An important natural signal of this tendency is the optical extinction coefficient, as a function of altitude in a cloud. Cloud droplets become larger as vapor condenses on them in an updraft. Reconstruction of the volumetric structure of clouds is important for climate research. Data for such reconstruction is multi-view images of each cloud taken simultaneously. This acquisition mode is expected by upcoming future spaceborne imagers. We achieve three-dimensional volumetric reconstruction through stochastic scattering tomography, which is based on optimization of a cost function. Part of the cost is the monotonicity prior, which helps to improve the reconstruction quality. The stochastic tomography is based on Monte-Carlo (MC) radiative transfer. It is formulated and implemented in a coarse-to-fine form, making it scalable to large fields.

Keywords

Scattering Regularization Physics-based vision 

Notes

Acknowledgments

We thank Ilan Koren, Eshkol Eytan Liebeskind, and Tom Dror-Schwartz for useful discussions. We thank Johanan Erez, Ina Talmon and Daniel Yagodin for technical support. Yoav Schechner is the Mark and Diane Seiden Chair in Science at the Technion. He is a Landau Fellow - supported by the Taub Foundation. His work was conducted in the Ollendorff Minerva Center. Minvera is funded through the BMBF. This research is funded by the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation program (grant agreement No 810370: CloudCT). Aviad Levis is a Zuckerman Postdoctoral Fellow.

Supplementary material

504473_1_En_17_MOESM1_ESM.pdf (656 kb)
Supplementary material 1 (pdf 655 KB)

References

  1. 1.
    Aides, A., Levis, A., Holodovsky, V., Schechner, Y.Y., Althausen, D., Vainiger, A.: Distributed sky imaging radiometry and tomography. In: ICCP, pp. 1–12. IEEE (2020)Google Scholar
  2. 2.
    Aides, A., Schechner, Y.Y., Holodovsky, V., Garay, M.J., Davis, A.B.: Multi sky-view 3D aerosol distribution recovery. Opt. Express 21(22), 25820–25833 (2013)CrossRefGoogle Scholar
  3. 3.
    Alterman, M., Schechner, Y.Y., Vo, M., Narasimhan, S.G.: Passive tomography of turbulence strength. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 47–60. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10593-2_4CrossRefGoogle Scholar
  4. 4.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 2, 218–233 (2003)CrossRefGoogle Scholar
  5. 5.
    Berman, D., Treibitz, T., Avidan, S.: Air-light estimation using haze-lines. In: ICCP, pp. 1–9. IEEE (2017)Google Scholar
  6. 6.
    Binzoni, T., Leung, T.S., Gandjbakhche, A.H., Ruefenacht, D., Delpy, D.: The use of the Henyey-Greenstein phase function in Monte Carlo simulations in biomedical optics. Phys. Med. Biol. 51(17), N313 (2006)CrossRefGoogle Scholar
  7. 7.
    Chung, D., Matheou, G.: Large-eddy simulation of stratified turbulence. Part i: a vortex-based subgrid-scale model. J. Atmos. Sci. 71(5), 1863–1879 (2014)CrossRefGoogle Scholar
  8. 8.
    Davis, C., Emde, C., Harwood, R.: A 3-D polarized reversed Monte Carlo radiative transfer model for Millimeter and submillimeter passive remote sensing in cloudy atmospheres. IEEE Trans. Geosci. Remote Sens. 43(5), 1096–1101 (2005)CrossRefGoogle Scholar
  9. 9.
    Deng, X., Jiao, S., Bitterli, B., Jarosz, W.: Photon surfaces for robust, unbiased volumetric density estimation. ACM Trans. Graph. 38(4), 46 (2019)CrossRefGoogle Scholar
  10. 10.
    Diner, D.J., Martonchik, J.V.: Atmospheric transmittance from spacecraft using multiple view angle imagery. Appl. Opt. 24(21), 3503–3511 (1985)CrossRefGoogle Scholar
  11. 11.
    Evans, K.F.: The spherical harmonics discrete ordinate method for three-dimensional atmospheric radiative transfer. J. Atmos. Sci. 55(3), 429–446 (1998)CrossRefGoogle Scholar
  12. 12.
    Frisvad, J.R.: Importance sampling the Rayleigh phase function. JOSA 28(12), 2436–2441 (2011)CrossRefGoogle Scholar
  13. 13.
    Georgiev, I., Misso, Z., Hachisuka, T., Nowrouzezahrai, D., Křivánek, J., Jarosz, W.: Integral formulations of volumetric transmittance. ACM Trans. Graph. 38(6), 1–17 (2019)CrossRefGoogle Scholar
  14. 14.
    Geva, A., Schechner, Y.Y., Chernyak, Y., Gupta, R.: X-ray computed tomography through scatter. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 37–54. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01264-9_3CrossRefGoogle Scholar
  15. 15.
    Gkioulekas, I., Levin, A., Zickler, T.: An evaluation of computational imaging techniques for heterogeneous inverse scattering. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 685–701. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_42CrossRefGoogle Scholar
  16. 16.
    Gkioulekas, I., Zhao, S., Bala, K., Zickler, T., Levin, A.: Inverse volume rendering with material dictionaries. ACM Trans. Graph. 32(6), 162 (2013)CrossRefGoogle Scholar
  17. 17.
    Gorbunov, M.E., Sokolovky, S., Bengtsson, L.: Space refractive tomography of the atmosphere: modeling of direct and inverse problems (1996)Google Scholar
  18. 18.
    Gregson, J., Krimerman, M., Hullin, M.B., Heidrich, W.: Stochastic tomography and its applications in 3D imaging of mixing fluids. ACM Trans. Graph. 31(4), 1–52 (2012)CrossRefGoogle Scholar
  19. 19.
    Holodovsky, V., Schechner, Y.Y., Levin, A., Levis, A., Aides, A.: In-situ multi-view multi-scattering stochastic tomography. In: ICCP, pp. 1–12. IEEE (2016)Google Scholar
  20. 20.
    Iwabuchi, H.: Efficient Monte Carlo methods for radiative transfer modeling. J. Atmos. Sci. 63(9), 2324–2339 (2006)CrossRefGoogle Scholar
  21. 21.
    Jakob, W.: Mitsuba renderer (2010). http://www.mitsuba-renderer.org
  22. 22.
    Kaftory, R., Schechner, Y.Y., Zeevi, Y.Y.: Variational distance-dependent image restoration. In: CVPR, pp. 1–8. IEEE (2007)Google Scholar
  23. 23.
    Kalashnikova, O.V., Garay, M.J., Davis, A.B., Diner, D.J., Martonchik, J.V.: Sensitivity of multi-angle photo-polarimetry to vertical layering and mixing of absorbing aerosols: quantifying measurement uncertainties. J. Quant. Spectrosc. Radiat. Transf. 112(13), 2149–2163 (2011)CrossRefGoogle Scholar
  24. 24.
    Kato, H., Ushiku, Y., Harada, T.: Neural 3D mesh renderer. In: CVPR, pp. 3907–3916. IEEE (2018)Google Scholar
  25. 25.
    Khungurn, P., Schroeder, D., Zhao, S., Bala, K., Marschner, S.: Matching real fabrics with micro-appearance models. ACM Trans. Graph. 35(1), 1–1 (2015)CrossRefGoogle Scholar
  26. 26.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  27. 27.
    Kokhanovsky, A.A.: Light Scattering Media Optics. Springer, Heidelberg (2004)Google Scholar
  28. 28.
    Kratz, L., Nishino, K.: Factorizing scene albedo and depth from a single foggy image. In: ICCV, pp. 1701–1708. IEEE (2009)Google Scholar
  29. 29.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)CrossRefGoogle Scholar
  30. 30.
    Levis, A., Aides, A.: pyshdom (2019). https://github.com/aviadlevis/pyshdom
  31. 31.
    Levis, A., Schechner, Y.Y., Aides, A., Davis, A.B.: Airborne three-dimensional cloud tomography. In: ICCV, pp. 3379–3387. IEEE (2015)Google Scholar
  32. 32.
    Levis, A., Schechner, Y.Y., Davis, A.B.: Multiple-scattering microphysics tomography. In: CVPR, pp. 6740–6749. IEEE (2017)Google Scholar
  33. 33.
    Levis, A., Schechner, Y.Y., Davis, A.B., Loveridge, J.: Multi-view polarimetric scattering cloud tomography and retrieval of droplet size. arXiv preprint arXiv:2005.11423 (2020)
  34. 34.
    Lu, M.-L., et al.: Aerosol-cloud relationships in continental shallow cumulus. J. Geophys. Res. 113, D15201 (2008).  https://doi.org/10.1029/2007JD009354
  35. 35.
    Marshak, A., Davis, A.: 3D Radiative Transfer in Cloudy Atmospheres. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  36. 36.
    Martonchik, J.V., et al.: Techniques for the retrieval of aerosol properties over land and ocean using multiangle imaging. IEEE Trans. Geosci. Remote Sens. 36(4), 1212–1227 (1998)CrossRefGoogle Scholar
  37. 37.
    Matheou, G., Chung, D.: Large-eddy simulation of stratified turbulence. Part ii: application of the stretched-vortex model to the atmospheric boundary layer. J. Atmos. Sci. 71(12), 4439–4460 (2014)CrossRefGoogle Scholar
  38. 38.
    Mayer, B.: Radiative transfer in the cloudy atmosphere. In: EPJ Web of Conferences, vol. 1, pp. 75–99. EDP Sciences (2009)Google Scholar
  39. 39.
    McFarlane, S.A., Grabowski, W.W.: Optical properties of shallow tropical cumuli derived from ARM ground-based remote sensing. Geophys. Res. Lett. 34, L06808 (2007).  https://doi.org/10.1029/2006GL028767
  40. 40.
    Messer, H., Zinevich, A., Alpert, P.: Environmental sensor networks using existing wireless communication systems for rainfall and wind velocity measurements. Instrum. Meas. Mag. 15(2), 32–38 (2012)Google Scholar
  41. 41.
    Mobley, C.D.: Light and Water: Radiative Transfer in Natural Waters. Academic Press, Cambridge (1994)Google Scholar
  42. 42.
    Narasimhan, S.G., Ramamoorthi, R., Nayar, S.K.: Analytic rendering of multiple scattering in participating media. Technical report, Columbia University (2004)Google Scholar
  43. 43.
    Narasimhan, S.G., Gupta, M., Donner, C., Ramamoorthi, R., Nayar, S.K., Wann-Jensen, H.: Acquiring scattering properties of participating media by dilution. ACM Trans. Graph. 25(3), 1003–1012 (2006)CrossRefGoogle Scholar
  44. 44.
    Narasimhan, S.G., Nayar, S.K.: Vision and the atmosphere. Int. J. Comput. Vis. 48(3), 233–254 (2002)CrossRefGoogle Scholar
  45. 45.
    Narasimhan, S.G., Nayar, S.K.: Interactive (de) weathering of an image using physical models. In: IEEE Workshop on Color and Photometric Methods in Computer Vision, vol. 6, p. 1. France (2003)Google Scholar
  46. 46.
    Narasimhan, S.G., Nayar, S.K., Sun, B., Koppal, S.J.: Structured light in scattering media. In: ICCV, vol. 1, pp. 420–427. IEEE (2005)Google Scholar
  47. 47.
    Nguyen-Phuoc, T.H., Li, C., Balaban, S., Yang, Y.: RenderNet: a deep convolutional network for differentiable rendering from 3D shapes. In: NeurIPS, pp. 7891–7901 (2018)Google Scholar
  48. 48.
    Nimier-David, M., Vicini, D., Zeltner, T., Jakob, W.: Mitsuba 2: a retargetable forward and inverse renderer. ACM Trans. Graph. 38(6) (2019).  https://doi.org/10.1145/3355089.3356498
  49. 49.
    Novák, J., Georgiev, I., Hanika, J., Jarosz, W.: Monte Carlo methods for volumetric light transport simulation. In: Computer Graphics Forum, vol. 37, pp. 551–576. Wiley Online Library (2018)Google Scholar
  50. 50.
    Okabe, T., Sato, I., Sato, Y.: Spherical harmonics vs. haar wavelets: basis for recovering illumination from cast shadows. In: CVPR, vol. 1, pp. 50–57. IEEE (2004)Google Scholar
  51. 51.
    Pfeiffer, G.T., Sato, Y.: On stochastic optimization methods for Monte Carlo least-squares problems. arXiv preprint arXiv:1804.10079 (2018)
  52. 52.
    Pharr, M., Jakob, W., Humphreys, G.: Physically Based Rendering: from Theory to Implementation. Morgan Kaufmann, Burlington (2016)Google Scholar
  53. 53.
    Schaul, L., Fredembach, C., Süsstrunk, S.: Color image dehazing using the near-infrared. In: ICIP, pp. 1629–1632. IEEE (2009)Google Scholar
  54. 54.
    Schilling, K., Schechner, Y.Y., Koren, I.: CloudCT - computed tomography of clouds by a small satellite formation. In: IAA symposium on Small Satellites for Earth Observation (2019)Google Scholar
  55. 55.
    Sheinin, M., Schechner, Y.Y.: The next best underwater view. In: CVPR, pp. 3764–3773. IEEE (2016)Google Scholar
  56. 56.
    Spier, O., Treibitz, T., Gilboa, G.: In situ target-less calibration of turbid media. In: ICCP, pp. 1–9. IEEE (2017)Google Scholar
  57. 57.
    Vainiger, A., Schechner, Y.Y., Treibitz, T., Avni, A., Timor, D.S.: Optical wide-field tomography of sediment resuspension. Opt. Express 27(12), A766–A778 (2019)CrossRefGoogle Scholar
  58. 58.
    Villefranque, N., et al.: A path-tracing Monte Carlo library for 3-D radiative transfer in highly resolved cloudy atmospheres. J. Adv. Model. Earth Syst. 11(8), 2449–2473 (2019)CrossRefGoogle Scholar
  59. 59.
    Wright, T.E., Burton, M., Pyle, D.M., Caltabiano, T.: Scanning tomography of \({\rm SO}_2\) distribution in a volcanic gas plume. Geophys. Res. Lett. 35, L17811 (2008).  https://doi.org/10.1029/2008GL034640
  60. 60.
    Zhang, S., Xue, H., Feingold, G.: Vertical profiles of droplet effective radius in shallow convective clouds. Atmos. Chem. Phys. 11(10), 4633–4644 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Viterbi Faculty of Electrical EngineeringTechnion - Israel Institute of TechnologyHaifaIsrael
  2. 2.Computing and Mathematical Sciences DepartmentCalifornia Institute of TechnologyPasadenaUSA

Personalised recommendations