Skip to main content

Advertisement

SpringerLink
Log in

Computational end-to-end and super-resolution methods to improve thermal infrared remote sensing for agriculture

  • Published:
Precision Agriculture Aims and scope Submit manuscript

Abstract

Increasing global water deficit and demand for yield improvement call for high-resolution monitoring of irrigation, crop water stress, and crops' general condition. To provide high spatial resolution with high-temperature accuracy, remote sensing is conducted at low altitudes using radiometric longwave thermal infrared cameras. However, the radiometric cameras' price, and the low altitude leading to low coverage in a given time, limit the use of radiometric aerial surveys for agricultural needs. This paper presents progress toward solving both limitations using algorithmic and computational imaging methods: stabilizing the readout of low-cost thermal cameras to obtain radiometric data, and improving the latter's low resolution by applying convolutional neural network-based super-resolution. The two methods were merged by an end-to-end algorithm pipeline, providing a large mosaicked image of the field. First, the potential capabilities of a joint estimation method to correct unknown offset and gain were simulated on remotely sensed agricultural data. Comparison to ground-truth measurements showed radiometric accuracy with a root mean square error (RMSE) of 1.3 °C to 1.8 °C. Then, the proposed super-resolution method was demonstrated on experimental and simulated remotely sensed agricultural data. Preliminary experimental results showed 50% improvement in image sharpness relative to bicubic interpolation. The performance of the algorithm was evaluated on 22 simulated cases at × 2 and × 4 magnification. Finally, image mosaicking using the proposed pipeline was demonstrated. A mosaicked image composed of sub-images pre-processed by the proposed computational methods resulted in a RMSE in temperature of 0.8 °C, as compared to 8.2 °C without the initial processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Source images were taken with a FLIR A655Sc camera. Ground sampling distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 4

Source images were taken with a FLIR A655Sc camera. Ground spatial distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 5

Source images were taken with a FLIR A655Sc camera. Ground sampling distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 6

Source images were taken with a FLIR A655Sc camera. Ground sampling distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 7

Source images were taken with FLIR A655Sc camera. Ground sampling distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 8

Source images were taken with a FLIR A655Sc camera. Ground spatial distance was 35 cm, sub-image area is 90.6 × 90.6 m2

Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Alchanatis, V., Cohen, Y., Cohen, S., Möller, M., Sprinstin, M., Meron, M., et al. (2010). Evaluation of different approaches for estimating and mapping crop water status in cotton with thermal imaging. Precision Agriculture, 11, 27–41.

    Article  Google Scholar 

  • Ben-Gal, A., Kool, D., Agam, N., van Halsema, G. E., Yermiyahu, U., Yafe, A., et al. (2010). Whole-tree water balance and indicators for short-term drought stress in non-bearing 'Barnea’ olives. Agricultural Water Management, 98, 124–133. https://doi.org/10.1016/j.agwat.2010.08.008.

    Article  Google Scholar 

  • Berni, J. A. J., Zarco-Tejada, P. J., Sepulcre-Canto, G., Fereres, E., & Villalobos, F. (2009). Mapping canopy conductance and CWSI in olive orchards using high resolution thermal remote sensing imagery. Remote Sensing of Environment, 113, 2380–2388.

    Article  Google Scholar 

  • Bhan, R. K., Saxena, R. S., Jalwania, C. R., & Lomash, S. K. (2009). Uncooled infrared microbolometer arrays and their characterisation techniques. Defence Science Journal, 59(6), 580–589.

    Article  CAS  Google Scholar 

  • Cohen, Y., Alchanatis, V., Meron, M., Saranga, Y., & Tsipris, J. (2005). Estimation of leaf water potential by thermal imagery and spatial analysis. Journal of Experimental Botany, 56(417), 1843–1852.

    Article  CAS  Google Scholar 

  • Cohen, Y., Alchanatis, V., Prigojin, A., Levi, A., & Soroker, V. (2012). Use of aerial thermal imaging to estimate water status of palm trees. Precision Agriculture. https://doi.org/10.1007/s11119-011-9232-7.

    Article  Google Scholar 

  • Costa, J. M., Grant, O. M., & Chaves, M. M. (2013). Thermography to explore plant–environment interactions. Journal of Experimental Botany, 64(13), 3937–3949.

    Article  CAS  Google Scholar 

  • Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. In D. Fleet, T. Pajdla, B. Schiele, & T. Tuytelaars (Eds.), Computer vision—ECCV 2014. ECCV 2014. Lecture notes in computer science (Vol 8692, pp. 184–199). Cham: Springer.

    Google Scholar 

  • Ehrler, W. L. (1973). Cotton leaf temperatures as related to soil water depletion and meteorological factors. Agronomy Journal, 65, 404–409.

    Article  Google Scholar 

  • Elad, M., & Feuer, A. (1997). Restoration of a single super-resolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing, 6, 1646–1658.

    Article  CAS  Google Scholar 

  • Farsiu, S., Dirk Robinson, M., Elad, M., & Milanfar, P. (2004). Fast and robust multiframe super resolution. IEEE Transactions on Image Processing, 13, 1327–1343.

    Article  Google Scholar 

  • Gates, D. M. (1964). Leaf temperature and transpiration. Agronomy Journal, 56, 273–277.

    Article  Google Scholar 

  • Grant, O. M., Tronina, L., Jones, H. G., & Chaves, M. M. (2007). Exploring thermal imaging variables for the detection of stress responses in grapevine under different irrigation regimes. Journal of Experimental Botany, 58, 815–825.

    Article  CAS  Google Scholar 

  • Holman, J. P. (1989). Heat transfer. Boston: McGraw Hill.

    Google Scholar 

  • Jia, W., Zhang, H., He, X., & Wu, Q. (2006). A comparison on histogram based image matching methods. In 2006 IEEE International Conference on Video and Signal Based Surveillance, Sydney, Australia (pp. 97–97). https://doi.org/10.1109/AVSS.2006.5

  • Jones, H. G. (1994). Plant and microclimate (2nd ed.). Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.182

  • Klapp, I., Brand, O., Yafin, P., Papini, S., Oz, N., Bahat, I., et al. (2019). Using computational optics for agricultural monitoring with an emphasis on irrigation management zones. In J. V. Stafford (Ed.), Precision agriculture ’19. Wageningen: Wageningen Academic Publishers. https://doi.org/10.3920/978-90-8686-888-9_82

    Google Scholar 

  • Klapp, I., Papini, S., & Sochen, N. (2017). Radiometric imaging by double exposure and gain calibration. Applied Optics, 56, 5639–5647.

    Article  Google Scholar 

  • Kopeika, N. S. (1998). A system engineering approach to imaging. Bellingham, WA: SPIE Optical Engineering Press.

    Book  Google Scholar 

  • Kusnierek, K., & Korsaeth, A. (2014). Challenges in using an analog uncooled microbolometer thermal camera to measure crop temperature. International Journal of Agricultural and Biological Engineering, 7, 60–74.

    Google Scholar 

  • LSQR (2018). Sparse Equations and Least Squares. Retrieved September 15, 2020, from https://web.stanford.edu/group/SOL/software/lsqr/.

  • Mandanici, E., Tavasci, L., Corsini, F., & Gandolfi, S. (2019). A multi-image super-resolution algorithm applied to thermal imagery. Applied Geomatics, 11, 215–228.

    Article  Google Scholar 

  • Manickavasagan, A., Digvir, J. S., White, N. D. G., & Paliwal, J. (2005). Applications of thermal imaging in agriculture—a review. In: CSAE/SCGR 2005 Meeting, Paper No. 05–002. Retrieved September 15, 2020, from https://www.csbe-scgab.ca/docs/meetings/2005/05-002.pdf.

  • Mendelowitz, S., Klapp, I., & Mendlovic, D. (2013). Design of an image restoration algorithm for the TOMBO imaging system. Journal of the Optical Society of America A, 30, 1193–1204.

    Article  Google Scholar 

  • Nugent, P. W., & Shaw, J. A. (2014). Calibration of uncooled LWIR microbolometer imagers to enable long-term field deployment. In: Proceedings, Infrared imaging systems: design, analysis, modeling, and testing XXV (Vol 9071, p. 90710V). https://doi.org/10.1117/12.2053082

  • Oz, N., Sochen, N., & Klapp, I. (2020). Rapid super resolution for infrared imagery. Optics Express. https://doi.org/10.1364/OE.389926.

    Article  PubMed  Google Scholar 

  • Papini, S., Yafin, P., Klapp, I., & Sochen, N. (2018). Joint estimation of unknown radiometric data, gain and offset from thermal images. Applied Optics, 57, 10390–10401.

    Article  Google Scholar 

  • Petrović, I., Marjanović, M., Ćosić, M., Savić, S., & Cvijanović, S. (2016). Infra-red thermography for detecting drought in agriculture crops and scheduling irrigation. Economics of Agriculture, 2, 461–469.

    Google Scholar 

  • Rivenson, Y., Göröcs, Z., Günaydin, H., Zhang, Y., Wang, H., & Ozcan, A. (2017). Deep learning microscopy. Optica, 4(11), 1437–1443.

    Article  Google Scholar 

  • Sun, J., Xu, Z., & Shum, H.-Y. (2008). Image super-resolution using gradient profile prior. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2008.4587659

  • Tanner, C. B. (1963). Plant temperatures. Agronomy Journal, 55, 210–211.

    Article  Google Scholar 

  • Tempelhahn, A., Budzier, H., Krause, V., & Gerlach, G. (2016). Shutter-less calibration of uncooled infrared cameras. Journal of Sensors and Sensor Systems, 5, 916.

    Article  Google Scholar 

  • Tilling, K., O’Leary, G. J., Ferwerda, J. G., Jones, S. D., Fitzgerald, G. J., Rodriguez, D., et al. (2007). Remote sensing of nitrogen and water stress in wheat. Field Crops Research, 104, 77–85.

    Article  Google Scholar 

  • United Nations (2017). Water for sustainable food and agriculture. Rome: FAO, ISBN 978-92-5-109977-3

  • Vidal, D., & Pitarma, R. (2019). Infrared thermography applied to tree health assessment: A review. Agriculture, 9(7), 156. https://doi.org/10.3390/agriculture9070156.

    Article  Google Scholar 

  • Zhou, J., Kwan, C., & Budavari, B. (2016). Hyperspectral image super-resolution: A hybrid color mapping approach. Journal of Applied Remote Sensing. https://doi.org/10.1117/1.JRS.10.035024.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Israeli Ministry of Agriculture's Kandel Program for funding this research under Grant No. 20-12-0030.

Author information

Authors and Affiliations

  1. Institute of Agricultural Engineering, Agricultural Research Organization, Volcani Center, Rishon LeZion, Israel

    Iftach Klapp, Peretz Yafin, Navot Oz, Omri Brand, Idan Bahat, Eitan Goldshtein, Yafit Cohen & Victor Alchanatis

  2. Department of Computer Science, Tel Aviv University, 69978, Tel Aviv, Israel

    Peretz Yafin

  3. School of Electrical Engineering, Tel Aviv University, 69978, Tel Aviv, Israel

    Navot Oz

  4. Department of Mathematics, Tel Aviv University, 69978, Tel Aviv, Israel

    Omri Brand & Nir Sochen

  5. The Robert H. Smith Institute of Plant Sciences and Genetics in Agriculture, Faculty of Agriculture, Food and Environment, The Hebrew University of Jerusalem, 76100, Rehovot, Israel

    Idan Bahat

Authors
  1. Iftach Klapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peretz Yafin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Navot Oz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Omri Brand
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Idan Bahat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eitan Goldshtein
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yafit Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Victor Alchanatis
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Nir Sochen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iftach Klapp.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klapp, I., Yafin, P., Oz, N. et al. Computational end-to-end and super-resolution methods to improve thermal infrared remote sensing for agriculture. Precision Agric 22, 452–474 (2021). https://doi.org/10.1007/s11119-020-09746-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11119-020-09746-y

Keywords

Search

Navigation