Skip to main content
SpringerLink
Log in

CorNet: Unsupervised Deep Homography Estimation for Agricultural Aerial Imagery

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 Workshops (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12540))

Included in the following conference series:

Abstract

Efficient and accurate estimation of homographies among images is the first step in mosaicking crop fields for phenotyping. The current strategy uses sophisticated vehicles that have excellent telemetry to hover over a grid of waypoints, imaging each one. This approach simplifies homography estimation, but precludes more flexible, adaptive protocols that can collect richer information. It also makes aerial phenotyping impractical for many researchers and farmers. We are developing an alternative strategy that uses consumer-grade vehicles, freely flown over a variety of trajectories, to collect video. We have developed an unsupervised deep learning network that estimates the sequence of planar homography matrices of our corn fields from imagery, without using any metadata to correct estimation errors. The vehicle was freely flown using a variety of trajectories and camera views. Our system, CorNet, performed faster than and with comparable accuracy to the gold standard ASIFT algorithm in many challenging cases.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016). https://arxiv.org/abs/1805.09662

  2. Agarwal, S., et al.: Building Rome in a day. Commun. ACM 54(10), 105–112 (2011)

    Google Scholar 

  3. Aktar, R., Aliakbarpour, H., Bunyak, F., Seetharaman, G., Palaniappan, K.: Performance evaluation of feature descriptors for aerial imagery mosaicking. In: Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–7. IEEE (2018)

    Google Scholar 

  4. Aktar, R., Prasath, V.S., Aliakbarpour, H., Sampathkumar, U., Seetharaman, G., Palaniappan, K.: Video haze removal and Poisson blending based mini-mosaics for wide area motion imagery. In: Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–7. IEEE (2016)

    Google Scholar 

  5. Aktar, R., et al.: Robust mosaicking of maize fields from aerial imagery. Appl. Plant Sci. 8, e11387 (2020). https://doi.org/10.1002/aps3.11387, https://bsapubs.onlinelibrary.wiley.com/doi/full/10.1002/aps3.11387

  6. Aliakbarpour, H., Palaniappan, K., Seetharaman, G.: Stabilization of airborne video using sensor exterior orientation with analytical homography modeling. In: Sergiyenko, O., Flores-Fuentes, W., Mercorelli, P. (eds.) Machine Vision and Navigation, pp. 579–595. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-22587-2_17

    Chapter  Google Scholar 

  7. Avola, D., Cinque, L., Foresti, G.L., Martinel, N., Pannone, D., Piciarelli, C.: A UAV video dataset for mosaicking and change detection from low-altitude flights. IEEE Trans. Syst. Man Cybern. Syst. PP, 1–11 (2018)

    Google Scholar 

  8. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32

    Chapter  Google Scholar 

  9. Bentoutou, Y., Taleb, N., Kpalma, K., Ronsin, J.: An automatic image registration for applications in remote sensing. IEEE Trans. Geosci. Remote Sens. 43(9), 2127–2137 (2005)

    Article  Google Scholar 

  10. Blancon, J., et al.: A high-throughput model-assisted method for phenotyping maize green leaf area index dynamics using unmanned aerial vehicle imagery. Fron. Pl. Sci. 10, 685 (2019). https://doi.org/10.3389/fpls.2019.00685

    Article  Google Scholar 

  11. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. (CSUR) 24(4), 325–376 (1992)

    Article  Google Scholar 

  12. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2006). https://doi.org/10.1007/s11263-006-0002-3

  13. Chen, H.M., Arora, M.K., Varshney, P.K.: Mutual information-based image registration for remote sensing data. Int. J. Remote Sens. 24(18), 3701–3706 (2003)

    Article  Google Scholar 

  14. Condorelli, G.E., et al.: Comparative aerial and ground based high throughput phenotyping for the genetic dissection of NDVI as a proxy for drought adaptive traits in durum wheat. Frontiers Plant Sci. 9 (2018). https://doi.org/10.3389/fpls.2018.00893

  15. DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation. arXiv preprint arXiv:1606.03798 (2016)

  16. Enciso, J., et al.: Validation of agronomic UAV and field measurements for tomato varieties. Comput. Electron. Agric. 158, 278–283 (2019). https://doi.org/10.1016/j.compag.2019.02.011

    Article  Google Scholar 

  17. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In: Readings in Computer Vision, pp. 726–740. Elsevier (1987)

    Google Scholar 

  18. Gao, K., AliAkbarpour, H., Palaniappan, K., Seetharaman, G.: Evaluation of feature matching in aerial imagery for structure-from motion and bundle adjustment. In: Geospatial Informatics, Motion Imagery, and Network Analytics VIII, vol. 10645, p. 106450J. International Society for Optics and Photonics (2018)

    Google Scholar 

  19. Gnädinger, F., Schmidhalter, U.: Digital counts of maize plants by unmanned aerial vehicles (UAVs). Remote Sens. 9(6), 544 (2017)

    Article  Google Scholar 

  20. Gracia-Romero, A., Kefauver, S.C., Fernandez-Gallego, J.A., Vergara-Díaz, O., Teresa Nieto-Taladriz, M., Araus, J.L.: UAV and ground image-based phenotyping: a proof of concept with durum wheat. Remote Sens. 11(10), 1244 (2019). https://doi.org/10.3390/rs11101244

    Article  Google Scholar 

  21. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  22. Huang, Y., Thomson, S.J., Hoffmann, W.C., Lan, Y., Fritz, B.K.: Development and prospect of unmanned aerial vehicle technologies for agricultural production management. Int. J. Agric. Biol. Eng. 6(3), 1–10 (2013)

    Google Scholar 

  23. Jain, P.M., Shandliya, V.: A review paper on various approaches for image mosaicing. Int. J. Comput. Eng. Res. 3(4), 106–109 (2013)

    Google Scholar 

  24. Johansen, K., Morton, M.J.L., Malbeteau, Y.M., Aragon, B., Al-Mashharawi, S.K., Ziliani, M.G., Angel, Y., Fiene, G.M., Negrão, S.S.C., Mousa, M.A.A., Tester, M.A., McCabe, M.F.: Unmanned Aerial Vehicle-based phenotyping using morphometric and spectral analysis can quantify responses of wild tomato plants to salinity stress. Frontiers Plant Sci. 10, 370 (2019). https://doi.org/10.3389/fpls.2019.00370

    Article  Google Scholar 

  25. Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J.J., Dellaert, F.: iSAM2: incremental smoothing and mapping using the Bayes tree (2011). https://doi.org/10.1177/0278364911430419, https://arxiv.org/abs/1908.02002v1

  26. Kang, L., Wei, Y., Xie, Y., Jiang, J., Guo, Y.: Combining convolutional neural network and photometric refinement for accurate homography estimation. IEEE Access, pp. 109460–109473 (2019)

    Google Scholar 

  27. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  28. Kriegman, D.: Homography estimation. Lecture Computer Vision I, CSE a 252 (2007)

    Google Scholar 

  29. Le Moigne, J., Netanyahu, N.S., Eastman, R.D.: Image Registration for Remote Sensing. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  30. Li, Q., Wang, G., Liu, J., Chen, S.: Robust scale-invariant feature matching for remote sensing image registration. IEEE Geosci. Remote Sens. Lett. 6(2), 287–291 (2009)

    Article  Google Scholar 

  31. Lin, Y., Medioni, G.: Map-enhanced UAV image sequence registration and synchronization of multiple image sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR 2007, pp. 1–7. IEEE (2007)

    Google Scholar 

  32. López-Granados, F., Torres-Sánchez, J., De Castro, A.I., Serrano-Pérez, A., Mesas-Carrascosa, F.J., Peña, J.M.: Object-based early monitoring of a grass weed in a grass crop using high resolution UAV imagery. Agron. Sustain. Dev. 36(4), 67 (2016)

    Article  Google Scholar 

  33. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  34. Molina, E., Zhu, Z.: Persistent aerial video registration and fast multi-view mosaicing. IEEE Trans. Image Process. 23(5), 2184–2192 (2014)

    Article  MathSciNet  Google Scholar 

  35. Morel, J.M., Yu, G.: ASIFT: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009)

    Article  MathSciNet  Google Scholar 

  36. Nasir, A.K., Tharani, M.: Use of Greendrone UAS system for maize crop monitoring. In: ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 263–268 (2017)

    Google Scholar 

  37. Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. IEEE Rob. Autom. Lett. 3(3), 2346–2353 (2018)

    Article  Google Scholar 

  38. Ono, Y., Trulls, E., Fua, P., Yi, K.M.: LF-Net: learning local features from images. arXiv preprint arXiv:1805.09662v2 (2018)

  39. Pohl, C., Van Genderen, J.L.: Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)

    Article  Google Scholar 

  40. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: 2011 International Conference on Computer Vision, pp. 2564–2571. IEEE (2011). https://doi.org/10.1109/ICCV.2011.6126544

  41. Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)

    Google Scholar 

  42. Seetharaman, G., Palaniappan, K., Akbarpour, H.A.: Method for fast camera pose refinement for wide area motion imagery (2019). U.S. Patent 9,959,625

    Google Scholar 

  43. Shi, Y., et al.: Unmanned Aerial Vehicles for high-throughput phenotyping and sgronomic research. PLoS ONE 11(7), e0159781 (2016). https://doi.org/10.1371/journal.pone.0159781

    Article  Google Scholar 

  44. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  45. Teters, E., AliAkbarpour, H., Palaniappan, K., Seetharaman, G.: Real-time geoprojection and stabilization on airborne GPU-enabled embedded systems. In: Geospatial Informatics, Motion Imagery, and Network Analytics VIII, vol. 10645, p. 106450H. International Society for Optics and Photonics (2018)

    Google Scholar 

  46. Wang, X., et al.: Dynamic plant height QTL revealed in maize through remote sensing phenotyping using a high-throughput unmanned aerial vehicle (UAV). Sci. Rep. 9(1) (2019). https://doi.org/10.1038/s41598-019-39448-z

  47. Wirth, M.A.: Shape Analysis & Measurement. University of Guelph (2004). http://www.cyto.purdue.edu/cdroms/micro2/content/education/wirth10.pdf

  48. Woodward Crossings: Jang TD-1 Push Planter. Woodward Crossings (2019)

    Google Scholar 

  49. Wu, C.: Towards linear-time incremental structure from motion. In: 2013 International Conference on 3D Vision, pp. 127–134 (2013)

    Google Scholar 

  50. Zhu, Z., Riseman, E.M., Hanson, A.R., Schultz, H.: An efficient method for geo-referenced video mosaicing for environmental monitoring. Mach. Vis. Appl. 16(4), 203–216 (2005)

    Article  Google Scholar 

  51. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful for the support of the NSF Midwest Big Data Hub Digital Agriculture Spoke, the Missouri Maize Center, the National Corn Growers Association, the U.S. Army Research Laboratory (cooperative agreement W911NF1820285), the Army Research Office (DURIP W911NF1910181), an Executive Women’s Forum doctoral fellowship through the University of Missouri College of Engineering (to R.A.), and an anonymous gift. We thank Shizeng Zhou, Behirah Hartranft, Steven Suddarth, and Vinny for helpful discussions. We are especially grateful to the referees for their very helpful comments. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors’ and do not necessarily reflect the views of the U. S. Government or any agency thereof. The authors declare no conflict of interest.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, USA

    Dewi Endah Kharismawati, Hadi Ali Akbarpour, Rumana Aktar, Filiz Bunyak, Kannappan Palaniappan & Toni Kazic

  2. MU Plant Science Foundry, University of Missouri, Columbia, USA

    Dewi Endah Kharismawati, Hadi Ali Akbarpour, Rumana Aktar, Filiz Bunyak, Kannappan Palaniappan & Toni Kazic

  3. Missouri Maize Center, University of Missouri, Columbia, USA

    Dewi Endah Kharismawati & Toni Kazic

  4. Interdisciplinary Plant Group, University of Missouri, Columbia, USA

    Dewi Endah Kharismawati & Toni Kazic

Authors
  1. Dewi Endah Kharismawati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hadi Ali Akbarpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rumana Aktar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Filiz Bunyak
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kannappan Palaniappan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Toni Kazic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Toni Kazic .

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kharismawati, D.E., Akbarpour, H.A., Aktar, R., Bunyak, F., Palaniappan, K., Kazic, T. (2020). CorNet: Unsupervised Deep Homography Estimation for Agricultural Aerial Imagery. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12540. Springer, Cham. https://doi.org/10.1007/978-3-030-65414-6_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-65414-6_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-65413-9

  • Online ISBN: 978-3-030-65414-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation