Skip to main content
SpringerLink
Log in

Parallel and distributed processing for high resolution agricultural tomography based on big data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the field of high-resolution tomography, there is currently a notable increase in the volume of tomographic projections and data produced. Such a context has been demanding new computational approaches to the process of reconstruction and processing of the resulting digital images. This paper presents a new approach to meet such a demand, such as optimizing the set of tomographic projections for the reconstruction process, parallelizing algorithm reconstruction, and processing the data in a distributed manner. In this context, a customized method for the high-resolution tomographic reconstruction of agricultural samples has been validated. Hence, tomographic projections with greater amounts of information based on measurements of the spectral density of the projections can be prioritized, and the reconstructive process parallelization using the known filtered back-projection can be considered (i.e., distributed data flow and the use of the Apache Spark environment). For the operation, such an approach based on the big data environment has been organized, that is considering a cluster installed on the Amazon Web Services platform, whose configuration has been defined after the evaluation of the speedup and efficiency metrics. The developed method proved to be useful for carrying out high-resolution tomography analyses of large quantities of agricultural samples, based on the paradigms of precision agriculture for gains in sustainability and competitiveness of the production process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Data availability statement

The data presented in this study are available on request from the corresponding author.

Notes

  1. https://github.com/Yelp/mrjob

  2. https://spark.apache.org/docs/latest/api/python/index.html

  3. https://numpy.org/

  4. The diameters of the holes were: 3.00, 4.00, 4.60, 7.40, 8.60, 11.13, 13.70, 24.00 and 34.55mm

  5. Disponível em: https://github.com/InsightSoftwareConsortium/itkwidgets

  6. Available at: https://jupyter.org/

References

  1. Ahmed MR, Yasmin J, Wakholi C, Mukasa P, Cho B-K (2020) Classification of pepper seed quality based on internal structure using x-ray CT imaging 179:105839. https://doi.org/10.1016/j.compag.2020.105839

  2. Ahmed MR, Yasmin J, Collins W, Cho B-K (2018) X-ray ct image analysis for morphology of muskmelon seed in relation to germination. Biosyst Eng 175:183–193. https://doi.org/10.1016/j.biosystemseng.2018.09.015

    Article  Google Scholar 

  3. Alves GM, Cruvinel PE (2018) Big data infrastructure for agricultural tomographic images reconstruction. 2018 IEEE 12th International Conference on Semantic Computing (ICSC). https://doi.org/10.1109/icsc.2018.00071

  4. Balogun F, Cruvinel P (2003) Compton scattering tomography in soil compaction study. Nucl Inst Methods Phys Res A: Accelerators, Spectrometers, Detectors and Associated Equipment 505(1–2):502–507. https://doi.org/10.1016/s0168-9002(03)01133-1

    Article  Google Scholar 

  5. Beutler FJ, Leneman OA (1966) Random sampling of random processes: Stationary point processes. Inf Control 9(4):325–346. https://doi.org/10.1016/s0019-9958(66)80001-3

    Article  MathSciNet  Google Scholar 

  6. Bronson K, Knezevic I (2016) Big data in food and agriculture. Big Data and Society 3(1). https://doi.org/10.1177/2053951716648174

  7. Cruvinel P, Cesareo R, Crestana S, Mascarenhas S (1990) X- and gamma-rays computerized minitomograph scanner for soil science. IEEE Trans Instrum Meas 39(5):745–750. https://doi.org/10.1109/19.58619

    Article  Google Scholar 

  8. Cruvinel P, Pereira M, Saito J, Costah LDF (2009) Performance improvement of tomographic image reconstruction based on DSP processors. IEEE Trans. Instrum. Meas. 58(9):3295–3304. https://doi.org/10.1109/tim.2009.2022378

    Article  Google Scholar 

  9. Ding C, Wang W, He H, Yang W (2020) Research on tomographic image reconstruction algorithms based on fixed-point rotation x-CT system 79(35–36):25463–25496. https://doi.org/10.1007/s11042-020-08861-2

  10. Diniz PSR, Silva EAB, Netto SL (2010) Digital Signal Processing. Cambridge University Press

    Google Scholar 

  11. Ditter A, Fey D, Schon T, Oeckl S (2014) On the way to big data applications in industrial computed tomography 792–793. https://doi.org/10.1109/bigdata.congress.2014.125

  12. Hajjaji Y, Boulila W, Farah IR, Romdhani I, Hussain A (2021) Big data and IoT-based applications in smart environments: A systematic review 39:100318. https://doi.org/10.1016/j.cosrev.2020.100318

    Article  Google Scholar 

  13. Heeraman D, Hopmans J, Clausnitzer V (1997) Three dimensional imaging of plant roots in situ with x-ray computed tomography. Plant Soil 189(2):167–179. https://doi.org/10.1023/b:plso.0000009694.64377.6f

    Article  Google Scholar 

  14. Hsieh J (2009) Computed Tomography: Principles, Design, Artifacts, and Recent Advances. John Wiley & Sons Inc

    Google Scholar 

  15. Janßen R (1987) A note on superlinear speedup 4(2):211–213. https://doi.org/10.1016/0167-8191(87)90053-6

    Article  Google Scholar 

  16. Kak AC, Slaney M (1989) Principles of Computerized Tomographic Imaging. IEEE Press

    Google Scholar 

  17. Kamilaris A, Kartakoullis A, Prenafeta-Boldu FX (2017) A review on the practice of big data analysis in agriculture. Comput Electron Agric 143:23–37. https://doi.org/10.1016/j.compag.2017.09.037

    Article  Google Scholar 

  18. Kontoghiorghes EJ (2005) Handbook of Parallel Computing and Statistics. Chapman and Hall/CRC

  19. Liakos K, Busato P, Moshou D, Pearson S, Bochtis D (2018) Machine learning in agriculture: A review. Sensors 18(8):2674. https://doi.org/10.3390/s18082674

    Article  Google Scholar 

  20. Naime JM (1994) Projeto e construção de um Tomógrafo portátil para estudos de ciência do solo e plantas, em campo. Master’s thesis, USP

  21. Oppenheim AV, Schafer RW (1975) Digital Signal Processing. Prentice Hall

    Google Scholar 

  22. Pereira M, Cruvinel P (2015) A model for soil computed tomography based on volumetric reconstruction, wiener filtering and parallel processing. Comput Electron Agric 111:151–163. https://doi.org/10.1016/j.compag.2014.12.006

    Article  Google Scholar 

  23. Pires LF, Bacchi OOS (2010) Mudanças na estrutura do solo avaliada com uso de tomografia computadorizada. Pesq Agrop Brasileira 45(4):391–400. https://doi.org/10.1590/s0100-204x2010000400007

    Article  Google Scholar 

  24. Pires LF, Borges JA, Bacchi OO, Reichardt K (2010) Twenty-five years of computed tomography in soil physics: A literature review of thebrazilian contribution. Soil Tillage Res 110(2):197–210. https://doi.org/10.1016/j.still.2010.07.013

    Article  Google Scholar 

  25. Rangayyani RM (2004) Biomedical Image Analysis (Biomedical Engineering). CRC Press

    Book  Google Scholar 

  26. Ribarics P (2016) Big data and its impact on agriculture. Ecocycles 2(1):33–34. https://doi.org/10.19040/ecocycles.v2i1.54

    Article  Google Scholar 

  27. Scannavino FA (2013) Tomógrafo de espalhamento Compton para estudos da física de solos agrícolas em ambiente de campo. Ph.D. thesis, USP

  28. Serrano E, Garcia-Blas J, Carretero J, Desco M, Abella M (2020) Accelerated iterative image reconstruction for cone-beam computed tomography through big data frameworks 106:534–544. https://doi.org/10.1016/j.future.2019.12.042

    Article  Google Scholar 

  29. Shannon CE (1948) A mathematical theory of communication. Bell System Technical Journal 27(3):379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

    Article  MathSciNet  Google Scholar 

  30. Shannon C (1949) Communication in the presence of noise. Proc IRE 37(1):10–21. https://doi.org/10.1109/jrproc.1949.232969

    Article  MathSciNet  Google Scholar 

  31. Silva AM (1997) Construção e uso de um tomógrafo com resolução micrométrica para aplicações em ciências do solo e ambi. Ph.D. thesis,USP

  32. Ullah R, Arslan T (2020) PySpark-based optimization of microwave image reconstruction algorithm for head imaging big data on high-performance computing and google cloud platform 10(10):3382. https://doi.org/10.3390/app10103382

    Article  Google Scholar 

  33. Verdu S (1998) Fifty years of shannon theory. IEEE Trans Inf Theory 44(6):2057–2078. https://doi.org/10.1109/18.720531

    Article  MathSciNet  Google Scholar 

  34. Wang G (2016) A perspective on deep imaging. IEEE Access 4:8914–8924. https://doi.org/10.1109/access.2016.2624938

    Article  Google Scholar 

  35. Zhang H et al (2016) Image Prediction for Limited-angle Tomography via Deep Learning with Convolutional Neural Network. ArXiv e-prints. arXiv:1607.08707 [physics.med-ph]

  36. Zhao J, Fu Y, Tan Y, Cao F (2013) A reduction algorithm for the big data in 3D surface reconstruction. https://doi.org/10.1109/smc.2013.824

Download references

Acknowledgements

The authors acknowledge the support from Embrapa Instrumentation (CNPDIA), Federal Institute of São Paulo (IFSP), the Coordination for the Improvement of Higher Education Personnel (CAPES) and the Post-Graduation Program in Computer Science of the Federal University of São Carlos (UFSCar).

Funding

This work was partially supported by Embrapa Instrumentation (CNPDIA) in partnership with the São Paulo Research Foundation (Fapesp), and Federal Institute of São Paulo (IFSP), with grants MPI No. 547 01.14.09.0.01.05.05, Process No. 17/19350-2, and Process No. 23311.000102 /2015-10 respectively.

Author information

Authors and Affiliations

  1. Post-Graduation Program in Computer Science, Federal University of São Carlos (UFSCar), Rod. Washington Luís, Km 235, 13565-905, São Carlos, São Paulo, Brazil

    Gabriel M. Alves & Paulo E. Cruvinel

  2. Embrapa Instrumentation, 15 de Novembro, 1452, 13560-970, São Carlos, São Paulo, Brazil

    Gabriel M. Alves & Paulo E. Cruvinel

  3. Federal Institute of Education, Science and Technology, Av. Marginal, 585, 13871-298, São João da Boa Vista, São Paulo, Brazil

    Gabriel M. Alves

Authors
  1. Gabriel M. Alves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paulo E. Cruvinel
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This work was conducted in collaboration with both authors. Conceptualization, G. M. A., and P.E.C.; methodology, computational model, software and validation; also writing-original draft preparation.

Corresponding author

Correspondence to Gabriel M. Alves.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Paulo E. Cruvinel contributed equally to this work.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alves, G.M., Cruvinel, P.E. Parallel and distributed processing for high resolution agricultural tomography based on big data. Multimed Tools Appl 83, 10115–10146 (2024). https://doi.org/10.1007/s11042-023-15686-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15686-2

Keywords

Search

Navigation