Skip to main content
SpringerLink
Log in

Surface color estimation in 3D spatial coordinate remote sensing by a technical vision system

  • Published:
Optical and Quantum Electronics Aims and scope Submit manuscript

Abstract

This paper proposes to enhance the capabilities of a Technical Vision System (TVS) based on optical scanning and dynamic triangulation for 3D spatial coordinates measurements to determine and add to the record of its measurements the color of the objects coordinates scanned. Optoelectronics signals output from the TVS scanning aperture contain a huge amount of data that cannot be friendly or obvious to human understanding and interpretation. However, time domain characteristic parameters can be extracted from the optoelectronics signals, which can describe aspects of the optoelectronic signal and the 3D spatial coordinate to whom it belongs; that is, it is specific and punctual information from the area of the object under scanning. In this theoretical and experimental evaluation, the parameters obtained from optoelectronics signals have been used as feature inputs for supervised learning and as an approach to implement artificial intelligence to the TVS signal processing by a model able to predict in real time the color of the 3D spatial coordinates measured.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Availability of data and materials

Dataset is available in the following link: https://drive.google.com/drive/folders/1ZWXpHs0kANoroZOYaXEI-v6zhXF0fvPb.

Code availability

The Code is not available to share, is just for the use of authors. However, dataset is available for comparison with the results of codes of further research.

References

  • Alslaity, A., Orji, R.: Machine learning techniques for emotion detection and sentiment analysis: current state, challenges, and future directions. Behav. Inf. Technol. 1–26 (2022). https://www.tandfonline.com/doi/abs/10.1080/0144929X.2022.2156387

  • Altın, C., Er, O.: Comparison of different time and frequency domain feature extraction methods on elbow gesture’s EMG. Eur. J. Interdiscipl. Stud. 2(3), 25–34 (2016)

    Article  Google Scholar 

  • Básaca-Preciado, L.C., Sergiyenko, O.Y., Rodríguez-Quinonez, J.C., et al.: Optical 3d laser measurement system for navigation of autonomous mobile robot. Opt. Lasers Eng. 54, 159–169 (2014)

    Article  Google Scholar 

  • Bini, D., Pamela, D., Prince, S.: Machine vision and machine learning for intelligent agrobots: a review. In: 2020 5th International Conference on Devices, Circuits and Systems (ICDCS), pp. 12–16. IEEE (2020)

  • Castro-Toscano, M.J., Rodríguez-Quiñonez, J.C., Sergiyenko, O., et al.: Novel sensing approaches for structural deformation monitoring and 3d measurements. IEEE Sens. J. 21(10), 11318–11328 (2020)

    Article  ADS  Google Scholar 

  • Flores-Fuentes, W., Arellano-Vega, E., Sergiyenko, O., et al.: Conjunto de datos de señales optoelectrónicas. In: Certificado de Registro Público del Derecho de Autor. Rama: Compilación de datos (Base de datos). Número de Registro: 03-2023-020111081800-01 (2023a). https://drive.google.com/drive/folders/1ZWXpHs0kANoroZOYaXEI-v6zhXF0fvPb. Instituto Nacional del Derecho de Autor

  • Flores-Fuentes, W., Arellano-Vega, E., Sergiyenko, O., et al.: Extracción de características en el dominio del tiempo de señales optoelectrónicas de un sistema de visión técnica. In: Certificado de Registro Público del Derecho de Autor. Rama: Programas de computación. Número de Registro: 03-2023-020111075200-01. Instituto Nacional del Derecho de Autor (2023b)

  • Flores-Fuentes, W., Rodríguez-Quiñonez, J.C., Hernandez-Balbuena, D., et al.: Machine vision supported by artificial intelligence. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), pp. 1949–1954. IEEE (2014)

  • Flores-Fuentes, W., Sergiyenko, O., Gonzalez-Navarro, F.F., et al.: Multivariate outlier mining and regression feedback for 3d measurement improvement in opto-mechanical system. Opt. Quant. Electron. 48, 1–21 (2016)

    Article  Google Scholar 

  • Fu, L., Gao, F., Wu, J., et al.: Application of consumer RGB-d cameras for fruit detection and localization in field: a critical review. Comput. Electron. Agric. 177, 105687 (2020). https://www.sciencedirect.com/science/article/abs/pii/S0168169920319530

    Article  Google Scholar 

  • He, Y., Deng, B., Wang, H., et al.: Infrared machine vision and infrared thermography with deep learning: a review. Infrared Phys. Technol. 116, 103754 (2021). https://www.sciencedirect.com/science/article/abs/pii/S1350449521001262

    Article  CAS  Google Scholar 

  • Horaud, R., Hansard, M., Evangelidis, G., et al.: An overview of depth cameras and range scanners based on time-of-flight technologies. Mach. Vis. Appl. 27(7), 1005–1020 (2016)

    Article  Google Scholar 

  • Ivanov, M., Sergiyenko, O., Tyrsa, V., et al.: Data exchange and task of navigation for robotic group. Mach. Vis. Navig. 389–430 (2020a). https://link.springer.com/chapter/10.1007/978-3-030-22587-2_13

  • Ivanov, M., Sergyienko, O., Tyrsa, V., et al.: Influence of data clouds fusion from 3d real-time vision system on robotic group dead reckoning in unknown terrain. IEEE/CAA J. Autom. Sin. 7(2), 368–385 (2020b)

    Article  Google Scholar 

  • Ke, D., Wang, X., Huang, K., et al.: Minimum power adversarial attacks in communication signal modulation classification with deep learning. Cogn. Comput. 15(2), 580–589 (2023)

    Article  Google Scholar 

  • Lénárt, J.: Extending an industrial robot with image processing system. In: Vehicle and Automotive Engineering, pp. 568–574. Springer (2022)

  • Li, J., Huang, W., Zhao, C.: Machine vision technology for detecting the external defects of fruits—a review. Imaging Sci. J. 63(5), 241–251 (2015)

    Article  Google Scholar 

  • Li, M., Xu, T., Wang, S., et al.: Probe pulse design in Brillouin optical time-domain reflectometry. IET Optoelectron. 16(6), 238–252 (2022)

    Article  ADS  Google Scholar 

  • Lin, Y., Wang, Y., Wang, S., et al.: Noise point detection from airborne lidar point cloud based on spatial hierarchical directional relationship. IEEE Access 10, 82076–82091 (2022)

    Article  MathSciNet  Google Scholar 

  • Lindner, L., Sergiyenko, O., Rivas-López, M., et al.: Exact laser beam positioning for measurement of vegetation vitality. Ind. Robot: Int. J. 44(4), 532–541 (2017)

    Article  Google Scholar 

  • Liu, J., Zhang, F., Kudreyko, A., et al.: Novel laser tracking measurement system based on the position sensitive detector. Math. Biosci. Eng.: MBE 20(1), 572–586 (2023)

    Article  PubMed  Google Scholar 

  • Lorenz, S., Salehi, S., Kirsch, M., et al.: Radiometric correction and 3d integration of long-range ground-based hyperspectral imagery for mineral exploration of vertical outcrops. Remote Sens. 10(2), 176 (2018). https://www.mdpi.com/2072-4292/10/2/176

    Article  ADS  Google Scholar 

  • Lu, J., Li, Y., Zuo, Z.: Satmvs: A novel 3d reconstruction pipeline for remote sensing satellite imagery. In: International Conference on Aerospace System Science and Engineering, pp. 521–538. Springer (2021)

  • Ma, C., Xia, W., Chen, F., et al.: A content-based remote sensing image change information retrieval model. ISPRS Int. J. Geo Inf. 6(10), 310 (2017). https://www.mdpi.com/2220-9964/6/10/310

    Article  Google Scholar 

  • Marlow, P.J., Gegenfurtner, K.R., Anderson, B.L.: The role of color in the perception of three-dimensional shape. Curr. Biol. 32(6), 1387–1394 (2022)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Murakami, K., Islam, M., Onodera, H.: CDF distance based statistical parameter extraction using nonlinear delay variation models. In: 2021 IEEE 27th International Symposium on On-Line Testing and Robust System Design (IOLTS), pp. 1–6. IEEE (2021)

  • Palkhiwala, S., Shah, M., Shah, M.: Analysis of machine learning algorithms for predicting a student’s grade. J. Data, Inf. Manag. 4(3–4), 329–341 (2022)

    Article  Google Scholar 

  • Petković, T., Pribanić, T.: Multiprojector multicamera structured light surface scanner. IEEE Access 10, 90321–90337 (2022)

    Article  Google Scholar 

  • Real, O.R., Castro-Toscano, M.J., Rodríguez-Quiñonez, J.C., et al.: Surface measurement techniques in machine vision: operation, applications, and trends. In: Optoelectronics in Machine Vision-based Theories and Applications, pp. 79–104. IGI Global (2019)

  • Rivas, M., Sergiyenko, O., Aguirre, M., et al.: Spatial data acquisition by laser scanning for robot or SHM task. In: 2008 IEEE International Symposium on Industrial Electronics, pp. 1458–1462. IEEE (2008)

  • Rivas, M., Flores, W., Rivera, J., et al.: A method and electronic device to detect the optoelectronic scanning signal energy centre. Optoelectronics-Advanced Materials and Devices, Mexico (2013)

    Book  Google Scholar 

  • Rivas-Lopez, M., Sergiyenko, O., Flores-Fuentes, W., et al.: Optoelectronics in Machine Vision-based Theories and Applications. IGI Global (2018)

  • Rodriguez-Quinonez, J.C., Sergiyenko, O., Gonzalez-Navarro, F.F., et al.: Surface recognition improvement in 3d medical laser scanner using Levenberg–Marquardt method. Signal Process. 93(2), 378–386 (2013)

    Article  Google Scholar 

  • Rodríguez-Quiñonez, J., Sergiyenko, O., Hernandez-Balbuena, D., et al.: Improve 3d laser scanner measurements accuracy using a FFBP neural network with Widrow–Hoff weight/bias learning function. Opto-Electron. Rev. 22, 224–235 (2014)

    Article  ADS  Google Scholar 

  • Rodríguez-Quiñonez, J.C., Sergiyenko, O.Y., Preciado, L.C.B., et al.: Optical monitoring of scoliosis by 3d medical laser scanner. Opt. Lasers Eng. 54, 175–186 (2014)

    Article  Google Scholar 

  • Sabitha, N., Thampi, S.G., Kumar, D.S.: Application of a distributed hydrologic model to assess the impact of climate and land-use change on surface runoff from a small urbanizing watershed. Water Resour. Manage 37(6–7), 2347–2368 (2023)

    Article  Google Scholar 

  • Sergiyenko, O.Y.: Optoelectronic system for mobile robot navigation. Optoelectron., Instrum. Data Process. 46, 414–428 (2010)

    Article  Google Scholar 

  • Sergiyenko, O.Y., Ivanov, M.V., Tyrsa, V., et al.: Data transferring model determination in robotic group. Robot. Auton. Syst. 83, 251–260 (2016)

    Article  Google Scholar 

  • Shiode, N.: 3d urban models: recent developments in the digital modelling of urban environments in three-dimensions. GeoJournal 52, 263–269 (2000)

    Article  Google Scholar 

  • Trujillo-Hernández, G., Rodríguez-Quiñonez, J.C., Flores-Fuentes, W., et al.: Development of an integrated podometry system for mechanical load measurement and visual inspection. Measurement 203, 111866 (2022). https://www.sciencedirect.com/science/article/abs/pii/S0263224122010636

    Article  Google Scholar 

  • Yu, R., Lyu, M., Lu, J., et al.: Spatial coordinates correction based on multi-sensor low-altitude remote sensing image registration for monitoring forest dynamics. IEEE Access 8, 18483–18496 (2020)

    Article  Google Scholar 

  • Zhang, P.Y., Wang, H.Y.: A framework for automatic time-domain characteristic parameters extraction of human pulse signals. Eur. J. Adv. Signal Process. 2008, 1–9 (2007)

    Google Scholar 

  • Zhao, C., Lv, J., Du, S.: Geometrical deviation modeling and monitoring of 3d surface based on multi-output gaussian process. Measurement 199, 111569 (2022). https://www.sciencedirect.com/science/article/abs/pii/S0263224122007849

    Article  Google Scholar 

  • Zheng, Y., Zeng, G., Li, H., et al.: Colorful 3d reconstruction at high resolution using multi-view representation. J. Vis. Commun. Image Represent. 85, 103486 (2022). https://www.sciencedirect.com/science/article/abs/pii/S1047320322000402

    Article  Google Scholar 

  • Zollhöfer, M., Stotko, P., Görlitz, A., et al.: State of the art on 3d reconstruction with RGB-d cameras. In: Computer Graphics Forum, pp. 625–652. Wiley Online Library (2018)

Download references

Funding

Declaration is “not applicable”.

Author information

Author notes

  1. Wendy Flores-Fuentes, Eduardo Arellano-Vega, Oleg Sergiyenko, Iván Y. Alba-Corpus, Julio C. Rodríguez-Quiñonez, Moises J. Castro-Toscano, Félix F. González-Navarro, S. Vasavi, Jesús E. Miranda-Vega, Daniel Hernández-Balbuena, Fabián N. Murrieta-Rico and Moisés Rivas-López are contributed equally to this work.

Authors and Affiliations

  1. Facultad de Ingeniería, Universidad Autónoma de Baja California, Blvd. Benito Juárez, 21280, Mexicali, Baja California, Mexico

    Wendy Flores-Fuentes, Eduardo Arellano-Vega, Iván Y. Alba-Corpus, Julio C. Rodríguez-Quiñonez, Moises J. Castro-Toscano & Daniel Hernández-Balbuena

  2. Instituto de Ingeniería, Universidad Autónoma de Baja California, Calle Normal, 21100, Mexicali, Baja California, Mexico

    Oleg Sergiyenko & Félix F. González-Navarro

  3. Department of Computer Science and Engineering, VR Siddhartha Engineering College, Kanuru, Vijayawada, Andhra Pradesh, 520007, India

    S. Vasavi

  4. Departamento de Sistemas Computacionales, Tecnológico Nacional de México/IT de Mexicali, Instituto Tecnológico, 21376, Mexicali, Baja California, Mexico

    Jesús E. Miranda-Vega

  5. Ingeniería Mecatrónica, Universidad Politécnica de Baja California, 21376, Claridad, Mexicali, Baja California, Mexico

    Fabián N. Murrieta-Rico & Moisés Rivas-López

Authors
  1. Wendy Flores-Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Arellano-Vega
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oleg Sergiyenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Iván Y. Alba-Corpus
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julio C. Rodríguez-Quiñonez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Moises J. Castro-Toscano
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Félix F. González-Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. S. Vasavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jesús E. Miranda-Vega
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Daniel Hernández-Balbuena
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Fabián N. Murrieta-Rico
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Moisés Rivas-López
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors declare all have participated in the paper and agree to submit it for publication. Rivas-López, Sergiyenko, Hernández-Balbuena, Rodríguez-Quiñonez and Flores-Fuentes patented, manufactured and instrumented the technical vision system used for the experimentation. Murrieta-Rico, Miranda-Vega and Alba-Corpus, implemented the control system for scanning frequency measurement and programmed the signal processing procedure for the coordinate calculation to enhance the measurements accuracy. Flores-Fuentes, Sergiyenko, González-Navarro conceptualized the idea of extracting information from optoelectronic signals through data mining. Arellano-Vega, Alba-Corpus and Flores-Fuentes designed and executed the experiments, acquired measurements, created the computer program that process the signals to extract parameters in the time domain and constructed the data set. Arellano-Vega, Flores-Fuentes, Vasavi and González-Navarro implemented the classifiers. Castro-Toscano carefully checked the whole manuscript for possible technical and language corrections. Flores-Fuentes and Arellano-Vega wrote the experimentation methodology followed and results for the paper. All the authors participated in the edition of text, figures, tables, and reviewed the final manuscript.

Corresponding author

Correspondence to Wendy Flores-Fuentes.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

Declaration is “not applicable”.

Consent to participate

Declaration is “the authors consent to participate”.

Consent for publication

Declaration is “the authors consent the publication”.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Flores-Fuentes, W., Arellano-Vega, E., Sergiyenko, O. et al. Surface color estimation in 3D spatial coordinate remote sensing by a technical vision system. Opt Quant Electron 56, 406 (2024). https://doi.org/10.1007/s11082-023-05646-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11082-023-05646-3

Keywords

Search

Navigation