Skip to main content
SpringerLink
Log in

Pose Measurement at Small Scale by Spectral Analysis of Periodic Patterns

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The retrieval of an observed object’s pose is an essential computer vision problem. The challenge arises in many different fields, among them robotics control, contactless metrology, or augmented reality. When the observed object shrinks from the macroscopic scale to the microscopic, pose estimation is further complicated by the weaker perspective of imaging macroscale lenses down to the quasi-orthographic projection inherent to microscope objectives. This paper tackles this issue of microscale pose estimation in two complementary steps that rely on the use of planar periodic targets. We first consider the orthographic projection case as a means of presenting the theory of the method and showing how the pose of periodic patterns can be directly retrieved from the Fourier frequency spectrum of a given image. We then address the perspective case with long focal lengths, in which the full six-degrees of freedom (6-DOF) pose can be retrieved without ambiguities by following the same theoretical background. In addition to theoretically justifying pose retrieval via Fourier analysis of acquired images, this paper demonstrates the method’s actual performance. Both simulations and experimentation are conducted to validate the method and confirm an experimental resolution lower than \(1/1000{\mathrm{th}}\) of a pixel for translations. For orientation measurement, resolutions below 1 \(\upmu \)rad. for in-plane orientation, and below 100 \(\upmu \)rad. for off-axis orientations can be achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://projects.femto-st.fr/vernier/en.

  2. OpenCV PnP solver documentation available here: https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html.

  3. https://sourcesup.renater.fr/www/vernierlibrary/data/.

References

  • Abawi, D. F., Bienwald, J., & Dorner, R. (2004). Accuracy in optical tracking with fiducial markers: an accuracy function for artoolkit. In Third IEEE and ACM international symposium on mixed and augmented reality (pp. 260–261), IEEE.

  • André, A. N., Sandoz, P., Jacquot, M., & Laurent, G. J. (2020). Robust, precise and scalable: A phase-encoded pattern for visual x, y, \(\theta \) positioning. In 2020 international conference on manipulation, automation and robotics at small scales (MARSS) (pp. 1–5). IEEE.

  • Andre, A. N., Sandoz, P., Mauze, B., Jacquot, M., & Laurent, G. J. (2020). Sensing one nanometer over ten centimeters: A micro-encoded target for visual in-plane position measurement. IEEE/ASME Transactions on Mechatronics, 25(3), 1193–1201. https://doi.org/10.1109/TMECH.2020.2965211

    Article  Google Scholar 

  • André, A. N., Sandoz, P., Mauzé, B., Jacquot, M., & Laurent, G. J. (2021). Robust phase-based decoding for absolute (x, y, \(\theta \)) positioning by vision. IEEE Transactions on Instrumentation and Measurement, 70, 1–10. https://doi.org/10.1109/TIM.2020.3009353

    Article  Google Scholar 

  • Azar, E. R., Feng, C., & Kamat, V. R. (2015). Feasibility of in-plane articulation monitoring of excavator arm using planar marker tracking. Journal of Information Technology in Construction (ITcon), 20(15), 213–229.

    Google Scholar 

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In European conference on computer vision (pp. 404–417). Springer.

  • Bomarito, G., Hochhalter, J., Ruggles, T., & Cannon, A. (2017). Increasing accuracy and precision of digital image correlation through pattern optimization. Optics and Lasers in Engineering, 91, 73–85.

    Article  Google Scholar 

  • Bouguet, J. Y. (2004). Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html.

  • Bruckstein, A. M., Holt, R. J., Huang, T. S., & Netravali, A. N. (1999). Optimum fiducials under weak perspective projection. International Journal of Computer Vision, 35(3), 223–244.

    Article  Google Scholar 

  • Bruckstein, A. M., Holt, R. J., Huang, T. S., & Netravali, A. N. (2000). New devices for 3d pose estimation: Mantis eyes, Agam paintings, sundials, and other space fiducials. International Journal of Computer Vision, 39(2), 131–139.

    Article  MATH  Google Scholar 

  • Chen, X., Fan, R., Wu, J., Song, X., Liu, Q., Wang, Y., Wang, Y., & Tao, B. (2020). Fourier-transform-based two-stage camera calibration method with simple periodical pattern. Optics and Lasers in Engineering, 133, 106121.

    Article  Google Scholar 

  • Chen, Z. H., & Huang, P. S. (2016). A vision-based method for planar position measurement. Measurement Science and Technology, 27(12), 125018.

    Article  Google Scholar 

  • Chu, H. K., Mills, J. K., & Cleghorn, W. L. (2012). Dual-arm micromanipulation and handling of objects through visual images. In 2012 IEEE international conference on mechatronics and automation (pp. 813–818). IEEE.

  • Collins, T., & Bartoli, A. (2014). Infinitesimal plane-based pose estimation. International Journal of Computer Vision, 109(3), 252–286.

    Article  MathSciNet  MATH  Google Scholar 

  • Didier, J. Y., Ababsa, F. E., & Mallem, M. (2008). Hybrid camera pose estimation combining square fiducials localization technique and orthogonal iteration algorithm. International Journal of Image and Graphics, 8(01), 169–188.

    Article  Google Scholar 

  • Drummond, T., & Cipolla, R. (2002). Real-time visual tracking of complex structures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 932–946.

    Article  Google Scholar 

  • Fiala, M. (2005). Artag, a fiducial marker system using digital techniques. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) (Vol. 2, pp 590–596). IEEE.

  • Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F. J., & Marín-Jiménez, M. J. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition, 47(6), 2280–2292.

    Article  Google Scholar 

  • Guelpa, V., Laurent, G. J., Sandoz, P., Zea, J. G., & Clévy, C. (2014). Subpixelic measurement of large 1D displacements: Principle, processing algorithms, performances and software. Sensors, 14(3), 5056–5073.

    Article  Google Scholar 

  • Kato, H., & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR’99) (pp. 85–94). IEEE.

  • Kim, J. A., Kim, J. W., Kang, C. S., & Jin, J. (2018). Note: An absolute xy-\(\theta \) position sensor using a two-dimensional phase-encoded binary scale. Review of Scientific Instruments, 89(4), 046105.

    Article  Google Scholar 

  • Kim, Y. S., Yang, S. H., Yang, K. W., & Dagalakis, N. G. (2015). Design of mems vision tracking system based on a micro fiducial marker. Sensors and Actuators A: Physical, 234, 48–56.

    Article  Google Scholar 

  • Li, H., Zhu, B., Chen, Z., & Zhang, X. (2019). Realtime in-plane displacements tracking of the precision positioning stage based on computer micro-vision. Mechanical Systems and Signal Processing, 124, 111–123.

    Article  Google Scholar 

  • Liu, A., Marschner, S., & Snavely, N. (2016). Caliber: Camera localization and calibration using rigidity constraints. International Journal of Computer Vision, 118(1), 1–21.

    Article  MathSciNet  MATH  Google Scholar 

  • Liu, J., Gong, Z., Tang, K., Lu, Z., & Sun, Y. (2013). Locating end-effector tips in automated micromanipulation. In 2013 IEEE international conference on robotics and automation (pp. 1724–1729). IEEE.

  • Loing, V., Marlet, R., & Aubry, M. (2018). Virtual training for a real application: Accurate object-robot relative localization without calibration. International Journal of Computer Vision, 126(9), 1045–1060.

    Article  Google Scholar 

  • Marturi, N., Tamadazte, B., Dembélé, S., & Piat, N. (2016). Image-guided nanopositioning scheme for SEM. IEEE Transactions on Automation Science and Engineering, 15(1), 45–56.

    Article  Google Scholar 

  • Moreels, P., & Perona, P. (2007). Evaluation of features detectors and descriptors based on 3D objects. International Journal of Computer Vision, 73(3), 263–284.

    Article  Google Scholar 

  • Naimark, L., & Foxlin, E. (2002). Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker. In Proceedings, international symposium on mixed and augmented reality (pp. 27–36). IEEE.

  • Pentenrieder, K., Meier, P., Klinker, G. (2006). Analysis of tracking accuracy for single-camera square-marker-based tracking. In Proceedings Dritter Workshop Virtuelle und Erweiterte Realitt der GIFachgruppe VR/AR, Koblenz, Germany, Citeseer.

  • Ri, S., Hayashi, S., Ogihara, S., & Tsuda, H. (2014). Accurate full-field optical displacement measurement technique using a digital camera and repeated patterns. Optics Express, 22(8), 9693–9706.

    Article  Google Scholar 

  • Sandoz, P., Bonnans, V., & Gharbi, T. (2002). High-accuracy position and orientation measurement of extended two-dimensional surfaces by a phase-sensitive vision method. Applied Optics, 41(26), 5503–5511.

    Article  Google Scholar 

  • Sattar, J., Bourque, E., Giguere, P., Dudek, G. (2007). Fourier tags: Smoothly degradable fiducial markers for use in human-robot interaction. In Fourth Canadian conference on computer and robot vision (CRV’07) (pp. 165–174).

  • Shang, W., Lu, H., Wan, W., Fukuda, T., & Shen, Y. (2016). Vision-based nano robotic system for high-throughput non-embedded cell cutting. Scientific Reports, 6(1), 1–14.

    Article  Google Scholar 

  • Sugiura, H., Sakuma, S., Kaneko, M., & Arai, F. (2015). On-chip method to measure mechanical characteristics of a single cell by using moiré fringe. Micromachines, 6(6), 660–673.

    Article  Google Scholar 

  • Tamadazte, B., Marchand, E., Dembélé, S., & Le Fort-Piat, N. (2010). Cad model-based tracking and 3D visual-based control for mems microassembly. The International Journal of Robotics Research, 29(11), 1416–1434.

    Article  Google Scholar 

  • Yamahata, C., Sarajlic, E., Krijnen, G. J., & Gijs, M. A. (2010). Subnanometer translation of microelectromechanical systems measured by discrete Fourier analysis of CCD images. Journal of Microelectromechanical Systems, 19(5), 1273–1275.

    Article  Google Scholar 

  • Yao, S., Li, H., Pang, S., Zhu, B., Zhang, X., & Fatikow, S. (2021). A review of computer microvision-based precision motion measurement: Principles, characteristics, and applications. IEEE Transactions on Instrumentation and Measurement, 70, 1–28. https://doi.org/10.1109/TIM.2021.3065436.

    Article  Google Scholar 

  • Zhong, L., & Zhang, L. (2019). A robust monocular 3D object tracking method combining statistical and photometric constraints. International Journal of Computer Vision, 127(8), 973–992.

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by Région Bourgogne Franche-Comté, by the ANR project Holo-Control (ANR-21-CE42-0009), by the I-SITE BFC project HoloNet (ANR-15-IDEX-03), by Cross-disciplinary Research (EIPHI) Graduate School (ANR-17-EURE-0002). The encoded target was realized thanks to the RENATECH technological network and its FEMTO-ST facility MIMENTO. The experiments was conducted within the ROBOTEX robotics network (ANR-10-EQPX-44-01) and its FEMTO-ST micro-nano-robotics center. Authors acknowledge G. Jutzi, L. Robert, M. Suarez and L. Gauthier-Manuel for technological and experimental assistance.

Author information

Authors and Affiliations

  1. FEMTO-ST Institute, 15B Avenue des Montboucons, 25000, Besançon, France

    A. N. André, P. Sandoz, M. Jacquot & G. J. Laurent

Authors
  1. A. N. André
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Sandoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Jacquot
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. J. Laurent
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. N. André.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by Adrien Bartoli.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors are with the FEMTO-ST Institute, Univ. Bourgogne Franche-Comté, UMR CNRS 6174, 25000 Besançon, France.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 3620 KB)

Supplementary file 2 (mp4 5697 KB)

Supplementary file 3 (mp4 8626 KB)

Supplementary file 4 (mp4 3845 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

André, A.N., Sandoz, P., Jacquot, M. et al. Pose Measurement at Small Scale by Spectral Analysis of Periodic Patterns. Int J Comput Vis 130, 1566–1582 (2022). https://doi.org/10.1007/s11263-022-01607-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01607-7

Keywords

Search

Navigation