Skip to main content
SpringerLink
Log in

Fixed Pattern Noise Analysis for Feature Descriptors in CMOS APS Images

  • Original Paper
  • Published:
Sensing and Imaging Aims and scope Submit manuscript

Abstract

This paper provides a comparative performance evaluation of local features for images from CMOS APS sensors affected by fixed pattern noise for different combinations of common detectors and descriptors. Although numerous studies report comparisons of local features designed for ordinary visual images, their performance on images with fixed pattern noise is far less assessed. The goal of this work is to develop a tool that allows to evaluate the performance of computer vision algorithms and their implementations subject to deviations of the physical parameters of the CMOS sensor. This tool will facilitate the quantification of the high-level effects produced by circuit random noise, enabling the optimization of the sensor during the design flow with specifications much closer to the application scope. Likewise, this tool will provide the electronic designer with a relationship between high-level algorithm accuracy and maximum fixed pattern noise. Thus the contribution is double: (1) to evaluate the performance of both local float type and more recent binary type detectors and descriptors when combined under a variety of image transformations, and (2) to extract relevant information from circuit-level simulation and to develop a basic noise model to be employed in the design of the feature descriptor evaluation. The utility of this approach is illustrated by the evaluation of the effect of column-wise and pixel-wise fixed pattern noise at the sensor on the performance of different local feature descriptors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://www.embedded-vision.com/technology/cameras-sensors.

References

  1. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In CVPR (pp. 510–517). IEEE Computer Society (2012). http://dblp.uni-trier.de/db/conf/cvpr/cvpr2012.html#AlahiOV12.

  2. Alcantarilla, P. F., Bartoli, A., & Davison, A. J. (2012). Kaze features. In Proceedings of the 12th European conference on computer vision, ECCV’12 (Vol. Part VI, pp. 214–227). Springer, Berlin. https://doi.org/10.1007/978-3-642-33783-3_16.

  3. Alcantarilla, P. F., & Solutions, T. (2011). Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1281–1298.

    Google Scholar 

  4. Bay, H., Ess, A., Tuytelaars, T., Gool, L. V., & Leuven, B. K. U. (2008). Speeded-up robust features (surf). Computer Vision and Image Understanding, 110(3), 346–359.

    Article  Google Scholar 

  5. Bay, H., Tuytelaars, T., & Gool, L. V. (2006). Surf: Speeded up robust features. In ECCV (pp. 404–417).

  6. Brown, M., & Lowe, D. (2002). Invariant features from interest point groups. In British machine vision conference (pp. 656–665).

  7. Calonder, M., Lepetit, V., Ozuysal, M., Trzcinski, T., Strecha, C., & Fua, P. (2012). Brief: computing a local binary descriptor very fast. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1281–1298. https://doi.org/10.1109/TPAMI.2011.222.

    Article  Google Scholar 

  8. Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). Brief: binary robust independent elementary features. In K. Daniilidis, P. Maragos, & N. Paragios (Eds.), Computer vision—ECCV 2010 (pp. 778–792). Berlin: Springer.

    Chapter  Google Scholar 

  9. Clark, L. T., Beiley, M. A., & Hoffman, E. J. (2000). Sensor cell having a soft saturation circuit. https://lens.org/022-956-747-444-748.

  10. Dickinson, A. G., Inglis, D. A., & Eid El-sayed, I. (1996). Active pixel sensor and imaging system having differential mode. https://lens.org/099-895-948-098-172.

  11. Official website of the EMVA1288 standard. Retrieved June 30, 2010, from http://www.standard1288.org/ (2010).

  12. Fernández-Berni, J., Carmona-Galán, R., Río, R., & Rodríguez-Vázquez, A. (2014). Bottom-up performance analysis of focal-plane mixed-signal hardware for viola-jones early vision tasks. International Journal of Circuit Theory and Applications, 43(8), 1063–1079. https://doi.org/10.1002/cta.1996.

    Article  Google Scholar 

  13. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  14. Fossum, E. R. (1993). Active pixel sensors: Are CCDs dinosaurs? https://doi.org/10.1117/12.148585.

  15. Fossum, E. R. (1995). CMOS image sensors: Electronic camera on a chip. In Proceedings of international electron devices meeting (pp. 17–25). https://doi.org/10.1109/IEDM.1995.497174.

  16. Fua, P., Tola, E., & Lepetit, V. (2008). A fast local descriptor for dense matching. In 2008 IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 00, pp. 1–8). https://doi.org/10.1109/CVPR.2008.4587673.

  17. Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Proceedings of fourth alvey vision conference (pp. 147–151).

  18. Jähne, B. (2010). EMVA 1288 standard for machine vision: Objective specification of vital camera data. Optik & Photonik, 5, 53–54. https://doi.org/10.1002/opph.201190082.

    Article  Google Scholar 

  19. Leutenegger, S., Chli, M., & Siegwart, R. (2011). Brisk: binary robust invariant scalable keypoints. In D. N. Metaxas, L. Quan, A. Sanfeliu, & L. J. V. Gool (Eds.), ICCV (pp. 2548–2555). IEEE Computer Society. http://dblp.uni-trier.de/db/conf/iccv/iccv2011.html#LeuteneggerCS11.

  20. Leutenegger, S., Chli, M., & Siegwart, Y. (2011). Brisk: binary robust invariant scalable keypoints. In 2011 IEEE international conference on computer vision (ICCV) (pp. 2548–2555).

  21. Levi, G., & Hassner, T. (2015). LATCH: learned arrangements of three patch codes. CoRRarXiv.org/abs/1501.03719.

  22. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 91–110.

    Article  Google Scholar 

  23. Matsumoto, K., Nakamura, T., Yusa, A., & Nagai, S. (1985). A new MOS phototransistor operating in a non-destructive readout mode. Japanese Journal of Applied Physics, 24(5A), L323. http://stacks.iop.org/1347-4065/24/i=5A/a=L323.

  24. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., et al. (2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1–2), 43–72.

    Article  Google Scholar 

  25. Nakamura, J. (2005). Image sensors and signal processing for digital still cameras. Boca Raton, FL: CRC Press Inc.

    Google Scholar 

  26. Norouzi, M., Punjani, A., & Fleet, D. J. (2012). Fast search in hamming space with multi-index hashing. In 2012 IEEE conference on computer vision and pattern recognition (pp. 3108–3115). https://doi.org/10.1109/CVPR.2012.6248043.

  27. Ohta, J. (2007). Smart CMOS image sensors and applications. Boca Raton: CRC Press.

    Google Scholar 

  28. Pelgrom, M. J. M., Duinmaijer, A. C. J., & Welbers, A. P. G. (1989). Matching properties of mos transistors. IEEE Journal of Solid-State Circuits, 24(5), 1433–1439. https://doi.org/10.1109/JSSC.1989.572629.

    Article  Google Scholar 

  29. Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011). ORB: an efficient alternative to SIFT or SURF. In 2011 IEEE international conference on computer vision (ICCV) (pp. 2564–2571), IEEE.

  30. Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011). Orb: an efficient alternative to sift or surf. In Proceedings of the 2011 international conference on computer vision, ICCV ’11 (pp. 2564–2571). IEEE Computer Society, Washington, DC, USA. https://doi.org/10.1109/ICCV.2011.6126544.

  31. Simonyan, K., Vedaldi, A., & Zisserman, A. (2012). Descriptor learning using convex optimisation. In A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, & C. Schmid (Eds.), Computer vision—ECCV 2012 (pp. 243–256). Berlin: Springer.

    Chapter  Google Scholar 

  32. Tian, H., Fowler, B., & Gamal, A. E. (2001). Analysis of temporal noise in CMOS photodiode active pixel sensor. IEEE Journal of Solid-State Circuits, 36(1), 92–101.

    Article  Google Scholar 

  33. Villegas-Pachón, C., Carmona-Galán, R., Fernández-Berni, J., & Rodríguez-Vázquez, Á. (2016). Hardware-aware performance evaluation for the co-design of image sensors and vision algorithms. In 2016 13th International conference on synthesis, modeling, analysis and simulation methods and applications to circuit design (SMACD) (pp. 1–4). https://doi.org/10.1109/SMACD.2016.7520722.

  34. Zimmermann, H. (2000). Integrated silicon optoelectronics. Berlin: Springer. https://books.google.es/books?id=maRCv9SQfowC.

Download references

Acknowledgements

The author wants to acknowledge the work of the rest of the participants in this project, namely: J.A. López-Alcantud, P. Rubio-Ibáñez, Universidad Politécnica de Cartagena, J. A. Díaz-Madrid, Centro Universitario de la Defensa-UPCT and T. J. Kazmierski, University of Southampton.

Funding

This work was mainly supported by Spanish Government MICINN and the European Region Development Fund (ERDF/FEDER) through project RTI2018-097088-B-C33, and project RTI2018-097088-B-C31, by European Union H2020 MSCA through project ACHIEVE-ITN (Grant No. 765866) and by the US Office of Naval Research through Grant No. N00014-19-1-2156.

Author information

Authors and Affiliations

  1. Dpto. de Electrónica y Tecnología de Computadoras, Universidad Politécnica Cartagena, Cartagena, Spain

    Juan Zapata-Pérez, Ginés Doménech-Asensi, Ramón Ruiz-Merino & Jose Javier Martínez-Álvarez

  2. Instituto de Microelectrónica de Sevilla, IMSE-CNM, CSIC-Universidad de Sevilla, Seville, Spain

    Jorge Fernández-Berni & Ricardo Carmona-Galán

Authors
  1. Juan Zapata-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ginés Doménech-Asensi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramón Ruiz-Merino
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose Javier Martínez-Álvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jorge Fernández-Berni
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ricardo Carmona-Galán
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Zapata-Pérez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zapata-Pérez, J., Doménech-Asensi, G., Ruiz-Merino, R. et al. Fixed Pattern Noise Analysis for Feature Descriptors in CMOS APS Images. Sens Imaging 21, 14 (2020). https://doi.org/10.1007/s11220-020-0278-3

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-020-0278-3

Keywords

Search

Navigation