Skip to main content

Advertisement

SpringerLink
Log in

Parallax correction via disparity estimation in a multi-aperture camera

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, an image fusion algorithm is proposed for a multi-aperture camera. Such camera is a feasible alternative to traditional Bayer filter camera in terms of image quality, camera size and camera features. The camera consists of several camera units, each having dedicated optics and color filter. The main challenge of a multi-aperture camera arises from the fact that each camera unit has a slightly different viewpoint. Our image fusion algorithm corrects the parallax error between the sub-images using a disparity map, which is estimated from the single-spectral images. We improve the disparity estimation by combining matching costs over multiple views using trifocal tensors. Images are matched using two alternative matching costs, mutual information and Census transform. We also compare two different disparity estimation methods, graph cuts and semi-global matching. The results show that the overall quality of the fused images is near the reference images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Kolehmainen, T., Rytivaara, M., Tokkonen, T., Mäkelä, J., Ojala, K.: Imaging device. US Patent No. 7453510 (2008)

  2. Hirakawa, K.: Cross-talk explained. In: 15th IEEE International Conference on Image Processing, pp. 677–680 (2008)

  3. van Walree, P.: Chromatic aberrations. http://toothwalker.org/optics/chromatic.html (2016). Accessed 5 Apr (2016)

  4. Hernández, C.: Lens blur in the new google camera app (2014), googleresearch.blogspot.com/2014/04/lens-blur-in-new-google-camera-app.html

  5. LinX Imaging. Technology presentation. http://linximaging.com/imaging/ (2014)

  6. Venkataraman, K., Lelescu, D., Duparre, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., Nayar, S.: PiCam: an ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32(6), 13 (2013)

    Article  Google Scholar 

  7. Suda, Y.: Image sensing apparatus and its control method, control program, and storage medium for correcting position deviation of images. US Patent No. 7847843 (2010)

  8. Yu, Y., Zhang, Z.: Digital cameras using multiple sensors with multiple lenses. US Patent No. 6611289 (2003)

  9. Gere, D.S.: Image capture using luminance and chrominance sensors. US Patent No. 8497897 (2013)

  10. Sung, G.-Y., Park, D.-S., Lee, H.-Y., Kim, S.-S., Kim, C.-Y.: Camera module. European Patent No. 1871091 (2007)

  11. Light. https://light.co/camera 2015. Accessed 25 Nov (2015)

  12. Kim, J., Kolmogorov, V., Zabih, R.: Visual correspondence using energy minimization and mutual information. In: The Proceedings of the 9th IEEE International Conference on Computer Vision, 2, pp. 1033–1040 (2003)

  13. Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)

    Article  Google Scholar 

  14. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, United States of America (2003)

    MATH  Google Scholar 

  15. Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1582–1599 (2008)

    Article  Google Scholar 

  16. Egnal, G.: Mutual information as a stereo correspondence measure. University of Pennsylvania, Department of Computer and Information Science, Technical Report No. MS-CIS-00-20 (2000)

  17. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Lecture Notes in Computer Science vol. 801, pp. 151–158 (1994)

  18. Middlebury Stereo Evaluation. http://vision.middlebury.edu/stereo/eval/. Accessed 5 Apr (2016)

  19. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  20. Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  MATH  Google Scholar 

  21. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  MATH  Google Scholar 

  22. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  23. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1, 195–202 (2003)

    Google Scholar 

  24. Kolmogorov, V., Zabih, R., Gortler, S.: Generalized Multi-camera Scene Reconstruction Using Graph Cuts. In: Energy Minimization Methods in Computer Vision and Pattern Recognition. Lecture Notes in Computer Science, vol. 2683, pp. 501–516 (2003)

  25. Levin, A., Lischinski, D., Weiss, Y.: Colorization Using Optimization. ACM SIGGRAPH, 689–694 (2004)

  26. Banz, C., Blume, H., Pirsch, P.: Real-time semi-global matching disparity estimation on the GPU. In IEEE International Conference on Computer Vision Workshops, pp. 514-521 (2011)

Download references

Author information

Authors and Affiliations

  1. The Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, P.O Box 4500, 90014, Oulu, Finland

    Janne Mustaniemi & Janne Heikkilä

  2. Department of Computer Science, Aalto University, P.O. Box 15400, 00076, Aalto, Finland

    Juho Kannala

Authors
  1. Janne Mustaniemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juho Kannala
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Janne Heikkilä
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janne Mustaniemi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mustaniemi, J., Kannala, J. & Heikkilä, J. Parallax correction via disparity estimation in a multi-aperture camera. Machine Vision and Applications 27, 1313–1323 (2016). https://doi.org/10.1007/s00138-016-0773-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-016-0773-7

Keywords

Search

Navigation