Skip to main content
SpringerLink
Log in

An efficient image-mosaicing method based on multifeature matching

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Mosaicing is connecting two or more images and making a new wide area image with no visible seam-lines. Several algorithms have been proposed to construct mosaics from image sequence where the camera motion is more or less complex. Most of these methods are based either on the interest points matching or on theoretical corner models. This paper describes a fully automated image-mosaicing method based on the regions and the Harris points primitives. Indeed, in order to limit the search window of potential homologous points, for each point of interest, regions segmentation and matching steps are being performed. This enables us to improve the reliability and the robustness of the Harris points matching process by estimating the camera motion. The main originality of the proposed system resides in the preliminary manipulation of regions matching, thus making it possible to estimate the rotation, the translation and the scale factor between two successive images of the input sequence. This estimation allows an initial alignment of the images along with the framing of the interest points search window, and therefore reducing considerably the complexity of the interest points matching algorithm. Then, the resolution of a minimization problem, altogether considering the couples of matched-points, permits us to perform the homography. In order to improve the mosaic continuity around junctions, radiometric corrections are applied. The validity of the herewith described method is illustrated by being tested on several sequences of complex and challenging images captured from real-world indoor and outdoor scenes. These simulations proved the validity of the proposed method against camera motions, illumination variations, acquirement conditions, moving objects and image noise. To determine the importance of the regions matching stage in motion estimation, as well as for the framing of the search window associated to a point of interest, we compared the matching points results of this described method with those produced using the zero-mean normalized cross correlation score (without regions matching). We made this comparison in the case of a simple motion (without the presence of a rotation around optical axis and/or a scale factor), in the case of a rotation and in the general case of an homothety. For justifying the effectiveness of this method, we proposed an objective assessment by defining a reconstruction error.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Barhoumi, W., Zagrouba, E.: Segmentation en régions guidée par l’appariement pour un couple stéréoscopique non calibré. In: Proceedings of the 3eme Workshop Traitements et Analyse d’Information: Méthodes et Applications, Hammamet, Tunisia, pp. 287–292 (2003)

  2. Baumberg, A.: Reliable feature matching across widely separated views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 774–781 (2000)

  3. Bhosle, U., Chaudhuri, S., DuttaRoy, S.: Fast method for image mosaicing using geometric hashing. IETE J. Res. Spec. Issue Visual Media Process. 48(4), 317–324 (2002)

    Google Scholar 

  4. Brown, M., Lowe, D.: Invariant features from interest point groups. In: Proceedings of the 13th British Machine Vision Conference, Cardiff, pp. 253–262 (2002)

  5. Brown, M., Lowe, D.G.: Recognising panoramas. In: Proceedings of the IEEE International Conference on Computer Vision, Nice, pp. 1218–1225 (2003)

  6. Brown, M.: Multi-image matching using invariant features. PhD thesis, The University of British Columbia (2005)

  7. Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)

    Article  Google Scholar 

  8. Can, A., Stewart, C.V., Roysam, B.: Robust hierarchical algorithm for constructing mosaic from images of the curved human retina. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado, pp. 286–292 (1999)

  9. Choe, T.E., Cohen, I., Lee, M., Medioni, G.: Optimal global mosaic generation from retinal images. In: Proceedings of 18th International Conference on Pattern Recognition, Hong Kong, pp. 681–684 (2006)

  10. Colombo, C., DelBimbo, A., Pernici, F.: Image mosaicing from uncalibrated views of a surface of revolution. In: Proceedings of British Machine Vision Conference, London (2004)

  11. Davis, J.: Mosaics of scenes with moving objects. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara (1998)

  12. Dellaert, F., Thrun, S., Thorpe, C.: Mosaicing a large number of widely dispersed, noisy, and distorted images: a bayesian approach. Technical Report, CMU-RI-TR-9934, Robotics Institute, Carnegie Mellon University, USA (1999)

  13. Deriche, R., Blaszka, T.: Recovering and characterizing image features using an efficient model based approach. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, New York, pp. 530–535 (1993)

  14. Deriche, R.: Recursively implementing the Gaussian and its derivatives. Technical Report, RR 2422 INRIA, France (1993)

  15. Douze, M., Charvillat, V., Thiesse, B.: Mosaïques d’images par approximations successives. In: Proceedings of Journées Francophones des Jeunes Chercheurs en Analyse d’Images et Perception Visuelle (ORASIS’01), Cahors, pp 97–102 (2001)

  16. Duplaquet, M.L.: Building large image mosaics with invisible seam-lines. In: Proceedings of the SPIE Congres AeroSence, Orlando, pp. 369–377 (1998)

  17. Feldman, D., Zomet, A.: Generating mosaics with minimum distortions. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 163–170 (2004)

  18. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  19. Fraundorfer, F., Bischof, H.: Evaluation of local detectors on non-planar scenes. In: Proceedings of the 28th Workshop of the Austrian Association for Pattern Recognition, Hagenberg, pp. 125–132 (2004)

  20. Fusiello, A., Aprile, M., Marzotto, R., Murino, V.: Mosaic of a video shot with multiple moving objects. In: Proceedings of the International Conference on Image Processing, Barcelona, pp. 307–310 (2003)

  21. Gong, Y., Proietti, G., LaRose., D.: A robust image mosaicing technique capable of creating integrated panoramas. In: Proceedings of the IEEE International Conference on Information Visualization, London, pp. 24–29 (1999)

  22. Gracias, X., Negahdaripour, S.: Underwater mosaic creation using video sequences from different altitudes. In: Proceedings of OCEANS’05 MTS/IEEE Conference, Boston, pp. 1295–1300 (2005)

  23. Harris, C., Stephens, A.: Combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, Manchester, pp. 147–151 (1988)

  24. Heikkila, M., Pietikainen, M.: An image mosaicing module for wide-area surveillance. In: Proceedings of the Third ACM International Workshop on Video Surveillance & Sensor Networks, Singapore, pp. 11–18 (2005)

  25. Huang, X., Sun, Y., Metaxas, D., Sauer, F., Xu, C.: Hybrid image registration based on configural matching of scale-invariant salient region features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 167–177 (2004)

  26. Irani, M., Hsu, S., Anandan, P.: Video compression using mosaic representations. Elsevier-Sig. Process. Image Commun. 7, 529–552 (1995)

    Article  Google Scholar 

  27. Jethwa, M., Zisserman, A., Fitzgibbon, A.W.: Real-Time panoramic mosaics and augmented reality. In: Proceedings of the Britsh Machine Vision Conference, London, pp. 852–862 (1998)

  28. Jianchao, Y., Chern, C.T.: The practice of automatic satellite image registration. Asian J. Geoinf. 3(4), 11–18 (2003)

    Google Scholar 

  29. Kanade, T., Rander, P.W., Narayaman, P.J.: Virtualized reality: constructing virtual worlds from real scenes. IEEE Trans. Multimedia 4(1), 34–47 (1997)

    Article  Google Scholar 

  30. Richmond, K., Rock, S.M.: An operational real-time large-scale visual mosaicking and navigation system. In: Proceedings of OCEANS’06 MTS/IEEE Conference, Boston, pp. 1–6 (2006)

  31. Krotkov, E., Hebert, M., Simmons, R.: Stereo perception and dead reckoning for a prototype lunar rover. Auton. Robots 2(4), 313–331 (1995)

    Article  Google Scholar 

  32. Lhuillier, M., Quan, L.: Edge-constrained joint view triangulation for image interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 218–224 (2000)

  33. Lowe, D.: Object recognition from local scale- invariant features. In: Proceedings of the International Conference on Computer Vision, Corfu, pp. 1150–1157 (1999)

  34. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  35. Mallick, S.P.: Feature based image mosaicing. Technical Report, CSE 252C University of California, San Diego (2002)

  36. Mann, S., Picard, R.: Virtual bellows: Constructing high quality stills from video. In: First IEEE International Conference on Image Processing, Austin, pp. 363–367 (1994)

  37. Marzotto, R., Fusiello, A., Murino, V.: High resolution video mosaicing with global alignment. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, pp. 692–698 (2004)

  38. Matas, J., Burianek, J., Kittler, J.: Object recognition using the invariant pixel-set signature. In: Proceedings of the British Machine Vision Conference, London, pp. 606–615 (2000)

  39. Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference, Cardiff, pp. 384–393 (2002)

  40. McLauchlan, P.F., Jaenicke, A.: Image mosaicing using sequential bundle adjustment. In: Proceedings of the British Machine Vision Conference, Bristol, pp. 616–625 (2000)

  41. McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: Proceedings of the 22nd Annual Conference on Computer Graphics (SIGGRAPH’95), Los Angeles, pp. 39–46 (1995)

  42. Mehnert, A., Jackway, P.: An improved seeded region growing algorithm. Pattern Recogn. Lett. 18, 1065–1071 (1997)

    Article  Google Scholar 

  43. Merino, L., Wiklund, J., Caballero, F., Moe, A., De Dios, J.R.M., Forssen, P.E., Nordberg, K., Ollero, A.: Vision-based multi-UAV position estimation: Localization based on blob features for exploration missions. IEEE Robot. Autom. Magaz. 13(3), 53–62 (2006)

    Article  Google Scholar 

  44. Meunier, L., Borgmann, M.: High-resolution panoramas using image mosaicing. Final Project, EE368 (Digital Image Processing), Stanford University, USA (2000)

  45. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: Proceedings of the Interational Conference on Computer Vision and Pattern Recognition (2003)

  46. Mikolajczyk, K., Schmid, C.: Comparaison of affine-invariant local detectors and descriptors. In: Proceedings of the 12th European Signal Processing Conference, Vienna (2004)

  47. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparaison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2006)

    Google Scholar 

  48. NagaoM. Matsuyama, T.: Edge preserving smoothing. Comput. Graph. Image Process. 9, 394–407 (1979)

    Article  Google Scholar 

  49. Nicolas, H.: New methods for dynamic mosaicing. IEEE Trans. Image Process. 10(8), 1239–1251 (2001)

    Article  MATH  Google Scholar 

  50. Peleg, S., Rousso, B., Rav-Acha, A., Zomet, A.: Mosaicing on adaptive manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1144–1154 (2000)

    Article  Google Scholar 

  51. Peleg, S., Herman, J.: Panoramic mosaics by manifold projection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, pp. 338–343 (1997)

  52. Remondino, F.: Detectors and descriptors for photogrammetric applications. In: Proceedings of the Symposium on Photogrammetric Computer Vision, Bonn, pp. 49–54 (2006)

  53. Robinson, J.A.: A simplex-based projective transform estimator. In: Proceedings of the International Conference on Visual Information Engineering, Guildford, pp. 290–293 (2003)

  54. Rousso, B., Peleg, S., Finci, I., Rav-Acha, A.: Universal mosaicing using pipe projection. In: Proceedings of the Sixth IEEE International Conference on Computer Vision, Bombay, pp. 945–952 (1998)

  55. Sawhney, H.S., Kumar, R., Gendel, G., Bergen, J., Dixon, D., Paragano, V.: VideoBrushTM: experiences with consumer video mosaicing. In: Proceedings of the Workshop on Applications of Computer Vision, Los Alamitos, pp. 56–62 (1998)

  56. Schechner, Y.Y., Nayar, S.K.: Generalized mosaicing. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1, pp. 17–24, Vancouver (2001)

  57. Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000)

    Article  MATH  Google Scholar 

  58. Schmid, C., Mohr, R.: Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 530–534 (1997)

    Article  Google Scholar 

  59. Shum, H.Y., Szeliski, R.: Systems and experiment paper: construction of panoramic mosaics with global and local alignment. Int. J. Comput. Vis. 36(2), 101–130 (2000)

    Article  Google Scholar 

  60. Shum, H., Szeliski, R.: Panoramic image mosaics. Technical Report MSR-TR-97-23, Microsoft Research, USA (1997)

  61. Singh, R., Vatsa, M., Ross, A., Noore, A.: Performance enhancement of 2D face recognition via mosaicing. In: Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies, MorganTown, pp. 63–68 (2005)

  62. Smith, P., Sinclair, D., Cipolla, R., Wood, K.: Effective corner matching. In: Proceedings of the 9th British Machine Vision Conference, London, pp. 545–556 (1998)

  63. Sun, C.: Dealing with dense rows in the solution of sparse linear least squares problems. Technical Report CTC95TR227, Advanced Computing Research Institute, Cornell University, Ithaca (1995)

  64. Szeliski, R., Kang, S.B.: Direct methods for visual scene reconstruction. In: IEEE Workshop on Representation of Visual Scenes, Cambridge, pp. 26–33 (1995)

  65. Tell, D., Carlsson, S.: Combining appearance and topology for wide baseline matching. In: Proceedings of the 7th European Conference on Computer Vision, Copenhagen, pp. 68–81 (2002)

  66. Teyssier, P.: Reconstruction panoramique. Technical Report, Ecole Polytechnique, France (1999)

  67. Trucco, E., Doull, A., Odone, F., Fusiello, A., Lane, D.: Dynamic video mosaicing and augmented reality for subsea inspection and monitoring. In: Oceonology International 2000, Bringhton (2000)

  68. Unnikrishnan, R., Kelly, A.: Mosaicing large cyclic environments for visual navigation in autonomous vehicles. In: Proceedings of the IEEE International Conference on Robotics and Automation, Washington DC, pp. 4299–4306 (2002)

  69. Van Gool, L., Tuytelaars, T., Turina, A.: Local features for image retrieval. In: State-of-the-Art in Content-Based Image and Video Retrieval, pp. 21–41. Kluwer, Dordercht (2001)

  70. Wang, J., Quan, L., Sun, J., Tang, X., Shum, H.Y.: Picture Collage. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, pp. 347–354 (2006)

  71. Winter, M., Bischof, H., Fraundorfer F.: Maximally stable corner clusters: a novel distinguished region detector and descriptor. In: Proceedings of the Austrian Cognitive Vision Workshop, pp. 59–66 (2005)

  72. You, X., Fang, B., Tang, Y.Y.: Mosaicing the retinal fundus images: a robust registration technique based approach. Lect. Notes Comput. Sci. Adv. Natural Comput. 3612, 663–667 (2005)

    Google Scholar 

  73. Zagrouba, E., Belhassen, S.: Construction Robuste de Mosaiques Généralisées. In: Proceedings of the International Conference on Image and Signal Processing (ICISP’2003), Agadir, pp. 519–526 (2003)

  74. Zagrouba, E.: 3D facets construction for stereovision. In: International Conference on Artificial Intelligence pp. 116–126. Springer, Rome (1997)

  75. Zuliani, M., Kenney, C., Manjunath, B.S.: A mathematical comparison of point detectors. In: Proceedings of the IEEE Image and Video Registration Workshop (IVR), Washington DC (2004)

  76. Zitova, B., Flusser, J., Kautsky, J., Peters, G.: Feature point detection in multiframe images. In: Proceedings of the Czech Pattern Recognition Workshop, Czech Pattern Recognition Society, Perslak, pp. 117–122 (2000)

  77. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)

    Article  Google Scholar 

  78. Zoghlami, I., Faugeras, O., Deriche, R.: Using geometric corners to build a 2D mosaic from a set of image. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 420–425 (1997)

  79. Zomet, A., Peleg, S., Arora, C.: Rectified mosaicing: mosaics without the curl. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 2459–2465 (2000)

  80. Zomet, A., Peleg, S.: Applying super-resolution to panoramic mosaics. In: Proceedings of the IEEE Workshop on Applications of Computer Vision, Princeton, pp. 286–287 (1998)

Download references

Author information

Authors and Affiliations

  1. Unité de Recherche Systèmes Intelligents en Imagerie et Vision Artificielle (URSIIVA), Institut Supérieur d’Informatique d’El Manar, 2 Rue Abou Rayhane El Bayrouni, 2080, Ariana, Tunisia

    Ezzeddine Zagrouba, Walid Barhoumi & Slim Amri

Authors
  1. Ezzeddine Zagrouba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Walid Barhoumi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Slim Amri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walid Barhoumi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zagrouba, E., Barhoumi, W. & Amri, S. An efficient image-mosaicing method based on multifeature matching. Machine Vision and Applications 20, 139–162 (2009). https://doi.org/10.1007/s00138-007-0114-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-007-0114-y

Keywords

Search

Navigation