Machine Vision and Applications

, Volume 20, Issue 3, pp 139–162 | Cite as

An efficient image-mosaicing method based on multifeature matching

Original Paper

Abstract

Mosaicing is connecting two or more images and making a new wide area image with no visible seam-lines. Several algorithms have been proposed to construct mosaics from image sequence where the camera motion is more or less complex. Most of these methods are based either on the interest points matching or on theoretical corner models. This paper describes a fully automated image-mosaicing method based on the regions and the Harris points primitives. Indeed, in order to limit the search window of potential homologous points, for each point of interest, regions segmentation and matching steps are being performed. This enables us to improve the reliability and the robustness of the Harris points matching process by estimating the camera motion. The main originality of the proposed system resides in the preliminary manipulation of regions matching, thus making it possible to estimate the rotation, the translation and the scale factor between two successive images of the input sequence. This estimation allows an initial alignment of the images along with the framing of the interest points search window, and therefore reducing considerably the complexity of the interest points matching algorithm. Then, the resolution of a minimization problem, altogether considering the couples of matched-points, permits us to perform the homography. In order to improve the mosaic continuity around junctions, radiometric corrections are applied. The validity of the herewith described method is illustrated by being tested on several sequences of complex and challenging images captured from real-world indoor and outdoor scenes. These simulations proved the validity of the proposed method against camera motions, illumination variations, acquirement conditions, moving objects and image noise. To determine the importance of the regions matching stage in motion estimation, as well as for the framing of the search window associated to a point of interest, we compared the matching points results of this described method with those produced using the zero-mean normalized cross correlation score (without regions matching). We made this comparison in the case of a simple motion (without the presence of a rotation around optical axis and/or a scale factor), in the case of a rotation and in the general case of an homothety. For justifying the effectiveness of this method, we proposed an objective assessment by defining a reconstruction error.

Keywords

Regions matching Harris points Correlation scores Mosaic Reconstruction error 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Barhoumi, W., Zagrouba, E.: Segmentation en régions guidée par l’appariement pour un couple stéréoscopique non calibré. In: Proceedings of the 3eme Workshop Traitements et Analyse d’Information: Méthodes et Applications, Hammamet, Tunisia, pp. 287–292 (2003)Google Scholar
  2. 2.
    Baumberg, A.: Reliable feature matching across widely separated views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 774–781 (2000)Google Scholar
  3. 3.
    Bhosle, U., Chaudhuri, S., DuttaRoy, S.: Fast method for image mosaicing using geometric hashing. IETE J. Res. Spec. Issue Visual Media Process. 48(4), 317–324 (2002)Google Scholar
  4. 4.
    Brown, M., Lowe, D.: Invariant features from interest point groups. In: Proceedings of the 13th British Machine Vision Conference, Cardiff, pp. 253–262 (2002)Google Scholar
  5. 5.
    Brown, M., Lowe, D.G.: Recognising panoramas. In: Proceedings of the IEEE International Conference on Computer Vision, Nice, pp. 1218–1225 (2003)Google Scholar
  6. 6.
    Brown, M.: Multi-image matching using invariant features. PhD thesis, The University of British Columbia (2005)Google Scholar
  7. 7.
    Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)CrossRefGoogle Scholar
  8. 8.
    Can, A., Stewart, C.V., Roysam, B.: Robust hierarchical algorithm for constructing mosaic from images of the curved human retina. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado, pp. 286–292 (1999)Google Scholar
  9. 9.
    Choe, T.E., Cohen, I., Lee, M., Medioni, G.: Optimal global mosaic generation from retinal images. In: Proceedings of 18th International Conference on Pattern Recognition, Hong Kong, pp. 681–684 (2006)Google Scholar
  10. 10.
    Colombo, C., DelBimbo, A., Pernici, F.: Image mosaicing from uncalibrated views of a surface of revolution. In: Proceedings of British Machine Vision Conference, London (2004)Google Scholar
  11. 11.
    Davis, J.: Mosaics of scenes with moving objects. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara (1998)Google Scholar
  12. 12.
    Dellaert, F., Thrun, S., Thorpe, C.: Mosaicing a large number of widely dispersed, noisy, and distorted images: a bayesian approach. Technical Report, CMU-RI-TR-9934, Robotics Institute, Carnegie Mellon University, USA (1999)Google Scholar
  13. 13.
    Deriche, R., Blaszka, T.: Recovering and characterizing image features using an efficient model based approach. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, New York, pp. 530–535 (1993)Google Scholar
  14. 14.
    Deriche, R.: Recursively implementing the Gaussian and its derivatives. Technical Report, RR 2422 INRIA, France (1993)Google Scholar
  15. 15.
    Douze, M., Charvillat, V., Thiesse, B.: Mosaïques d’images par approximations successives. In: Proceedings of Journées Francophones des Jeunes Chercheurs en Analyse d’Images et Perception Visuelle (ORASIS’01), Cahors, pp 97–102 (2001)Google Scholar
  16. 16.
    Duplaquet, M.L.: Building large image mosaics with invisible seam-lines. In: Proceedings of the SPIE Congres AeroSence, Orlando, pp. 369–377 (1998)Google Scholar
  17. 17.
    Feldman, D., Zomet, A.: Generating mosaics with minimum distortions. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 163–170 (2004)Google Scholar
  18. 18.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Fraundorfer, F., Bischof, H.: Evaluation of local detectors on non-planar scenes. In: Proceedings of the 28th Workshop of the Austrian Association for Pattern Recognition, Hagenberg, pp. 125–132 (2004)Google Scholar
  20. 20.
    Fusiello, A., Aprile, M., Marzotto, R., Murino, V.: Mosaic of a video shot with multiple moving objects. In: Proceedings of the International Conference on Image Processing, Barcelona, pp. 307–310 (2003)Google Scholar
  21. 21.
    Gong, Y., Proietti, G., LaRose., D.: A robust image mosaicing technique capable of creating integrated panoramas. In: Proceedings of the IEEE International Conference on Information Visualization, London, pp. 24–29 (1999)Google Scholar
  22. 22.
    Gracias, X., Negahdaripour, S.: Underwater mosaic creation using video sequences from different altitudes. In: Proceedings of OCEANS’05 MTS/IEEE Conference, Boston, pp. 1295–1300 (2005)Google Scholar
  23. 23.
    Harris, C., Stephens, A.: Combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, Manchester, pp. 147–151 (1988)Google Scholar
  24. 24.
    Heikkila, M., Pietikainen, M.: An image mosaicing module for wide-area surveillance. In: Proceedings of the Third ACM International Workshop on Video Surveillance & Sensor Networks, Singapore, pp. 11–18 (2005)Google Scholar
  25. 25.
    Huang, X., Sun, Y., Metaxas, D., Sauer, F., Xu, C.: Hybrid image registration based on configural matching of scale-invariant salient region features. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, Washington DC, pp. 167–177 (2004)Google Scholar
  26. 26.
    Irani, M., Hsu, S., Anandan, P.: Video compression using mosaic representations. Elsevier-Sig. Process. Image Commun. 7, 529–552 (1995)CrossRefGoogle Scholar
  27. 27.
    Jethwa, M., Zisserman, A., Fitzgibbon, A.W.: Real-Time panoramic mosaics and augmented reality. In: Proceedings of the Britsh Machine Vision Conference, London, pp. 852–862 (1998)Google Scholar
  28. 28.
    Jianchao, Y., Chern, C.T.: The practice of automatic satellite image registration. Asian J. Geoinf. 3(4), 11–18 (2003)Google Scholar
  29. 29.
    Kanade, T., Rander, P.W., Narayaman, P.J.: Virtualized reality: constructing virtual worlds from real scenes. IEEE Trans. Multimedia 4(1), 34–47 (1997)CrossRefGoogle Scholar
  30. 30.
    Richmond, K., Rock, S.M.: An operational real-time large-scale visual mosaicking and navigation system. In: Proceedings of OCEANS’06 MTS/IEEE Conference, Boston, pp. 1–6 (2006)Google Scholar
  31. 31.
    Krotkov, E., Hebert, M., Simmons, R.: Stereo perception and dead reckoning for a prototype lunar rover. Auton. Robots 2(4), 313–331 (1995)CrossRefGoogle Scholar
  32. 32.
    Lhuillier, M., Quan, L.: Edge-constrained joint view triangulation for image interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 218–224 (2000)Google Scholar
  33. 33.
    Lowe, D.: Object recognition from local scale- invariant features. In: Proceedings of the International Conference on Computer Vision, Corfu, pp. 1150–1157 (1999)Google Scholar
  34. 34.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  35. 35.
    Mallick, S.P.: Feature based image mosaicing. Technical Report, CSE 252C University of California, San Diego (2002)Google Scholar
  36. 36.
    Mann, S., Picard, R.: Virtual bellows: Constructing high quality stills from video. In: First IEEE International Conference on Image Processing, Austin, pp. 363–367 (1994)Google Scholar
  37. 37.
    Marzotto, R., Fusiello, A., Murino, V.: High resolution video mosaicing with global alignment. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, pp. 692–698 (2004)Google Scholar
  38. 38.
    Matas, J., Burianek, J., Kittler, J.: Object recognition using the invariant pixel-set signature. In: Proceedings of the British Machine Vision Conference, London, pp. 606–615 (2000)Google Scholar
  39. 39.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British Machine Vision Conference, Cardiff, pp. 384–393 (2002)Google Scholar
  40. 40.
    McLauchlan, P.F., Jaenicke, A.: Image mosaicing using sequential bundle adjustment. In: Proceedings of the British Machine Vision Conference, Bristol, pp. 616–625 (2000)Google Scholar
  41. 41.
    McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: Proceedings of the 22nd Annual Conference on Computer Graphics (SIGGRAPH’95), Los Angeles, pp. 39–46 (1995)Google Scholar
  42. 42.
    Mehnert, A., Jackway, P.: An improved seeded region growing algorithm. Pattern Recogn. Lett. 18, 1065–1071 (1997)CrossRefGoogle Scholar
  43. 43.
    Merino, L., Wiklund, J., Caballero, F., Moe, A., De Dios, J.R.M., Forssen, P.E., Nordberg, K., Ollero, A.: Vision-based multi-UAV position estimation: Localization based on blob features for exploration missions. IEEE Robot. Autom. Magaz. 13(3), 53–62 (2006)CrossRefGoogle Scholar
  44. 44.
    Meunier, L., Borgmann, M.: High-resolution panoramas using image mosaicing. Final Project, EE368 (Digital Image Processing), Stanford University, USA (2000)Google Scholar
  45. 45.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: Proceedings of the Interational Conference on Computer Vision and Pattern Recognition (2003)Google Scholar
  46. 46.
    Mikolajczyk, K., Schmid, C.: Comparaison of affine-invariant local detectors and descriptors. In: Proceedings of the 12th European Signal Processing Conference, Vienna (2004)Google Scholar
  47. 47.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparaison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2006)Google Scholar
  48. 48.
    NagaoM. Matsuyama, T.: Edge preserving smoothing. Comput. Graph. Image Process. 9, 394–407 (1979)CrossRefGoogle Scholar
  49. 49.
    Nicolas, H.: New methods for dynamic mosaicing. IEEE Trans. Image Process. 10(8), 1239–1251 (2001)MATHCrossRefGoogle Scholar
  50. 50.
    Peleg, S., Rousso, B., Rav-Acha, A., Zomet, A.: Mosaicing on adaptive manifolds. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1144–1154 (2000)CrossRefGoogle Scholar
  51. 51.
    Peleg, S., Herman, J.: Panoramic mosaics by manifold projection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Juan, pp. 338–343 (1997)Google Scholar
  52. 52.
    Remondino, F.: Detectors and descriptors for photogrammetric applications. In: Proceedings of the Symposium on Photogrammetric Computer Vision, Bonn, pp. 49–54 (2006)Google Scholar
  53. 53.
    Robinson, J.A.: A simplex-based projective transform estimator. In: Proceedings of the International Conference on Visual Information Engineering, Guildford, pp. 290–293 (2003)Google Scholar
  54. 54.
    Rousso, B., Peleg, S., Finci, I., Rav-Acha, A.: Universal mosaicing using pipe projection. In: Proceedings of the Sixth IEEE International Conference on Computer Vision, Bombay, pp. 945–952 (1998)Google Scholar
  55. 55.
    Sawhney, H.S., Kumar, R., Gendel, G., Bergen, J., Dixon, D., Paragano, V.: VideoBrushTM: experiences with consumer video mosaicing. In: Proceedings of the Workshop on Applications of Computer Vision, Los Alamitos, pp. 56–62 (1998)Google Scholar
  56. 56.
    Schechner, Y.Y., Nayar, S.K.: Generalized mosaicing. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1, pp. 17–24, Vancouver (2001)Google Scholar
  57. 57.
    Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000)MATHCrossRefGoogle Scholar
  58. 58.
    Schmid, C., Mohr, R.: Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 19(5), 530–534 (1997)CrossRefGoogle Scholar
  59. 59.
    Shum, H.Y., Szeliski, R.: Systems and experiment paper: construction of panoramic mosaics with global and local alignment. Int. J. Comput. Vis. 36(2), 101–130 (2000)CrossRefGoogle Scholar
  60. 60.
    Shum, H., Szeliski, R.: Panoramic image mosaics. Technical Report MSR-TR-97-23, Microsoft Research, USA (1997)Google Scholar
  61. 61.
    Singh, R., Vatsa, M., Ross, A., Noore, A.: Performance enhancement of 2D face recognition via mosaicing. In: Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies, MorganTown, pp. 63–68 (2005)Google Scholar
  62. 62.
    Smith, P., Sinclair, D., Cipolla, R., Wood, K.: Effective corner matching. In: Proceedings of the 9th British Machine Vision Conference, London, pp. 545–556 (1998)Google Scholar
  63. 63.
    Sun, C.: Dealing with dense rows in the solution of sparse linear least squares problems. Technical Report CTC95TR227, Advanced Computing Research Institute, Cornell University, Ithaca (1995)Google Scholar
  64. 64.
    Szeliski, R., Kang, S.B.: Direct methods for visual scene reconstruction. In: IEEE Workshop on Representation of Visual Scenes, Cambridge, pp. 26–33 (1995)Google Scholar
  65. 65.
    Tell, D., Carlsson, S.: Combining appearance and topology for wide baseline matching. In: Proceedings of the 7th European Conference on Computer Vision, Copenhagen, pp. 68–81 (2002)Google Scholar
  66. 66.
    Teyssier, P.: Reconstruction panoramique. Technical Report, Ecole Polytechnique, France (1999)Google Scholar
  67. 67.
    Trucco, E., Doull, A., Odone, F., Fusiello, A., Lane, D.: Dynamic video mosaicing and augmented reality for subsea inspection and monitoring. In: Oceonology International 2000, Bringhton (2000)Google Scholar
  68. 68.
    Unnikrishnan, R., Kelly, A.: Mosaicing large cyclic environments for visual navigation in autonomous vehicles. In: Proceedings of the IEEE International Conference on Robotics and Automation, Washington DC, pp. 4299–4306 (2002)Google Scholar
  69. 69.
    Van Gool, L., Tuytelaars, T., Turina, A.: Local features for image retrieval. In: State-of-the-Art in Content-Based Image and Video Retrieval, pp. 21–41. Kluwer, Dordercht (2001)Google Scholar
  70. 70.
    Wang, J., Quan, L., Sun, J., Tang, X., Shum, H.Y.: Picture Collage. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, pp. 347–354 (2006)Google Scholar
  71. 71.
    Winter, M., Bischof, H., Fraundorfer F.: Maximally stable corner clusters: a novel distinguished region detector and descriptor. In: Proceedings of the Austrian Cognitive Vision Workshop, pp. 59–66 (2005)Google Scholar
  72. 72.
    You, X., Fang, B., Tang, Y.Y.: Mosaicing the retinal fundus images: a robust registration technique based approach. Lect. Notes Comput. Sci. Adv. Natural Comput. 3612, 663–667 (2005)Google Scholar
  73. 73.
    Zagrouba, E., Belhassen, S.: Construction Robuste de Mosaiques Généralisées. In: Proceedings of the International Conference on Image and Signal Processing (ICISP’2003), Agadir, pp. 519–526 (2003)Google Scholar
  74. 74.
    Zagrouba, E.: 3D facets construction for stereovision. In: International Conference on Artificial Intelligence pp. 116–126. Springer, Rome (1997)Google Scholar
  75. 75.
    Zuliani, M., Kenney, C., Manjunath, B.S.: A mathematical comparison of point detectors. In: Proceedings of the IEEE Image and Video Registration Workshop (IVR), Washington DC (2004)Google Scholar
  76. 76.
    Zitova, B., Flusser, J., Kautsky, J., Peters, G.: Feature point detection in multiframe images. In: Proceedings of the Czech Pattern Recognition Workshop, Czech Pattern Recognition Society, Perslak, pp. 117–122 (2000)Google Scholar
  77. 77.
    Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)CrossRefGoogle Scholar
  78. 78.
    Zoghlami, I., Faugeras, O., Deriche, R.: Using geometric corners to build a 2D mosaic from a set of image. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 420–425 (1997)Google Scholar
  79. 79.
    Zomet, A., Peleg, S., Arora, C.: Rectified mosaicing: mosaics without the curl. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, South Carolina, pp. 2459–2465 (2000)Google Scholar
  80. 80.
    Zomet, A., Peleg, S.: Applying super-resolution to panoramic mosaics. In: Proceedings of the IEEE Workshop on Applications of Computer Vision, Princeton, pp. 286–287 (1998)Google Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  1. 1.Unité de Recherche Systèmes Intelligents en Imagerie et Vision Artificielle (URSIIVA)Institut Supérieur d’Informatique d’El ManarArianaTunisia

Personalised recommendations