International Journal of Computer Vision

, Volume 68, Issue 1, pp 53–64

Feature-Based Sequence-to-Sequence Matching


DOI: 10.1007/s11263-005-4842-z

Cite this article as:
Caspi, Y., Simakov, D. & Irani, M. Int J Comput Vision (2006) 68: 53. doi:10.1007/s11263-005-4842-z


This paper studies the problem of matching two unsynchronized video sequences of the same dynamic scene, recorded by different stationary uncalibrated video cameras. The matching is done both in time and in space, where the spatial matching can be modeled by a homography (for 2D scenarios) or by a fundamental matrix (for 3D scenarios). Our approach is based on matching space-time trajectories of moving objects, in contrast to matching interest points (e.g., corners), as done in regular feature-based image-to-image matching techniques. The sequences are matched in space and time by enforcing consistent matching of all points along corresponding space-time trajectories.

By exploiting the dynamic properties of these space-time trajectories, we obtain sub-frame temporal correspondence (synchronization) between the two video sequences. Furthermore, using trajectories rather than feature-points significantly reduces the combinatorial complexity of the spatial point-matching problem when the search space is large. This benefit allows for matching information across sensors in situations which are extremely difficult when only image-to-image matching is used, including: (a) matching under large scale (zoom) differences, (b) very wide base-line matching, and (c) matching across different sensing modalities (e.g., IR and visible-light cameras). We show examples of recovering homographies and fundamental matrices under such conditions.


sequence-to-sequence matchingalignment in space and timedynamic informationmulti-sensor alignmentwide base-line matchingtrajectory matching

Copyright information

© Springer Science + Business Media, LLC. 2006

Authors and Affiliations

  1. 1.Dept. of Computer Science and Applied MathThe Weizmann Institute of ScienceRehovotIsrael