Abstract
In order to review past accomplishments and discuss future challenges, we organized a panel session during the workshop. Four panelists were invited to participate: Steve Blask from Harris Corporation, Lisa Brown from IBM, Harpreet Sawhney from Sarnoff Corporation, and Rick Szeliski from Microsoft. The panelists were asked to select three or four questions from the following ten questions and express their views. In this section, I will discuss each question, and express some of my opinions. The next three sections deal with the opinions of the three panelists.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D. A. Adjeroh, M. C. Lee, I. King, “A Distance Measure for Video Sequences,” Computer Vision and Image Understanding, Vol. 75, Nos. 1&2, p25–45, July/August 1999.
Richard W. Cannata, Mubarak Shah, Steven G. Blask, John A. Van Workum, “Autonomous Video Registration Using Sensor Model Parameter Adjustments”, Applied Imagery Pattern Recognition Workshop (AIPR) 2000, Cosmos Club, Washington D.C., Oct 16–18, 2000.
L. Gottesfeld Brown, “Registration of Multi-Modal Medical Images: Exploiting Sensor Relationships,” Dissertation, Computer Science Dept., Columbia University, 1996.
L. Gottesfeld Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, Vol. 24, No.4, p325–376, December 1992.
Y. Caspi and M. Irani, “A Step Toward Sequence-to-Sequence Alignment,” IEEE Conf. on Computer Vision and Pattern Recognition,” Vol. 1, Hilton Head Island, S.C., p682–689, June 13–15 2000.
A.W. Fitzgibbon, “Stochastic Rigidity: Image Registration for Nowhere-static Scenes,” IEEE Int’l Conf. Computer Vision, Vancouver, BC, p662–669, July 9–12, 2001.
A. Hampapur and R. Bolle, “Comparison of Distance Measures fro Video Copy Detection,” Proc. of the Int’l Conf. on Multimedia and Expo, Japan, August 2001.
M. Hansen, P. Anandan, K. Dana, G. van der Wal, P. Burt, “Realtime Scene Stabilization and Mosaic Construction,” Proc. Image Understanding Workshop, Vol. 1, Monterey, CA, p457–65, 1994.
J. Helferty, C. Zhang, G. McLennan, W. Higgins, “Videoendoscopic Distortion Correction and Its Application to Virtual Guidance of Endoscopy,” IEEE Trans. on Medical Imaging, Vol. 20, No.7, p605–617, July 7, 2001.
Y.S. Heung, R. Szeliski, “Systems and Experiment Paper: Construction of panoramic Image Mosaics with Global and Local Alignment,” Int’l Journal Computer Vision (Netherlands) Vol. 36, No.2, p101–30, 2000.
S. Hsu, S. Samarasekera, R. Kumar, H. Sawhney, “Pose Estimation, Model Refinement, and Enhanced Visualization Using Video,” IEEE Conf. on Computer Vision and Pattern Recognition,” Vol. 1, Hilton Head Island, S.C., p488–495, June 13–15 2000.
G. Ionescu, S. Lavallee, J. Demongeot, J., “Automated Registration of Ultrasound and CT Images: Application to Computer Assisted Prostate Radiotherapy and Orthopedics,” Medical Image Computing and Computer Assisted Intervention — MICCAI’99, Vol. 1679, p768–77, Cambridge UK, Sept 19–22, 1999.
M. Irani and P. Anandan, “Robust Multi Sensor Image Alignment,” Sixth International Conference on Computer Vision, Bombay, India, p959–66, January 1998.
J.S. Jin, Z. Z. Zhu, G. Xu, “Digital Video Sequence Stabilization Based on 2.5D Motion Estimation and Inertial Motion Filtering,” Real Time Imaging (UK) Vol. 7, No.4, p357–65, August 2001.
L. Lee, R. Romano, G. Stein, “Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame,” IEEE Trans. on Pattern Analysis and Machin Intelligence, Vol. 22, No.8, p758–767, August 2000.
T.M. Lehmann, H-G Grndahl, and D.K. Benn, “Computer-based Registration for Digital Subtraction in Dental Radiology,” Dentomaxillofacial Radiology, Vol, 29, p323–346, 2000.
J.B. Antoine Maintz and Max. A. Viergever, “A Survey of Medical Image Registration,” Medical Image Analysis, Vol, 2, No.1, p1–36, 1998.
A. Mittal, D. Huttenlocher, “Scene Modeling for Wide Area Surveillance and Image Synthesis,” IEEE Proc. Computer Vision and Pattern Recognition, Hilton Head Island, SC, Vol. 2, p160–167, June 13–15, 2000.
F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality Image Registration by Maximization of Mutual Information,” IEEE Transactions on Medical Imaging, Vol. 16, No.2, p187–198, April 1997.
R. Mohan, “Video Sequence Matching,” Proc. of the 1998 IEEE Conf. on Acoustics, Speech and Signal Processing, Vol. 6, p3697–700, Seattle WA, May 12–15, 1998.
M. Otte, “Elastic Registration of fMRI Data Using Bzier-Spline Transformations,” IEEE Trans. on Medical Imaging, Vol. 20, No.2, p193–206, February 2001.
H.S. Sawhney and R. Kumar, “True Multi-Image Alignment and Its Applications to Mosaicing and Lens Distortion Correction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.21, No.3, p235–243, March 1999.
S. Satoh, “Comparative Evaluation of Face Sequence Matching for Content-based Video Access,” Proc. 4th Int’l Conf. on Automatic Face and Gesture Recognition” Grenoble, France, p163–8, March 28–30, 2000.
F. Schaffalitzky and A. Zisserman, “Viewpoint Invariant Texture Matching and Wide Baseline Stereo,” Conf. on Computer Vision, Vancouver, BC, Vol. II, p636–643, July 2001.
I. Stamos and P.K. Allen, “Automatic Registration of 2-D with 3-D Imagery in Urban Environments,” Int’l Conf. on Computer Vision, Vancouver, BC, Vol. II, p731–736, July 2001.
J.D. Stefansic, et. al., “Registration of Physical Space to Laparoscopic Image Space for Use in Minimally Invasive Hepatic Surgery,” IEEE Trans. on Medical Imaging, Vol. 19, No. 10, p1012–1023, October 2000.
J.-P. Thirion, “Non-rigid Matching using Demons,” Proc. Computer Vision and Pattern Recognition, San Francisco, CA, p245–251, June 18–20, 1996.
K. Toyama, “Prolegomena for Robust Face Tracking,” Workshop on Automatic Facial Image Analysis and Recognition Technology (ECCV 98).
P. Viola and W.M. Wells III, “Alignment by Maximization of Mutual Information,” Int’l Journal Computer Vision, Vol. 24, No.2, Netherlands, p137–54, 1997.
Yu, T., Zhang, Y. “Retrieval of Video Clips Using Global Motion Information,” Electronic Letters, Vol. 37, No. 14, p893–895, July 5, 2001
References
D. A. Adjeroh, M. C. Lee, I. King, “A Distance Measure for Video Sequences,” Computer Vision and Image Understanding, Vol. 75, Nos. 1&2, p25–45, July/August 1999.
Richard W. Cannata, Mubarak Shah, Steven G. Blask, John A. Van Workum, “Autonomous Video Registration Using Sensor Model Parameter Adjustments”, Applied Imagery Pattern Recognition Workshop (AIPR) 2000, Cosmos Club, Washington D.C., Oct 16–18, 2000.
L. Gottesfeld Brown, “Registration of Multi-Modal Medical Images: Exploiting Sensor Relationships,” Dissertation, Computer Science Dept., Columbia University, 1996.
L. Gottesfeld Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, Vol. 24, No.4, p325–376, December 1992.
Y. Caspi and M. Irani, “A Step Toward Sequence-to-Sequence Alignment,” IEEE Conf. on Computer Vision and Pattern Recognition,” Vol. 1, Hilton Head Island, S.C., p682–689, June 13–15 2000.
A.W. Fitzgibbon, “Stochastic Rigidity: Image Registration for Nowhere-static Scenes,” IEEE Int’l Conf. Computer Vision, Vancouver, BC, p662–669, July 9–12, 2001.
A. Hampapur and R. Bolle, “Comparison of Distance Measures fro Video Copy Detection,” Proc. of the Int’l Conf. on Multimedia and Expo, Japan, August 2001.
M. Hansen, P. Anandan, K. Dana, G. van der Wal, P. Burt, “Realtime Scene Stabilization and Mosaic Construction,” Proc. Image Understanding Workshop, Vol. 1, Monterey, CA, p457–65, 1994.
J. Helferty, C. Zhang, G. McLennan, W. Higgins, “Videoendoscopic Distortion Correction and Its Application to Virtual Guidance of Endoscopy,” IEEE Trans. on Medical Imaging, Vol. 20, No.7, p605–617, July 7, 2001.
Y.S. Heung, R. Szeliski, “Systems and Experiment Paper: Construction of panoramic Image Mosaics with Global and Local Alignment,” Int’l Journal Computer Vision (Netherlands) Vol. 36, No.2, p101–30, 2000.
S. Hsu, S. Samarasekera, R. Kumar, H. Sawhney, “Pose Estimation, Model Refinement, and Enhanced Visualization Using Video,” IEEE Conf. on Computer Vision and Pattern Recognition,” Vol. 1, Hilton Head Island, S.C., p488–495, June 13–15 2000.
G. Ionescu, S. Lavallee, J. Demongeot, J., “Automated Registration of Ultrasound and CT Images: Application to Computer Assisted Prostate Radiotherapy and Orthopedics,” Medical Image Computing and Computer Assisted Intervention — MICCAI’99, Vol. 1679, p768–77, Cambridge UK, Sept 19–22, 1999.
M. Irani and P. Anandan, “Robust Multi Sensor Image Alignment,” Sixth International Conference on Computer Vision, Bombay, India, p959–66, January 1998.
J.S. Jin, Z. Z. Zhu, G. Xu, “Digital Video Sequence Stabilization Based on 2.5D Motion Estimation and Inertial Motion Filtering,” Real Time Imaging (UK) Vol. 7, No.4, p357–65, August 2001.
L. Lee, R. Romano, G. Stein, “Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame,” IEEE Trans. on Pattern Analysis and Machin Intelligence, Vol. 22, No.8, p758–767, August 2000.
T.M. Lehmann, H-G Grndahl, and D.K. Benn, “Computer-based Registration for Digital Subtraction in Dental Radiology,” Dentomaxillofacial Radiology, Vol, 29, p323–346, 2000.
J.B. Antoine Maintz and Max. A. Viergever, “A Survey of Medical Image Registration,” Medical Image Analysis, Vol, 2, No.1, p1–36, 1998.
A. Mittal, D. Huttenlocher, “Scene Modeling for Wide Area Surveillance and Image Synthesis,” IEEE Proc. Computer Vision and Pattern Recognition, Hilton Head Island, SC, Vol. 2, p160–167, June 13–15, 2000.
F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality Image Registration by Maximization of Mutual Information,” IEEE Transactions on Medical Imaging, Vol. 16, No.2, p187–198, April 1997.
R. Mohan, “Video Sequence Matching,” Proc. of the 1998 IEEE Conf. on Acoustics, Speech and Signal Processing, Vol. 6, p3697–700, Seattle WA, May 12–15, 1998.
M. Otte, “Elastic Registration of fMRI Data Using Bzier-Spline Transformations,” IEEE Trans. on Medical Imaging, Vol. 20, No.2, p193–206, February 2001.
H.S. Sawhney and R. Kumar, “True Multi-Image Alignment and Its Applications to Mosaicing and Lens Distortion Correction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.21, No.3, p235–243, March 1999.
S. Satoh, “Comparative Evaluation of Face Sequence Matching for Content-based Video Access,” Proc. 4th Int’l Conf. on Automatic Face and Gesture Recognition” Grenoble, France, p163–8, March 28–30, 2000.
F. Schaffalitzky and A. Zisserman, “Viewpoint Invariant Texture Matching and Wide Baseline Stereo,” Conf. on Computer Vision, Vancouver, BC, Vol. II, p636–643, July 2001.
I. Stamos and P.K. Allen, “Automatic Registration of 2-D with 3-D Imagery in Urban Environments,” Int’l Conf. on Computer Vision, Vancouver, BC, Vol. II, p731–736, July 2001.
J.D. Stefansic, et. al., “Registration of Physical Space to Laparoscopic Image Space for Use in Minimally Invasive Hepatic Surgery,” IEEE Trans. on Medical Imaging, Vol. 19, No. 10, p1012–1023, October 2000.
J.-P. Thirion, “Non-rigid Matching using Demons,” Proc. Computer Vision and Pattern Recognition, San Francisco, CA, p245–251, June 18–20, 1996.
K. Toyama, “Prolegomena for Robust Face Tracking,” Workshop on Automatic Facial Image Analysis and Recognition Technology (ECCV 98).
P. Viola and W.M. Wells III, “Alignment by Maximization of Mutual Information,” Int’l Journal Computer Vision, Vol. 24, No.2, Netherlands, p137–54, 1997.
Yu, T., Zhang, Y. “Retrieval of Video Clips Using Global Motion Information,” Electronic Letters, Vol. 37, No. 14, p893–895, July 5, 2001
References
S. Ayer and H. S. Sawhney. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and mdl encoding. In International Conference on Computer Vision, pages 777–785, Cambridge, June 1995.
S. Baker, R. Szeliski, and P. Anandan. A layered approach to stereo reconstruction. In CVPR98, pages 434–441, 1998.
J. R. Bergen et al. Hierarchical model-based motion estimation. In Proc. 2nd European Conference on Computer Vision, pages 237–252, 1992.
Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. In ICCV99, pages 377–384, 1999.
M. Irani and P. Anandan. Robust multi-sensor image alignment. In Proc. Intl. Conf. on Computer Vision, 1998.
M. Isard and J.P. MacCormick. Bramble: A bayesian multiple-blob tracker. In ICCV01, pages II: 34–41, 2001.
R. Mandelbaum, G. Salgian, and H. S. Sawhney. Correlation-based estimation of ego-motion and structure from motion and stereo. In Proc. Intl. Conf. on Computer Vision, 1999.
P.J. Narayanan, P.W. Rander, and T. Kanade. Constructing virtual worlds using dense stereo. In ICCV, pages 3–10, 1998.
B. Rousso, S. Peleg, I. Finci, and A. Rav-Acha, Universal mosaicing using pipe projection. In ICCV98, pages 945–952, 1998.
Harpreet S. Sawhney, Steve Hsu, and R. Kumar. Robust video mosaicing through topology inference and local to global alignment. In ECCV, pages 103–119, 1998.
C. Schmid and R. Mohr. Combining greyvalue invariants with local constraints for object recognition. In Proc. Computer Vision and Pattern Recognition Conference, pages 872–877, 1996.
R. Szeliski. Prediction error as a quality metric for motion and stereo. In Proc. Intl. Conf. on Computer Vision, 1999.
H. Tao, H.S. Sawhney, and R. Kumar. Dynamic layer representation with applications to tracking. In CVPR00, pages II:134–141, 2000.
H. Tao, H.S. Sawhney, and R. Kumar. Dynamic depth recovery from multiple synchronized video streams. In CVPR01, 2001.
H. Tao, H.S. Sawhney, and R. Kumar. A global matching framework for stereo computation. In ICCV01, pages I: 532–539, 2001.
S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade. Three-dimensional scene flow. In ICCV, pages 722–729, 1999.
J. Y. A Wang and E. H. Adelson. Layered representation for motion analysis. In Proc. Computer Vision and Pattern Recognition Conference, pages 361–366, 1993.
R.P. Wildes, D.J. Hirvonen, et al. Video georegistration: Algorithm and quantitative evaluation. In ICCV01, pages II: 343–350, 2001.
References
J. L. Barron, D. J. Fleet, and S. S. Beauchemin. Performance of optical flow techniques. International Journal of Computer Vision, 12(1):43–77, January 1994.
R. Benosman and S. B. Kang, editors. Panoramic Vision: Sensors, Theory, and Applications, New York, 2001. Springer.
J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani. Hierarchical model-based motion estimation. In Second European Conference on Computer Vision (ECCV’92), pages 237–252, Santa Margherita Liguere, Italy, May 1992. Springer-Verlag.
M. J. Black and P. Anandan. The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104, 1996.
M. J. Black and A. Rangarajan. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1):57–91, 1996.
A. F. Bobick and S. S. Intille. Large occlusion stereo. International Journal of Computer Vision, 33(3):181–200, September 1999.
Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222–1239, November 2001.
S. E. Chen. QuickTime VR — an image-based approach to virtual environment navigation. Computer Graphics (SIGGRAPH’95), pages 29–38, August 1995.
J. Davis. Mosaics of scenes with moving objects. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’98), pages 354–360, Santa Barbara, June 1998.
M. Hansen, P. Anandan, K. Dana, G. van der Wal, and P. Burt. Real-time scene stabilization and mosaic construction. In IEEE Workshop on Applications of Computer Vision (WACV’94), pages 54–62, Sarasota, December 1994. IEEE Computer Society.
M. Irani and P. Anandan. Video indexing based on mosaic representations. Proceedings of the IEEE, 86(5):905–921, May 1998.
M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5–16, January 1994.
S. X. Ju, M. J. Black, and A. D. Jepson. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’96), pages 307–314, San Francisco, June 1996.
S. B. Kang, R. Szeliski, and J. Chai. Handling occlusions in dense multi-view stereo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2001), volume I, pages 103–110, Kauai, Hawaii, December 2001.
R. Kumar, P. Anandan, M. Irani, J. Bergen, and K. Hanna. Representation of scenes from collections of images. In IEEE Workshop on Representations of Visual Scenes, pages 10–17, Cambridge, Massachusetts, June 1995.
D. L. Milgram. Computer methods for creating photomosaics. IEEE Transactions on Computers, C-24(11):1113–1119, November 1975.
D. L. Milgram. Adaptive techniques for photomosaicking. IEEE Transactions on Computers, C-26(11):1175–1180, November 1977.
D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1):7–42, May 2002.
A. Schödl, R. Szeliski, D. H. Salesin, and I. Essa. Video textures. In Computer Graphics (SIGGRAPH’2000) Proceedings, pages 489–498, New Orleans, July 2000. ACM SIGGRAPH.
H.-Y. Shum and R. Szeliski. Construction of panoramic mosaics with global and local alignment. International Journal of Computer Vision, 36(2):101–130, February 2000.
A. R. Smith and J. F. Blinn. Blue screen matting. In Computer Graphics Proceedings, Annual Conference Series, pages 259–268, Proc. SIGGRAPH’96 (New Orleans), August 1996.ACM SIGGRAPH.
R. Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16(2):22–30, March 1996.
R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR’2000), volume 1, pages 246–253, Hilton Head Island, June 2000.
R. Szeliski and P. Golland. Stereo matching with transparency and matting. International Journal of Computer Vision, 32(1):45–61, August 1999. Special Issue for Marr Prize papers.
H. Tao, H.S. Sawhney, and R. Kumar. A global matching framework for stereo computation. In Eighth International Conference on Computer Vision (ICCV 2001), volume I, pages 532–539, Vancouver, Canada, July 2001.
M. Uyttendaele, A. Eden, and R. Szeliski. Eliminating ghosting and exposure artifacts in image mosaics. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2001), volume II, pages 509–516, Kauai, Hawaii, December 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer Science+Business Media New York
About this chapter
Cite this chapter
Shah, M., Brown, L., Sawhney, H.S., Szeliski, R. (2003). Panel Introduction Video Registration: Key Challenges and the Potential Impact of their Solutions to the Field of Computer Vision. In: Shah, M., Kumar, R. (eds) Video Registration. The International Series in Video Computing, vol 5. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-0459-7_9
Download citation
DOI: https://doi.org/10.1007/978-1-4615-0459-7_9
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-5087-3
Online ISBN: 978-1-4615-0459-7
eBook Packages: Springer Book Archive