Global Optimization of Object Pose and Motion from a Single Rolling Shutter Image with Automatic 2D-3D Matching

  • Ludovic Magerand
  • Adrien Bartoli
  • Omar Ait-Aider
  • Daniel Pizarro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


Low cost CMOS cameras can have an acquisition mode called rolling shutter which sequentially exposes the scan-lines. When a single object moves with respect to the camera, this creates image distortions. Assuming 2D-3D correspondences known, previous work showed that the object pose and kinematics can be estimated from a single rolling shutter image. This was achieved using a suboptimal initialization followed by local iterative optimization.

We propose a polynomial projection model for rolling shutter cameras and a constrained global optimization of its parameters. This is done by means of a semidefinite programming problem obtained from the generalized problem of moments method. Contrarily to previous work, our optimization does not require an initialization and ensures that the global minimum is achieved. This allows us to build automatically robust 2D-3D correspondences using a template to provide an initial set of correspondences.

Experiments show that our method slightly improves previous work on both simulated and real data. This is due to local minima into which previous methods get trapped. We also successfully experimented building 2D-3D correspondences automatically with both simulated and real data.


rolling shutter motion estimation matching 


  1. 1.
    Bradley, D., Atcheson, B., Ihrke, I., Heidrich, W.: Synchronization and rolling shutter compensation for consumer video camera arrays. In: PROCAMS (2009)Google Scholar
  2. 2.
    Liang, C., Chang, L., Chen, H.: Analysis and compensation of rolling shutter effect. IEEE Trans. on Image Processing (2008)Google Scholar
  3. 3.
    Forssén, P.E., Ringaby, E.: Rectifying rolling shutter video from hand-held devices. In: CVPR (2010)Google Scholar
  4. 4.
    Ait-Aider, O., Andreff, N., Lavest, J.-M., Martinet, P.: Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part II. LNCS, vol. 3952, pp. 56–68. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Ait-Aider, O., Bartoli, A.E., Andreff, N.: Kinematics from lines in a single rolling shutter image. In: CVPR (2007)Google Scholar
  6. 6.
    Meingast, M., Geyer, C., Sastry, S.: Geometric models of rolling-shutter cameras. CoRR (2005)Google Scholar
  7. 7.
    Dahmouche, R., Ait-Aider, O., Andreff, N., Mezouar, Y.: High-speed pose and velocity measurement from vision. In: ICRA (2008)Google Scholar
  8. 8.
    Ait-Aider, O., Berry, F.: Structure and kinematics triangulation with a rolling shutter stereo rig. In: ICCV (2009)Google Scholar
  9. 9.
    Magerand, L., Bartoli, A.: A generic rolling shutter camera model and its application to dynamic pose estimation. In: 3DPVT (2010)Google Scholar
  10. 10.
    Henrion, D., Lasserre, J., Loefberg, J.: Gloptipoly 3: moments, optimization and semidefinite programming. Optimization Methods and Software (2009)Google Scholar
  11. 11.
    Borchers, B., Young, J.G.: Implementation of a primal-dual method for sdp on a shared memory parallel architecture. Computational Optimization and Applications (2007)Google Scholar
  12. 12.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge Univ. Press (2004)Google Scholar
  13. 13.
    Paragios, N., Chen, Y., Faugeras, O.: Handbook of Mathematical Models in Computer Vision. Springer (2006)Google Scholar
  14. 14.
    Heikkila, J., Silvén, O.: A four-step camera calibration procedure with implicit image correction. In: CVPR (1997)Google Scholar
  15. 15.
    Tsai, R.Y.: An efficient and accurate camera calibration technique for 3D machine vision. In: CVPR (1986)Google Scholar
  16. 16.
    Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: ICCV (1999)Google Scholar
  17. 17.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: An accurate O(n) solution to the PnP problem. IJCV (2009)Google Scholar
  18. 18.
    Murray, R.M., Li, Z., Sastry, S.S.: A Mathematical Introduction to Robotic Manipulation. CRC Press Inc. (1994)Google Scholar
  19. 19.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar
  20. 20.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ludovic Magerand
    • 1
    • 2
  • Adrien Bartoli
    • 2
  • Omar Ait-Aider
    • 1
  • Daniel Pizarro
    • 3
  1. 1.Institut PascalUniversité Blaise PascalClermont-FerrandFrance
  2. 2.ISITUniversité d’AuvergneClermont-FerrandFrance
  3. 3.University of AlcalaAlcala de HenaresSpain

Personalised recommendations