Abstract
In this paper, we present a system for the estimation of the surface structure and the motion parameters of a free-flying object in a tele-robotics experiment. The system consists of two main components: (i) a vision-based invariant-surface and motion estimator and (ii) a Kalman filter state estimator. We present a new algorithm for motion estimation from sparse multi-sensor range data. The motion estimates from the vision-based estimator are input to a Kalman filter state estimator for continuously tracking a free-flying object in space under zero-gravity conditions. The predicted position and orientation parameters are then fed back to the vision module of the system and serve as an initial guess in the search for optimal motion parameters. The task of the vision module is two-fold: (i) estimating a piecewise-smooth surface from a single frame of multi-sensor data and (ii) determining the most likely (in the Bayesian sense) object motion that makes data in subsequent time frames to have been sampled from the same piecewise-smooth surface. With each incoming data frame, the piecewise-smooth surface is incrementally refined. The problem is formulated as an energy minimization and solved numerically resulting in a surface estimate invariant to 3D rigid motion and the vector of motion parameters. Performance of the system is depicted on simulated and real range data.
Similar content being viewed by others
References
S. Ullman, ‘The Interpretation of Structure from Motion,’ Proc. R. Soc. London, vol. B203, 1979, pp. 405–426.
J.W. Roach and J.K. Aggarwal, “Determining the Movement of Objects from a Sequence of Images,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-2, 1980, pp. 554–562.
J.A. Webb and J.K. Aggarwal, ‘Structure and Motion of Rigid and Jointed Objects,’ Artif. Intell., no. 19, 1981, pp. 107–130.
R.T. Tsai and T.S. Huang, ‘Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surface, IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-6, 1984, pp. 13–26.
A. Mitiche, S. Seida, and J.K. Aggarwal, ‘Line-Based Computation of Structure and Motion Using Angular Invariance,’ in Proceedings of the IEEE Computer Society Workshop on Motion, May 1986, pp. 175–180.
J. Weng, T.S. Huang, and N. Ahuja, ‘3-D Motion Estimation, Understanding and Prediction from Noisy Image Sequences,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-9, 1987, pp. 370–389.
J.K. Aggarwal and N. Nandhakumar, On the Computation of Motion from a Sequence of Images—A Review, Technical Report, University of Texas at Austin, TR-88-2-47, 1988.
O.D. Fugeras and M. Herbert, ‘The Representation, Recognition and Positioning of 3D Shapes from Range Data,’ in Three-Dimensional Machine Vision, T. Kanade, ed., Boston: Kluwer Academic Publishers, 1987, pp. 301–353.
Z.C. Lin, T.S. Huang, S.D. Bolstein, H. Lee, and E.A. Margerum, ‘Motion Estimation from 3D Point sets with and without Correspondence,’ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, June 22–26, 1986, pp. 194–201.
R.S. Szeliski, ‘Estimating Motion from Sparse Range Data without Correspondence,’ in Proceedings of the IEEE Second International Conference on Computer Vision, Tarpon Springs, FL, Dec. 5–8, 1988, pp. 207–216.
G. Hirzinger, J. Heindl, and K. Landzettel, ‘Predictive and Knowledge-Based Tele Robotics Control Concepts,’ IEEE International Conference on Robotics and Automation, Scottsdale, AZ, May 14–18, 1989.
D. Terzopoulos, ‘The Computation of Visible-Surface Representations,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-10, 1988, pp. 417–438.
B.C. Vemuri, D. Terzopoulos, and P.J. Lewicki, ‘Canonical Parameters for Invariant Surface Representation,’ SPIE Conference on Advances in Intelligent Robotic Systems, Pnhiladelphia, November 1989.
P.S. Maybeck, Stochastic Models, Estimation and Control, vol. 1, New York: Academic Press, 1979.
R.M. Bolle and B.C. Vemuri, ‘On 3D Surface Reconstruction Methods,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-13, 1991, pp. 1–13.
W.E.L. Grimson, ‘An Implementation of a Computational Theory of Visual Surface Interpolation,’ Comput. Vis., Graph., Image Process., vol. 22, 1983, pp. 39–69.
D. Terzopoulos, ‘Multilevel Computational Processes for Visible Surface Reconstruction, Comput. Vis., Graph., Image Process., vol. 24, 1983, pp. 52–96.
A. Blake and A. Zisserman, Visual Reconstruction, Cambridge, MA: MIT Press, 1987.
B.C. Vemuri, ‘Representation and Recognition of Objects from Dense Range Maps,’ Ph.D. thesis, Dept. of Electrical and Computer Engineering, University of Texas, Austin, TX, 1987.
A. Gelb, ed., Applied Optimal Estimation, Cambridge, MA: MIT Press, 1974.
B.C. Vemuri, A. Mitiche, and J.K. Aggarwal, ‘Curvature-Based Representation of Objects from Range Data,’ Intl. J. Image Vis. Comput., vol. 4, 1986, pp. 107–114.
S. Geman and D. Geman, ‘Stochastic Relaxation, Gibbs Distribution, and the Bayesian Restoration of Images,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-6, 1984, pp. 721–741.
D. Terzopoulos, ‘Regularization of Inverse Visual Problems Involving Discontinuities,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-8, 1986, pp. 413–424.
W. Press et al., Numerical Recipes: The Art Of Scientific Computing, Cambridge: Cambridge University Press, 1986.
R.S. Szeliski, ‘Fast Surface Interpolation Using Hierarchical Basis Functions,’ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, June 4–8, 1989, pp. 222–228.
G. Taubin, Algebraic Nonplanar Curve and Surface Estimation in 3-space with Applications to Position Estimation, IBM Technical Report RC-13873, 1988.
T.W. Seederberg, D.C. Anderson, and R.N. Goldman, ‘Implicit Representation of Parametric Curves and Surfaces, Comput. Vis., Graph., Image Process., vol.28, 1984, pp.72–84.
L.E. Elsgolc, Calculus of Variations, Reading, MA: Addison-Wesley Publishing Company, 1962.
J.A. Thorpe, Elementary Topics in Differential Geometry, Berlin: Springer-Verlag, 1979.
G. Dahlquist and A. Björck, Numerical Methods, Englewood Cliffs, NJ: Prentice-Hall, 1974.
D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Reading, MA: Addison-Wesley Publishing Company, 1989.
A. Witkin, D. Terzopoulous, and M. Kass, ‘Signal Matching through Scale Space,’ Intl. J. Comput. Vis., vol. 1, 1987, pp.133–144.
D. Kraft, A Software Package for Sequential Quadratic Programming, Technical Report DFVLR-FB 88–28, DFVLR, Obersfaffeuhofen, Germany, 1988.
T.J. Broida and R. Chellappa, ‘Estimation of Object Motion Parameters from Noisy Images,’ IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-8, 1986, pp. 90–99.
L. Matthies, R. Szeliski, and T. Kanade, ‘Incremental Estimation of Depth Maps from Image Sequences,’ IEEE Second International Conference on Computer Vision, Tarpon Springs, Fl, Dec. 5–8, 1988.
J.G. Balchen, F. Dessen, and G. Skofteland, ‘Sensor Integration Using State Estimators,’ NATO Advanced Research Workshop on Traditional and Non-Traditional Robotic Sensors, Maratea, Italy, August 28–September 2, 1989.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Vemuri, B.C., Skofteland, G. Invariant surface and motion estimation from sparse range data. J Math Imaging Vis 1, 43–64 (1992). https://doi.org/10.1007/BF00135224
Issue Date:
DOI: https://doi.org/10.1007/BF00135224