The Visual Computer

, Volume 29, Issue 1, pp 7–26 | Cite as

Real-time marker prediction and CoR estimation in optical motion capture

Original Article

Abstract

Optical motion capture systems suffer from marker occlusions resulting in loss of useful information. This paper addresses the problem of real-time joint localisation of legged skeletons in the presence of such missing data. The data is assumed to be labelled 3d marker positions from a motion capture system. An integrated framework is presented which predicts the occluded marker positions using a Variable Turn Model within an Unscented Kalman filter. Inferred information from neighbouring markers is used as observation states; these constraints are efficient, simple, and real-time implementable. This work also takes advantage of the common case that missing markers are still visible to a single camera, by combining predictions with under-determined positions, resulting in more accurate predictions. An Inverse Kinematics technique is then applied ensuring that the bone lengths remain constant over time; the system can thereby maintain a continuous data-flow. The marker and Centre of Rotation (CoR) positions can be calculated with high accuracy even in cases where markers are occluded for a long period of time. Our methodology is tested against some of the most popular methods for marker prediction and the results confirm that our approach outperforms these methods in estimating both marker and CoR positions.

Keywords

Computer vision Filtering Marker prediction Joint localisation Motion capture Inverse kinematics 

Supplementary material

(AVI 21.9 MB)

References

  1. 1.
    Aristidou, A., Cameron, J., Lasenby, J.: Predicting missing markers to drive real-time centre of rotation estimation. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 238–247 (2008) CrossRefGoogle Scholar
  2. 2.
    Aristidou, A., Lasenby, J.: Inverse kinematics: a review of existing techniques and introduction of a new fast iterative solver. Tech. Rep. F-INFENG/TR. 632, CUED (2009) Google Scholar
  3. 3.
    Aristidou, A., Lasenby, J.: FABRIK: a fast, iterative solver for the inverse kinematics problem. Graphical Models 73(5), 243–260 (2011) CrossRefGoogle Scholar
  4. 4.
    Aristidou, A., Lasenby, J., Cameron, J.: Methods for real-time restoration and estimation in optical motion capture. Tech. Rep. F-INFENG/TR. 619, CUED (2009) Google Scholar
  5. 5.
    Asseo, S.J., Ardila, R.J.: Sensor independent target state estimator design and evaluation. In: Proceedings of the National Aerospace and Electronics Conference (NAECON), pp. 916–924 (1982) Google Scholar
  6. 6.
    Backer, A.S.: Estimating missing motion capture data with accelerometers. MPhil thesis, Cambridge University Engineering Department. Cambridge, UK (August 2009) Google Scholar
  7. 7.
    Baillieul, J.: Kinematic programming alternatives for redundant manipulators. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 722–728 (1985) Google Scholar
  8. 8.
    Balestrino, A., De Maria, G., Sciavicco, L.: Robust control of robotic manipulators. In: Proceedings of the 9th IFAC World Congress, vol. 5, pp. 2435–2440 (1984) Google Scholar
  9. 9.
    Broeren, J., Sunnerhagen, K.S., Rydmark, M.: A kinematic analysis of a haptic handheld stylus in a virtual environment: a study in healthy subjects. Journal of NeuroEngineering and Rehabilitation 4, 13 (2007) CrossRefGoogle Scholar
  10. 10.
    Brown, J., Latombe, J.C., Montgomery, K.: Real-time knot-tying simulation. The Visual Computer 20(2), 165–179 (2004) CrossRefGoogle Scholar
  11. 11.
    Buss, S.R., Kim, J.S.: Selectively damped least squares for inverse kinematics. Journal of Graphics Tools 10(3), 37–49 (2005) Google Scholar
  12. 12.
    Cameron, J., Lasenby, J.: A real-time sequential algorithm for human joint localization. In: ACM SIGGRAPH Posters, p. 107. ACM, New York (2005) CrossRefGoogle Scholar
  13. 13.
    Chai, J., Hodgins, J.K.: Performance animation from low-dimensional control signals. ACM Transactions on Graphics (TOG) 24(3), 686–696 (2005) CrossRefGoogle Scholar
  14. 14.
    Chang, L.Y., Pollard, N.: Constrained least-squares optimization for robust estimation of center of rotation. Journal of Biomechanics 40(1), 1392–1400 (2007) CrossRefGoogle Scholar
  15. 15.
    Courty, N., Arnaud, E.: Inverse kinematics using sequential Monte Carlo methods. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 1–10 (2008) CrossRefGoogle Scholar
  16. 16.
    Courty, N., Cuzol, A.: Conditional stochastic simulation for character animation. Computer Animation and Virtual Worlds—CASA’ 2010 21(3–4), 443–452 (2010) Google Scholar
  17. 17.
    Der, K.G., Sumner, R.W., Popović, J.: Inverse kinematics for reduced deformable models. In: ACM SIGGRAPH Papers, pp. 1174–1179. ACM, New York (2006) CrossRefGoogle Scholar
  18. 18.
    Dessai, S.S., Hornung, A., Kobbelt, L.: Automatic data normalization and parameterization for optical motion tracking. J. Virtual Real. Broadcast. 3(3) (2006) Google Scholar
  19. 19.
    Doran, C., Lasenby, A.: Geometric Algebra for Physicists. Cambridge University Press, Cambridge (2003) MATHGoogle Scholar
  20. 20.
    Dorfmüller-Ulhaas, K.: Robust optical user motion tracking using a Kalman filter. Tech. Rep. TR-2003-6, Institut fuer Informatik, Universitatsstr. 2, 86159 Augsburg (2003) Google Scholar
  21. 21.
    Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001) MATHGoogle Scholar
  22. 22.
    Ehrig, R.M., Taylor, W.R., Duda, G.N., Heller, M.O.: A survey of formal methods for determining the centre of rotation of ball joints. Journal of Biomechanics 39(15), 2798–2809 (2006) CrossRefGoogle Scholar
  23. 23.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley-Interscience, New York (1987) MATHGoogle Scholar
  24. 24.
    Gamage, S.S.H.U., Lasenby, J.: New least squares solutions for estimating the average centre of rotation and the axis of rotation. Journal of Biomechanics 35(1), 87–93 (2002) CrossRefGoogle Scholar
  25. 25.
    Grochow, K., Martin, S.L., Hertzmann, A., Popović, Z.: Style-based inverse kinematics. In: ACM Transactions on Graphics (TOG), pp. 522–531. ACM, New York (2004) Google Scholar
  26. 26.
    Halvorsen, K.: Bias compensated least squares estimate of the center of rotation. Journal of Biomechanics 36, 999–1008 (2003) CrossRefGoogle Scholar
  27. 27.
    Halvorsen, K., Lesser, M., Lundberg, A.: A new method for estimating the axis of rotation and the center of rotation. Journal of Biomechanics 32, 1221–1227 (1999) CrossRefGoogle Scholar
  28. 28.
    Hashiguchi, J., Nivomiya, H., Tanaka, H., Nakamura, M., Nobuhara, K.: Biomechanical analysis of a golf swing using motion capture system. In: Proceedings of Annual Meeting of Japanese Society for Orthopaedic Biomechanics, vol. 27, pp. 325–330 (2006) Google Scholar
  29. 29.
    Hecker, C., Raabe, B., Enslow, R.W., Deweese, J., Maynard, J., van Prooijen, K.: Real-time motion retargeting to highly varied user-created morphologies. ACM Transactions on Graphics (TOG) 27(3), 1–11 (2008) CrossRefGoogle Scholar
  30. 30.
    Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Skeleton-based motion capture for robust reconstruction of human motion. In: Proceedings of the IEEE Computer Animation (CA’00), pp. 77–86 (2000) CrossRefGoogle Scholar
  31. 31.
    Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Using skeleton-based tracking to increase the reliability of optical motion capture. Human Movement Science Journal 20(3), 313–341 (2001) CrossRefGoogle Scholar
  32. 32.
    Hestenes, D., Sobczyk, G.: Clifford Algebra to Geometric Calculus: A Unified Language for Mathematics and Physics. Reidel, Dordrecht (1984) CrossRefMATHGoogle Scholar
  33. 33.
    Holzreiter, S.: Calculation of the instantaneous centre of rotation for a rigid body. Journal of Biomechanics 24(7), 643–647 (1991) CrossRefGoogle Scholar
  34. 34.
    Horn, B.: Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 629–642 (1987) CrossRefGoogle Scholar
  35. 35.
    Hornung, A., Sar-Dessai, S.: Self-calibrating optical motion tracking for articulated bodies. In: Proceedings of the IEEE Conference on Virtual Reality, VR’05, pp. 75–82. IEEE Computer Society, Washington (2005) Google Scholar
  36. 36.
    Hsu, E., Gentry, S., Popović, J.: Example-based control of human motion. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’04), pp. 69–77. Eurographics Association, Grenoble (2004) CrossRefGoogle Scholar
  37. 37.
    Ishigaki, S., White, T., Zordan, V.B., Liu, C.K.: Performance-based control interface for character animation. ACM Transaction on Graphics (TOG) 28(3), 1–8 (2009) CrossRefGoogle Scholar
  38. 38.
    Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Academic Press, San Diego (1970) MATHGoogle Scholar
  39. 39.
    Julier, S.J., Uhlmann, J.K.: A new extension of the kalman filter to nonlinear systems. In: Proceedings of the International Symposium on Aerospace/Defense Sensing, Simululation and Controls, vol. Acquisition, Tracking and Pointing XI, Florida, USA, pp. 182–193 (1997) Google Scholar
  40. 40.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME, J. Basic Eng., 35–45 (1960) Google Scholar
  41. 41.
    Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Dynammo: mining and summarization of coevolving sequences with missing values. In: Proceedings of the 15th International Conference on Knowledge Discovery and Data Mining, pp. 507–516. ACM, Paris (2009) CrossRefGoogle Scholar
  42. 42.
    Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Bolero: a principled technique for including bone length constraints in motion capture occlusion filling. In: Proceedings of the ACM Symposium on Computer Animation, Madrid, Spain (2010) Google Scholar
  43. 43.
    Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. Part I: Dynamic models. IEEE Transactions on Aerospace and Electronic Systems 39(4), 1333–1364 (2003) CrossRefGoogle Scholar
  44. 44.
    Lin, M.C., Gottschalk, S.: Collision detection between geometric models: a survey. In: Proceedings of IMA Conference on Mathematics of Surfaces, pp. 37–56 (1998) Google Scholar
  45. 45.
    Liu, G., McMillan, L.: Estimation of missing markers in human motion capture. The Visual Computer 22(9–11), 721–728 (2006) CrossRefGoogle Scholar
  46. 46.
    Liu, G., Zhang, J., Wang, W., McMillan, L.: Human motion estimation from a reduced marker set. In: I3D’06: Proceedings of the Symposium on Interactive 3D Graphics and Games, pp. 35–42. ACM, New York (2006) CrossRefGoogle Scholar
  47. 47.
    Maidi, M., Ababsa, F., Mallem, M.: Handling occlusions for robust augmented reality systems. EURASIP J. Image Video Process. 2010 (2010) Google Scholar
  48. 48.
    Menache, A.: Understanding Motion Capture for Computer Animation and Video Games. Morgan Kaufmann, San Francisco (1999) Google Scholar
  49. 49.
    Merwe, R.V.D., Doucet, A., Freitas, N.D., Wan, E.: The unscented particle filter. Tech. Rep. F-INFENG/TR. 380, CUED (2000) Google Scholar
  50. 50.
    Nakamura, Y., Hanafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. Trans. ASME Journal of Dynamic Systems, Measurement, and Control 108(3), 163–171 (1986) CrossRefMATHGoogle Scholar
  51. 51.
    O’Brien, J.F., Bodenheimer, R.E., Brostow, G.J., Hodgins, J.K.: Automatic joint parameter estimation from magnetic motion capture data. In: Proceedings of Graphic Interface, pp. 53–60 (2000) Google Scholar
  52. 52.
    Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Transaction on Graphics (TOG) 25(3), 881–889 (2006) CrossRefGoogle Scholar
  53. 53.
    Pechev, A.N.: Inverse kinematics without matrix invertion. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’08), Pasadena, CA, USA, pp. 2005–2012 (2008) Google Scholar
  54. 54.
    Phasespace inc: Optical motion capture systems. http://www.phasespace.com
  55. 55.
    Piazza, T., Lundström, J., Hugestrand, A., Kunz, A., Fjeld, M.: Towards solving the missing marker problem in realtime motion capture. In: Proceedings of the International Design Engineering Technical Conference (2009) Google Scholar
  56. 56.
    Rhijn, A.V., Mulder, J.D.: Optical tracking and automatic model estimation of composite interaction devices. In: IEEE Virtual Reality Conference, pp. 135–142 (2006) Google Scholar
  57. 57.
    Ringer, M., Lasenby, J.: A procedure for automatically estimating model parameters in optical motion capture. In: Proceedings of the British Machine Vision Conference, pp. 747–756 (2002) Google Scholar
  58. 58.
    Rose, C., Cohen, M., Bodenheimer, B.: Verbs and adverbs: multidimensional motion interpolation. IEEE Computer Graphics and Applications 18(5), 32–40 (1998) CrossRefGoogle Scholar
  59. 59.
    Silaghi, M.C., Plänkers, R., Boulic, R., Fua, P., Thalmann, D.: Local and global skeleton fitting techniques for optical motion capture. In: Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, London, UK, pp. 26–40 (1998) CrossRefGoogle Scholar
  60. 60.
    Singer, R.A.: Estimating optimal tracking filter performance for manned maneuvering targets. IEEE Transactions on Aerospace and Electronic Systems 6(1), 473–483 (1970) CrossRefGoogle Scholar
  61. 61.
    Singer, R.A., Behnke, K.W.: Real-time tracking filter evaluation and selection for tactical applications. IEEE Transactions on Aerospace and Electronic Systems 7(1), 100–110 (1971) CrossRefGoogle Scholar
  62. 62.
    Sumner, R.W., Zwicker, M., Gotsman, C., Popović, J.: Mesh-based inverse kinematics. ACM Transactions on Graphics (TOG) 24(3), 488–495 (2005) CrossRefGoogle Scholar
  63. 63.
    Tak, S., Ko, H.S.: A physically-based motion retargeting filter. ACM Transactions on Graphics (TOG) 24(1), 98–117 (2005) CrossRefGoogle Scholar
  64. 64.
    Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, pp. 1345–1352. MIT Press, Cambridge (2007) Google Scholar
  65. 65.
    Wampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics 16(1), 93–101 (1986) CrossRefMATHGoogle Scholar
  66. 66.
    Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 283–298 (2008) CrossRefGoogle Scholar
  67. 67.
    Wang, L.C.T., Chen, C.C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Transactions on Robotics and Automation 7(4), 489–499 (1991) CrossRefGoogle Scholar
  68. 68.
    Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., Colucci, D.: The HiBall tracker: high-performance wide-area tracking for virtual and augmented environments. In: Virtual Reality Software and Technology, VRST, pp. 1–10. ACM, New York (1999) Google Scholar
  69. 69.
    Wiley, D.J., Hahn, J.K.: Interpolation synthesis of articulated figure motion. IEEE Computer Graphics and Applications 17(6), 39–45 (1997) CrossRefGoogle Scholar
  70. 70.
    Wolovich, W.A., Elliott, H.: A computational technique for inverse kinematics. IEEE Conference on Decision and Control 23, 1359–1363 (1984) CrossRefGoogle Scholar
  71. 71.
    Yu, Q., Li, Q., Deng, Z.: Online motion capture marker labeling for multiple interacting articulated targets. Computer Graphics Forum (Proceedings of Eurographics) 27(7), 477–483 (2007) CrossRefGoogle Scholar
  72. 72.
    Zordan, V.B., Van Der Horst, N.C.: Mapping optical motion capture data to skeletal motion using a physical model. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’03), pp. 245–250. Eurographics Association, San Diego (2003) Google Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.Department of EngineeringUniversity of CambridgeCambridgeUK

Personalised recommendations