Development of an Object Recognition and Location System Using the Microsoft KinectTM Sensor

  • Jose Figueroa
  • Luis Contreras
  • Abel Pacheco
  • Jesus Savage
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7416)


This paper presents the development of an object recognition and location system using the Microsoft KinectTM, an off-the-shelf sensor for videogames console Microsoft Xbox 360TM which is formed by a color camera and depth sensor. This sensor is capable of capturing color images and depth information from a scene. This vision system uses a) data fusion of both color camera and depth sensor to segment objects by distance; b) scale-invariant features to characterize and recognize objects; and c) camera’s internal parameters combined with depth information to locate objects relative to the camera point of view. The system will be used along with a robotic arm to grab objects.


Keywords: Feature extraction Scale Invariant Feature Machine vision Object detection Pattern recognition 


  1. 1.
    Adafruit: The Open Kinect project - the ok prize - get $3,000 bounty for Kinect for Xbox 360 open source drivers (November 2010),
  2. 2.
    Adafruit: We have a winner - Open Kinect driver(s) released - winner will use $3k for more hacking - plus an additional $2k goes to the eff! (November 2010),
  3. 3.
    Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Proc. IEEE Conf. Comp. Vision Patt. Recog., pp. 1000–1006 (1997)Google Scholar
  4. 4.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975), zbMATHCrossRefGoogle Scholar
  5. 5.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In: Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, pp. 726–740. Morgan Kaufmann Publishers Inc., San Francisco (1987), Google Scholar
  6. 6.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  7. 7.
    Marr, D.: Vision: a computational investigation into the human representation and processing of visual information / David Marr. W.H. Freeman, San Francisco (1982)Google Scholar
  8. 8.
    PrimeSense: Primesense, reference design,
  9. 9.
    Zhang, R., Tsi, P.S., Cryer, J.E., Shah, M.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the 7th International Conference on Computer Vision, pp. 666–673 (September 1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jose Figueroa
    • 1
  • Luis Contreras
    • 1
  • Abel Pacheco
    • 1
  • Jesus Savage
    • 1
  1. 1.Biorobotics Laboratory, Department of Electrical EngineeringUniversidad Nacional Autonoma de Mexico, UNAMMexico

Personalised recommendations