Bin Picking Using Manifold Learning

  • Ashutosh KumarEmail author
  • Santanu Chaudhury
  • J. B. Srivastava
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 460)


Bin picking using vision based sensors requires accurate estimation of location and pose of the object for positioning the end effector of the robotic arm. The computational burden and complexity depends upon the parametric model adopted for the task. Learning based techniques to implement the scheme using low dimensional manifolds offer computationally more efficient alternatives. In this paper we have employed Locally Linear Embedding (LLE) and Deep Learning (with auto encoders) for manifold learning in the visual domain as well as for the parameters of robotic manipulator for visual servoing. Images of clusters of cylindrical pellets were used as the training data set in the visual domain. Corresponding parameters of the six degrees of freedom robot for picking designated cylindrical pellet formed the training dataset in the robotic configuration space. The correspondence between the weight coefficients of LLE manifold in the visual domain and robotic domain is established through regression. Autoencoders in conjunction with feed forward neural networks were used for learning of correspondence between the high dimensional visual space and low dimensional configuration space. We have compared the results of the two implementations for the same dataset and found that manifold learning using auto encoders resulted in better performance. The eye-in-hand configuration used with KUKA KR5 robotic arm and Basler camera offers a potentially effective and efficient solution to the bin picking problem through learning based visual servoing.


Computer vision Manifold Locally Linear Embedding (LLE) Visual servoing Auto-encoder 



We want to express our gratitude to Program for Autonomous Robotics at I.I.T. Delhi for allowing us to use their laboratory facilities. We would also like to thank Mr Manoj Sharma, Research Scholar, Department of Electrical Engg and Mr Riby Abraham, Research Scholar, Department of Mechanical Engg for their help in our work.


  1. 1.
    Ghita Ovidiu and Whelan Paul F. 2008. A Systems Engineering Approach to Robotic Bin Picking. Stereo Vision, Book edited by: Dr. Asim Bhatti, pp. 372. Google Scholar
  2. 2.
    Kelley, B.; Birk, J.R.; Martins, H. & Tella R. 1982. A robot system which acquires cylindrical workpieces from bins, IEEE Trans. Syst. Man Cybern., vol. 12, no. 2, pp. 204–213.Google Scholar
  3. 3.
    Faugeras, O.D. & Hebert, M. 1986. The representation, recognition and locating of 3-D objects, Intl. J. Robotics Res., vol. 5, no. 3, pp. 27–52.Google Scholar
  4. 4.
    Edwards, J. 1996. An active, appearance-based approach to the pose estimation of complex objects, Proc. of the IEEE Intelligent Robots and Systems Conference, Osaka, Japan, pp. 1458–1465.Google Scholar
  5. 5.
    Murase, H. & Nayar, S.K. 1995. Visual learning and recognition of 3-D objects from appearance, Intl. Journal of Computer Vision, vol. 14, pp. 5–24.Google Scholar
  6. 6.
    Ghita O. & Whelan, P.F. 2003. A bin picking system based on depth from defocus, Machine Vision and Applications, vol. 13, no. 4, pp. 234–244.Google Scholar
  7. 7.
    Mittrapiyanuruk, P.; DeSouza, G.N. & Kak, A. 2004. Calculating the 3D-pose of rigid objects using active appearance models, Intl. Conference in Robotics and Automation, New Orleans, USA.Google Scholar
  8. 8.
    Ghita, O.; Whelan, P.F.; Vernon D. & Mallon J. 2007. Pose estimation for objects with planar surfaces using eigen image and range data analysis, Machine Vision and Applications, vol. 18, no. 6, pp. 355–365.Google Scholar
  9. 9.
    Saul Lawrence K and Roweis Sam T. 2000, An Introduction to Locally Linear Embedding,
  10. 10.
    Deep Learning, An MIT Press book in preparation Yoshua Bengio, Ian Goodfellow and Aaron Courville,,2015.
  11. 11.
    Deep Learning Tool Box, Prediction as a candidate for learning deep hierarchical models of data, Rasmus Berg Palm,
  12. 12.
    Léonard, Simon, and Martin Jägersand. “Learning based visual servoing.” Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on. Vol. 1. IEEE, 2004.Google Scholar
  13. 13.
    Rigas Kouskouridas, Angleo Amanatiadis and Antonios Gasteratos, “Pose Manifolds for Efficient Visual Servoing”,
  14. 14.
    Chih-Chung Chang and Chih-Jen Lin, LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1–27:27, 2011. Software available at
  15. 15.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., & Manzagol, P. A. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research11, 3371–3408.Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  • Ashutosh Kumar
    • 1
    Email author
  • Santanu Chaudhury
    • 1
  • J. B. Srivastava
    • 2
  1. 1.Department of Electrical EngineeringI.I.TDelhiIndia
  2. 2.Department of MathematicsI.I.TDelhiIndia

Personalised recommendations