Skip to main content
Book cover

Asian Conference on Computer Vision

ACCV 2016: Computer Vision – ACCV 2016 pp 472–486Cite as

Precise Measurement of Cargo Boxes for Gantry Robot Palletization in Large Scale Workspaces Using Low-Cost RGB-D Sensors

Part of the Lecture Notes in Computer Science book series (LNIP,volume 10114)

Abstract

This paper presents a novel algorithm for extracting the pose and dimensions of cargo boxes in a large measurement space of a robotic gantry, with sub-centimetre accuracy using multiple low cost RGB-D Kinect sensors. This information is used by a bin-packing and path-planning software to build up a pallet. The robotic gantry workspaces can be up to 10 m in all dimensions, and the cameras cannot be placed top-down since the components of the gantry actuate within this space. This presents a challenge as occlusion and sensor noise is more likely.

This paper presents the system integration components on how point cloud information is extracted from multiple cameras and fused in real-time, how primitives and contours are extracted and corrected using RGB image features, and how cargo parameters from the cluttered cloud are extracted and optimized using graph based segmentation and particle filter based techniques. This is done with sub-centimetre accuracy irrespective of occlusion or noise from cameras at such camera placements and range to cargo.

Keywords

  • Point Cloud
  • Particle Filter
  • Depth Image
  • Move Little Square
  • Bilateral Filter

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-54190-7_29
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-54190-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.

References

  1. Har-peled, S.: A practical approach for computing the diameter of a point set. In: SCG 2001 (2001)

    Google Scholar 

  2. Boeing: boeing pallets @ONLINE (2012). http://www.boeing.com/resources/boeingdotcom/company/about_bca/pdf/CargoPalletsContainers.pdf

  3. Viegas, J.P.L., Vieira, S.M., Sousa, J.M.C., Henriques, E.M.P.: Metaheuristics for the 3D bin packing problem in the steel industry. In: Proceedings of the 2014 IEEE Congress on Evolutionary Computation, CEC 2014, pp. 338–343 (2014)

    Google Scholar 

  4. Robotics, U.: Universal robotics @ONLINE (2015). http://www.universalrobotics.com/applications/

  5. Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: efficient and robust 3D object recognition (2012)

    Google Scholar 

  6. Holz, D., Behnke, S., Holz, D., Topalidou-kyniazopoulou, A.: Real-time object detection, localization and verification for fast robotic depalletizing verification for fast robotic depalletizing. In: IROS (2015)

    Google Scholar 

  7. Lloyd, R., McCloskey, S.: Recognition of 3D package shapes for single camera metrology. In: 2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014, pp. 99–106 (2014)

    Google Scholar 

  8. Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., Silva, C.T.: Computing and rendering point set surfaces. IEEE Trans. Vis. Comput. Graphics 9, 3–15 (2003)

    CrossRef  Google Scholar 

  9. Richtsfeld, A., Morwald, T., Prankl, J., Zillich, M., Vincze, M.: Segmentation of unknown objects in indoor environments. In: IEEE International Conference on Intelligent Robots and Systems, pp. 4791–4796 (2012)

    Google Scholar 

  10. Wu, K., Ranasinghe, R., Dissanayake, G.: A fast pipeline for textured object recognition in clutter using an RGB-D sensor. In: 2014 13th International Conference on Control Automation Robotics and Vision, ICARCV 2014, pp. 1650–1655 (2014)

    Google Scholar 

  11. Somani, N., Cai, C., Perzylo, A., Rickert, M., Knoll, A.: Object recognition using constraints from primitive shape matching. In: 10th International Symposium on Visual Computing (ISVC 2014) (2014)

    Google Scholar 

  12. Anwer, A., Baig, A., Nawaz, R.: Calculating real world object dimensions from Kinect RGB-D image using dynamic resolution. In: Proceedings of 2015 12th International Bhurban Conference on Applied Sciences and Technology, IBCAST 2015, pp. 198–203 (2015)

    Google Scholar 

  13. Aouada, D., Ottersten, B., Mirbach, B., Garcia, F., Solignac, T.: Real-time depth enhancement by fusion for RGB-D cameras. IET Comput. Vision 7, 335–345 (2013)

    CrossRef  Google Scholar 

  14. Wang, H., Zhang, W., Chen, Y., Chen, M., Yan, K.: Semantic decomposition and reconstruction of compound buildings with symmetric roofs from LiDAR data and aerial imagery. Remote Sens. 7, 13945–13974 (2015)

    CrossRef  Google Scholar 

  15. libfreenect2 @ONLINE (2013). https://github.com/OpenKinect/libfreenect2

  16. iai-kinect2 @ONLINE (2015). https://github.com/code-iai/iai_kinect2

  17. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)

    Google Scholar 

  18. Zhang, Z.: A flexible new technique for camera calibration (technical report). IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2002)

    CrossRef  Google Scholar 

  19. Bradski, G.: Opencv. Dr. Dobb’s journal of software tools (2000)

    Google Scholar 

  20. Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China (2011)

    Google Scholar 

  21. Merry, B., Gain, J., Marais, P.: Moving least-squares reconstruction of large models with GPUs. IEEE Trans. Vis. Comput. Graphics 20, 249–261 (2014)

    CrossRef  Google Scholar 

  22. Johnson, D.B.: Finding all the elementary circuits of a directed graph. 4, 77–84 (1975)

    Google Scholar 

  23. Fox, D., Burgard, W., Dellaert, F., Thrun, S.: Monte carlo localization: efficient position estimation for mobile robots dieter fox, wolfram burgard. In: 16th National Conference on Artificial Intelligence (AAAI99), pp. 343–349 (1999)

    Google Scholar 

Download references

Acknowledgement

This work is funded by the Civil Aviation Authority of Singapore (CAAS) under the Aviation Challenge 2 grant.

Author information

Authors and Affiliations

  1. TUM CREATE, 1 CREATE Way, #10-02 CREATE Tower, Singapore, 138602, Singapore

    Yaadhav Raaj & Suraj Nair

  2. Technische Universität München (TUM), Institüt für Informatik, Robotics and Embedded System, Munich, Germany

    Alois Knoll

Authors
  1. Yaadhav Raaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suraj Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alois Knoll
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaadhav Raaj .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo, Tokyo, Japan

    Yoichi Sato

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 14260 KB)

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Raaj, Y., Nair, S., Knoll, A. (2017). Precise Measurement of Cargo Boxes for Gantry Robot Palletization in Large Scale Workspaces Using Low-Cost RGB-D Sensors. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10114. Springer, Cham. https://doi.org/10.1007/978-3-319-54190-7_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54190-7_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54189-1

  • Online ISBN: 978-3-319-54190-7

  • eBook Packages: Computer ScienceComputer Science (R0)