Skip to main content
SpringerLink
Log in

The 3D bin packing problem for multiple boxes and irregular items based on deep Q-network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Irregular packing in e-commerce warehouses is a special case of a three-dimensional box packing problem (3DBPP). It is necessary to select the type and quantity of boxes and determine the location and orientation of the items to maximize the use of the loading space. In this paper, a spatial particle model of the 3DBPP for multiple boxes and irregular items is constructed using the three-dimensional (3D) point cloud and granulation method. In the model, the 3D point cloud is used to describe the shapes of irregular items, and the granulation method is used for the transformation from sparse and uneven point clouds to spatial particle convex hulls. In addition, we designed an empirical simulation algorithm (ESA) based on the combination of expert rules extracted in practical packing activities and empirical simulation, and an intelligent algorithm for 3DBPPs with irregular items combined with the framework of the deep Q network (DQN) algorithm in deep reinforcement learning. An instance generator is proposed based on industry data to generate realistic projects with representative attributes for the above two algorithms, such as types of boxes, irregular items, 3D spatial plane convex hulls, and spatially granular data. The numerical results show that the ESA can quickly obtain a high-quality packing scheme, and the intelligent DQN packing algorithm in deep reinforcement learning can avoid the limitation of expert rules and achieve a better scheme with a certain time for the training process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

References

  1. Ojha A, Agarwal M, Singhal A, Sarkar C, Ghosh S, Sinha R (2021) A generalized algorithm and framework for online 3-dimensional bin packing in an automated sorting center. In: 2021 Seventh Indian Control Conference (ICC). IEEE, pp 135–140

  2. Wu H, Leung SC, Si Y-W, Zhang D, Lin A (2017) Three-stage heuristic algorithm for three-dimensional irregular packing problem. Appl Math Model 41:431–444

    Article  MathSciNet  MATH  Google Scholar 

  3. Wang F, Hauser K (2019) Stable bin packing of non-convex 3D objects with a robot manipulator. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE, pp 8698–8704

  4. Harrath Y (2022) A three-stage layer-based heuristic to solve the 3D bin-packing problem under balancing constraint. J King Saud Univ - Comput Inf Sci 34(8):6425–6431

    Google Scholar 

  5. Herrera-Franklin J, Rosete A, García-Borroto M (2021) A fuzzy approach for the variable cost and size bin packing problem allowing incomplete packing. Intel Artif 24(67):71–89

    Article  Google Scholar 

  6. Elhedhli S, Gzara F, Yildiz B (2019) Three-dimensional bin packing and mixed-case palletization. INFORMS J Comput 1(4):323–352

    MathSciNet  Google Scholar 

  7. Dell’Amico M, Furini F, Iori M (2020) A branch-and-price algorithm for the temporal bin packing problem. Comput Oper Res 114:104825

  8. Sato AK, Martins TC, Gomes AM, Tsuzuki MSG (2019) Raster penetration map applied to the irregular packing problem. Eur J Oper Res 279(2):657–671

    Article  MathSciNet  MATH  Google Scholar 

  9. Ma X, Liao Z (2019) Review and analysis of research on bin packing problem in logistics field—scientometric analysis based on web of science database (1989\(\sim \)2018). Sci Technol Dev 7:675–683

    Google Scholar 

  10. Gzara F, Elhedhli S, Yildiz BC (2020) The pallet loading problem: Three-dimensional bin packing with practical constraints. Eur J Oper Res 287(3):1062–1074

    Article  MathSciNet  MATH  Google Scholar 

  11. Mahvash B, Awasthi A, Chauhan S (2018) A column generation-based heuristic for the three-dimensional bin packing problem with rotation. J Oper Res Soc 69(1):78–90

    Article  Google Scholar 

  12. Wei L, Zhang Z, Zhang D, Leung SC (2018) A simulated annealing algorithm for the capacitated vehicle routing problem with two-dimensional loading constraints. Eur J Oper Res 265(3):843–859

    Article  MathSciNet  MATH  Google Scholar 

  13. Wei L, Wang Y, Cheng H, Huang J (2019) An open space based heuristic for the 2D strip packing problem with unloading constraints. Appl Math Model 70:67–81

    Article  MathSciNet  MATH  Google Scholar 

  14. Gupta N, Gupta K, Gupta D, Juneja S, Turabieh H, Dhiman G, Kautish S, Viriyasitavat W (2022) Enhanced virtualization-based dynamic bin-packing optimized energy management solution for heterogeneous clouds. Math Probl Eng 2022

  15. Polyakovskiy S, M’Hallah R (2022) A lookahead matheuristic for the unweighed variable-sized two-dimensional bin packing problem. Eur J Oper Res 299(1):104–117

    Article  MathSciNet  MATH  Google Scholar 

  16. Alonso MT, Alvarez-Valdés R, Parreño F (2020) A grasp algorithm for multi container loading problems with practical constraints. 4OR 18(1):49–72

  17. Leao AA, Toledo FM, Oliveira JF, Carravilla MA, Alvarez-Valdés R (2020) Irregular packing problems: A review of mathematical models. Eur J Oper Res 282(3):803–822

    Article  MathSciNet  MATH  Google Scholar 

  18. Sato AK, Bauab GES, de Castro Martins T, Tsuzuki MDSG, Gomes AM (2018) A study in pairwise clustering for bi-dimensional irregular strip packing using the dotted board model. IFAC-PapersOnLine 51(11):284–289

    Article  Google Scholar 

  19. Cherri LH, Carravilla MA, Ribeiro C, Toledo FMB (2019) Optimality in nesting problems: new constraint programming models and a new global constraint for non-overlap. Oper Res Perspect 6:1

    MathSciNet  Google Scholar 

  20. Guo B, Hu J, Wu F, Peng Q (2020) Automatic layout of 2D free-form shapes based on geometric similarity feature searching and fuzzy matching. J Manuf Syst 56:37–49

    Article  Google Scholar 

  21. Abeysooriya RP, Bennell JA, Martinez-Sykora A (2018) Jostle heuristics for the 2D-irregular shapes bin packing problems with free rotation. Int J Prod Econ 195:12–26

    Article  Google Scholar 

  22. Zhao Y, Rausch C, Haas C (2021) Optimizing 3D irregular object packing from 3D scans using metaheuristics. Adv Eng Inform 47:101234

  23. Lamas-Fernandez C, Bennell JA, Martinez-Sykora A (2022) Voxel-based solution approaches to the three-dimensional irregular packing problem. Oper Res

  24. Romanova T, Pankratov A, Litvinchev I, Dubinskyi V, Infante L (2022) Sparse layout of irregular 3D clusters. J Oper Res Soc 1–11

  25. Luo Q, Rao Y, Peng D (2022) GA and GWO algorithm for the special bin packing problem encountered in field of aircraft arrangement. Appl Soft Comput 114:108060

  26. Litvinchev I, Pankratov A, Romanova T (2019) 3D irregular packing in an optimized cuboid container. IFAC-PapersOnLine 52(13):2014–2019

    Article  Google Scholar 

  27. Romanova T, Bennell J, Stoyan Y, Pankratov A (2018) Packing of concave polyhedra with continuous rotations using nonlinear optimisation. Eur J Oper Res 268(1):37–53

    Article  MathSciNet  MATH  Google Scholar 

  28. Harrath Y (2022) A three-stage layer-based heuristic to solve the 3D bin-packing problem under balancing constraint. J King Saud Univ - Comput Inf Sci 34(8):6425–6431

  29. Mungwattana A, Piyachayawat T, Janssens GK (2022) A two-step evolutionary algorithm for the distributor’s pallet loading problem with multi-size pallets. Flex Serv Manuf J 1–20

  30. Shuai W, Gao Y, Wu P, Cui G, Zhuang Q, Chen R, Chen X (2023) Compliant-based robotic 3D bin packing with unavoidable uncertainties. IET Control Theory Appl

  31. Zhao H, Zhu C, Xu X, Huang H, Xu K (2022) Learning practically feasible policies for online 3D bin packing. Sci China Inf Sci 65(1):112105

  32. Stoyan Y, Gil N, Scheithauer G, Pankratov A, Magdalina I (2005) Packing of convex polytopes into a parallelepiped. Optimization 54(2):215–235

    Article  MathSciNet  MATH  Google Scholar 

  33. Egeblad J, Nielsen BK, Brazil M (2009) Translational packing of arbitrary polytopes. Comput Geom 42(4):269–288

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhao H, Zhu C, Xu X, Huang H, Xu K (2022) Learning practically feasible policies for online 3D bin packing. Sci China Inf Sci 65(1):1–17

    Article  MathSciNet  Google Scholar 

  35. Zhang D, Peng Y, Leung SC (2012) A heuristic block-loading algorithm based on multi-layer search for the container loading problem. Comput Oper Res 39(10):2267–2276

    Article  Google Scholar 

Download references

Funding

This study is supported by the National Social Science Fund of China, “Research on the Enhancement of Logistics Service Quality and Low-carbon Governance Mechanisms” (21FGLB046), and the Beijing Wuzi University Youth Research Fund, “Research on Intelligent E-commerce Unmanned Warehouse Packing and Loading Optimization Strategy and Green Recycling Mode” (2023XJQN14).

Author information

Author notes

  1. Huwei Liu, Li Zhou, and Jianglong Yang contributed equally to this work.

Authors and Affiliations

  1. School of Management and Engineering, Capital University of Economics and Business, 121 Zhangjialukou, Huaxiang, Fengtai District, 100070, Beijing, China

    Huwei Liu

  2. School of Information, Beijing Wuzi University, #1 Fuhe Street, Tongzhou Distric, 101149, Beijing, China

    Li Zhou, Jianglong Yang & Junhui Zhao

Authors
  1. Huwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianglong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junhui Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianglong Yang.

Ethics declarations

Conflict of interest

On behalf of all the authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Instance data of box sizes

The instance data of box sizes are shown in Table 8.

Table 8 Instance data of box sizes
Full size table
Table 9 3D point cloud instance data for 7 irregular items
Full size table
Table 10 3D point cloud instance data for 10 irregular items
Full size table
Table 11 3D point cloud instance data for 23 irregular items
Full size table

Appendix B: 3D point cloud instance data for irregular items

The 3D point cloud instance data for irregular items are shown in Tables 9 to 11.

Fig. 32
figure 32

3D graph of 7 irregular items

Full size image
Fig. 33
figure 33

3D graph of 10 irregular items

Full size image
Fig. 34
figure 34

3D graph of 23 irregular items

Full size image

Appendix C: 3D graphs of irregular items

The 3D graphs for irregular items are shown in Figs. 32, 33, and 34.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Zhou, L., Yang, J. et al. The 3D bin packing problem for multiple boxes and irregular items based on deep Q-network. Appl Intell 53, 23398–23425 (2023). https://doi.org/10.1007/s10489-023-04604-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04604-6

Keywords

Search

Navigation