OP-MR: the implementation of order picking based on mixed reality in a smart warehouse


This paper presents a mixed-reality (MR) application called order picking with mixed reality (OP-MR) for the order-picking activities in a smart warehouse. OP-MR is a set of applications operated by an administrator through a computer server and by the staff using the HoloLens MR device. OP-MR is built to reduce the operational time of an order-picking activity by providing the shortest route to the staff. The HoloLens device displays the order-picking instructions through the MR window, renders virtual navigation, and virtually marks the positions of items. For determining the shortest distance for an order picking, the proposed OP-MR method combines two different algorithms, namely the Held–Karp algorithm in the server and A* algorithm in the client. The Held–Karp algorithm sorts the items in the pick-up list based on the nearest position. Next, the A* algorithm determines the shortest route to ensure that a user travels the shortest distance to pick all the items. To show the effectiveness of the proposed OP-MR method, OP-MR is implemented and experiments are performed. The experimental results show that OP-MR outperforms paper-based order-picking from the viewpoint of completing all the order picking.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. 1.


  2. 2.


  3. 3.


  4. 4.



  1. 1.

    Bartholdi, J.J., Hackman, S.T.: Warehouse and Distribution Science: Release 0.89. Supply Chain and Logistics Institute, Atlanta (2008)

    Google Scholar 

  2. 2.

    Frazelle, E., Frazelle, E.: World-Class Warehousing and Material Handling, vol. 1. McGraw-Hill, New York (2002)

    Google Scholar 

  3. 3.

    De Koster, M.: Warehousing in the Global Supply Chain, pp. 457–473. Springer, Berlin (2012)

    Google Scholar 

  4. 4.

    Reif, R., Günthner, W.A.: Pick-by-vision: augmented reality supported order picking. Vis. Comput. 25(5–7), 461 (2009)

    Article  Google Scholar 

  5. 5.

    Schwerdtfeger, B., Klinker, G.: Supporting order picking with augmented reality. In: Proceedings of the 7th IEEE/ACM international Symposium on Mixed and Augmented Reality IEEE Computer Society, pp. 91–94 (2008)

  6. 6.

    Gerstweiler, G., Platzer, K., Kaufmann, H.: Dargs: dynamic ar guiding system for indoor environments. Computers 7(1), 5 (2017)

    Article  Google Scholar 

  7. 7.

    Gerstweiler, G., Vonach, E., Kaufmann, H.: Hymotrack: a mobile ar navigation system for complex indoor environments. Sensors 16(1), 17 (2015)

    Article  Google Scholar 

  8. 8.

    Shmoys, D.B., Williamson, D.P.: Analyzing the Held-Karp tsp bound: a monotonicity property with application. Inf. Process. Lett. 35(6), 281 (1990)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Cui, S.G., Wang, H., Yang, L.: A simulation study of A-star algorithm for robot path planning. In 16th International Conference on Mechatronics Technology. pp. 506–510 (2012)

  10. 10.

    Klein, G.: Visual Tracking for Augmented Reality. Ph.D. thesis, University of Cambridge (2006)

  11. 11.

    Ishii, H.: Augmented reality. Fundamentals and nuclear related applications. Int. Electron. J. Nucl. Saf. Simul. 1(4), 316 (2010)

    Google Scholar 

  12. 12.

    Funk, M., Shirazi, A.S., Mayer, S., Lischke, L., Schmidt, A.: Pick from here!: an interactive mobile cart using in-situ projection for order picking. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing ACM, pp. 601–609 (2015)

  13. 13.

    paaron301. How Inside-out Tracking Works—Enthusiast Guide. https://docs.microsoft.com/en-us/windows/mixed-reality/enthusiast-guide/tracking-system. Accessed 7 May 2018

  14. 14.

    Isler, C., Righetto, G., Morabito, R.: Optimizing the order picking of a scholar and office supplies warehouse. Int. J. Adv. Manuf. Technol. 87(5–8), 2327 (2016)

    Article  Google Scholar 

  15. 15.

    Bowman, D., Kruijff, E., LaViola Jr., J.J., Poupyrev, I.P.: 3D User Interfaces: Theory and Practice, CourseSmart eTextbook. Addison-Wesley, Boston (2004)

    Google Scholar 

  16. 16.

    Guo, A., Wu, X., Shen, Z., Starner, T., Baumann, H., Gilliland, S.: Order picking with head-up displays. Computer 48(6), 16 (2015)

    Article  Google Scholar 

  17. 17.

    Gerstweiler, G., Platzer, K., Kaufmann, H.: Dargs: dynamic AR guiding system for indoor environments. Computers 7(1), 5 (2018)

    Article  Google Scholar 

  18. 18.

    Bellman, R.: Dynamic programming treatment of the travelling salesman problem. J. ACM (JACM) 9(1), 61 (1962)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Johnson, R., Hoeller, J., Donald, K., Sampaleanu, C., Harrop, R., Risberg, T., Arendsen, A., Davison, D., Kopylenko, D., Pollack, M., et al.: The spring framework-reference documentation. Interface 21, 27 (2004)

    Google Scholar 

  20. 20.

    Held, M., Karp, R.M.: A dynamic programming approach to sequencing problems. J. Soc. Ind. Appl. Math. 10(1), 196 (1962). https://doi.org/10.1137/0110015

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Cui, X., Shi, H.: An overview of pathfinding in navigation mesh. Int. J. Comput. Sci. Netw. Secur. 12(12), 48 (2012)

    Google Scholar 

  22. 22.

    Cui, X., Shi, H.: A*-based pathfinding in modern computer games. Int. J. Comput. Sci. Netw. Secur. 11(1), 125 (2011)

    Google Scholar 

  23. 23.

    Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4(2), 100 (1968)

    Article  Google Scholar 

  24. 24.

    Goyal, A., Mogha, P., Luthra, R., Sangwan, N.: Path finding: A* or dijkstra’s? IJITE 2, 13–14 (2014)

    Google Scholar 

Download references


This work was supported by the Kumoh National Institute of Technology (KIT), Gumi, Republic of Korea (No.2019-104-139).

Author information



Corresponding author

Correspondence to Soo Young Shin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in the studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its subsequent amendments or comparable ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 32633 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Latif, U.K., Shin, S.Y. OP-MR: the implementation of order picking based on mixed reality in a smart warehouse. Vis Comput 36, 1491–1500 (2020). https://doi.org/10.1007/s00371-019-01745-z

Download citation


  • Order picking
  • Mixed reality
  • Route optimization