Journal of Intelligent Manufacturing

, Volume 28, Issue 5, pp 1061–1077 | Cite as

Predictive distant operation and virtual control of computer numerical control machines

  • Toly Chen
  • Yi-Chi WangEmail author
  • Zhirong Lin


With the advancements of automation technologies, factory operations have shifted from labor-based to semi-automatic and fully automatic and may even become unmanned in the future. Therefore, the conditions of a factory can be monitored from a distance and the machines can be remotely controlled. This study investigated the predictive distant operation of a computer numerical control (CNC) machine virtually controlled with hand gestures. Similar previous studies were conducted only involving robotic arms. Nevertheless, the attempt in this study is a crucial step toward entirely Internet- or cloud-based manufacturing. In the proposed methodology, meaningful hand gestures corresponding to eight commonly used CNC operations were established. Kinect was then used to track the skeletal information of an operator, and based on this information, the joint angles of the operator were derived using the space vector method. Four rules were then proposed to predict the hand gesture of the operator according to the changes of the joint angles when making the hand gestures to control the CNC machine. An experimental system was established to demonstrate the applicability of the proposed methodology. According to the experimental results, the eight hand gestures were recognized with favorable accuracy. In addition, analysis of variance was performed and the results indicated that the performance of the proposed methodology is robust to the operators’ differences.


Cloud manufacturing Kinect  Human–computer interaction Predictive  Distant operation Virtual control 



This work was supported by Ministry of Science and Technology of Taiwan under the Grant MOST 103-2221-E-035-073-MY3.


  1. Archimede, B., Letouzey, A., Memon, M. A., & Xu, J. (2014). Towards a distributed multi-agent framework for shared resources scheduling. Journal of Intelligent Manufacturing, 25(5), 1077–1087.CrossRefGoogle Scholar
  2. Binney, D., & Boehm, J. (2010). Performance evaluation of the PrimeSense IR projected pattern depth sensor.
  3. Borangiu, T., Raileanu, S., Trentesaux, D., Berger, T., & Iacob, I. (2014). Distributed manufacturing control with extended CNP interaction of intelligent products. Journal of Intelligent Manufacturing, 25(5), 1065–1075.CrossRefGoogle Scholar
  4. Chen, T. (2014). Strengthening the competitiveness and sustainability of a semiconductor manufacturer with cloud manufacturing. Sustainability, 6, 251–268.CrossRefGoogle Scholar
  5. Dang, Q.-V., Nielsen, I., Steger-Jensen, K., & Madsen, O. (2014). Scheduling a single mobile robot for part-feeding tasks of production lines. Journal of Intelligent Manufacturing, 25(6), 1271–1287.CrossRefGoogle Scholar
  6. Du, G., Zhang, P., Mai, J., & Li, Z. (2012). Markerless Kinect-based hand tracking for robot teleoperation. International Journal of Advanced Robotic Systems, 9, 36.CrossRefGoogle Scholar
  7. Fan, Y., Zhao, D., Zhang, L., Huang, S., & Liu, B. (2004). Manufacturing grid: Needs, concept and architecture. Lecture Notes in Computer Science, 3032, 653–656.CrossRefGoogle Scholar
  8. Gama, A. D., Chaves, T., Figueiredo, L., & Teichrieb, V. (2011). Improving motor rehabilitation process through a natural interaction based system using Kinect sensor. In IEEE symposium on 3D user interfaces (pp. 145–146).Google Scholar
  9. Gil, P., Mateo, C., & Torres, F. (2014). 3D visual sensing of the human hand for the remote operation of a robotic hand. International Journal of Advanced Robotic Systems, 11, 26.CrossRefGoogle Scholar
  10. Hsieh, F.-S., & Lin, J.-B. (2014). Context-aware workflow management for virtual enterprises based on coordination of agents. Journal of Intelligent Manufacturing, 25(3), 393–412.CrossRefGoogle Scholar
  11. Huang, T. S., & Pavlovic, V. I. (1995). Hand gesture modeling, analysis and synthesis. In Proceedings of international workshop on automatic face and gesture recognition (pp. 73–79).Google Scholar
  12. Jeon, M.-J., Lee, S. W., & Bien, Z. (2011). Hand gesture recognition using multivariate fuzzy decision tree and user adaptation. International Journal of Fuzzy System Applications, 1(3), 15–31.CrossRefGoogle Scholar
  13. Lambrecht, J., Kleinsorge, M., Rosenstrauch, M., & Krüger, J. (2013). Spatial programming for industrial robots through task demonstration. International Journal of Advanced Robotics Systems, 10, 254.CrossRefGoogle Scholar
  14. Lange, B., Koenig, S., McConnell, E., Chang, C. Y., Juang, R., Suma, E., Bolas, M. , & Rizzo, A. (2012). Interactive game-based rehabilitation using the Microsoft Kinect. In IEEE virtual reality (pp. 171–172).Google Scholar
  15. Li, B. H., Zhang, L., & Wang, S. L. (2010). Cloud manufacturing: A service-oriented new networked manufacturing model. Computer Integrated Manufacturing Systems, 16(1), 1–9.Google Scholar
  16. Lin, Z. (2014). Distant operation and virtual control of CNC machines. Master Thesis, Taiwan:Feng Chia University,Google Scholar
  17. Microsoft TechNet. (2014). Working of Microsoft’s PrimeSense technology based Kinect: An elaboration.
  18. Microsoft. (2014). Kinect for Windows.
  19. (2012). Welcome to the OpenKinect project.
  20. Pham, A. (2009). E3: Microsoft shows off gesture control technology for Xbox 360. Los Angeles Times.Google Scholar
  21. Sarraj, R., Ballot, E., Pan, S., & Montreuil, B. (2014). Analogies between Internet network and logistics service networks: Challenges involved in the interconnection. Journal of Intelligent Manufacturing, 25(6), 1207–1219.CrossRefGoogle Scholar
  22. Smid, P. (2008). CNC programming handbook (3rd ed.). New York: Industrial Press.Google Scholar
  23. Smid, P. (2010). CNC control setup for milling and turning. New York: Industrial Press.Google Scholar
  24. Valentino, J., & Goldenberg, J. (2003). Introduction to computer numerical control (CNC). Englewood Cliffs: Prentice Hall.Google Scholar
  25. Wu, D., Greer, M. J., Rosen, D. W., & Schaefer, D. (2013a). Cloud manufacturing: Strategic vision and state-of-the-art. Journal of Manufacturing Systems, 32(4), 564–579.CrossRefGoogle Scholar
  26. Wu, D., Thames, J. L., Rosen, D. W., & Schaefer, D. (2013b). Enhancing the product realization process with cloud-based design and manufacturing systems. Journal of Computing and Information Science in Engineering, 13(4), 1–14.CrossRefGoogle Scholar
  27. Wu, Q., Zhu, Q., & Zhou, M. (2014). A correlation-driven optimal service selection approach for virtual enterprise establishment. Journal of Intelligent Manufacturing, 25(6), 1441–1453.Google Scholar
  28. Wu, D., Rosen, D. W., Wang, L., & Schaefer, D. (2015). Cloud-based design and manufacturing: A new paradigm in digital manufacturing and design innovation. Computer-Aided Design, 59, 1–14.CrossRefGoogle Scholar
  29. Yang, S., Park, K., & Bien, Z. (2011). Gesture spotting using fuzzy garbage model and user adaptation. International Journal of Fuzzy System Applications, 1(3), 47–65.CrossRefGoogle Scholar
  30. Zhang, L., Guo, H., Tao, F., Luo, Y. L., & Si, N. (2010). Flexible management of resource service composition in cloud manufacturing. IEEE International Conference on Industrial Engineering and Engineering Management (pp. 2278–2282).Google Scholar
  31. Zhenyao, M., & Neumann, U. (2006). Real-time hand pose recognition using low-resolution depth images. Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. 1499–1505).Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Industrial Engineering and Systems ManagementFeng Chia UniversityTaichung CityTaiwan

Personalised recommendations