Abstract
De-slagging is a task of removing slag on the surface of molten metals, such as steel, in a ladle. In this paper, we propose a method of slag removal path estimation using CNN (Convolution Neural Network) to automate de-slagging task using a robotic machine. From a sequence on images captured from the top of the ladle, we first extract the 2-dimensional trajectory of the slag removal motion of an experienced human operator. Then several image blocks are obtained at sample points along the removal trajectory to train a neural network. The output of the network consists of four labels which represent the probability of four different removal directions of an input image block. To test the trained neural network, we uniformly divide a test ladle image to a fixed-size block with a given stride value. All image blocks are tested and the probability of the four directions are determined and recorded by the trained network. By multiplying the slag probability with the removal direction probability, joint probability of slag removal direction (JPSRD) is introduced. Finally, a slag removal path is estimated by applying the backward tracing method from the endpoint of the ladle so that the estimated path yields the highest JPSRD. A curve fitting is then applied to make smooth slag removal path. The path decision accuracy of an image block is about 90%. We also compare the estimated a slag removal path with that of the experienced operator.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436–444, 2015.
Y. LeCun, L. Botttou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, pp. 323–533, 1986.
I. Goodfellow, J. Pouget-Abadie, Jean, M. Mehdi, X. Bing, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems, pp. 2672–2680, 2014.
S. Sarata, Y. Weeramhaeng, and T. Tsubouchi, “Planning of scooping position and approach path for loading operation by wheel loader,” Proc. of the International Symposium on Automation and Robotics in Construction, 2005.
S. Sarata, “Model-based task planning for loading operation in mining,” Proc. of IEEE/RSJ International Conferecne on Intelligent Robots and Systems (IROS), pp. 439–445, 2001.
S. Lee, D. Hong, and H. Park, “Optimal path generation for excavator with neural networks based soil models,” Proc. of IEEE international Conference on Multisensor Fusion and integration for intelligent Systems, pp. 632–637, 2008.
R. Fukui, T. Niho, M. Nakao, and M. Uetake, “Imitation-based control of automated ore excavator to utilize human operator knowledge of bedrock condition estimation and excavating motion selection,” Proc. of IEEE/RSJ International Conferecne on Intelligent Robots and Systems (IROS), pp. 5910–5916, 2015.
L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees, Wadsworth, 1984.
J. Kim, G. Ahn, and S. Park, “A block-based path recognition of slag removal using convolutional neural network,” Proc. of the International Conference on Pattern Recognition and Artificial Intelligence, 2018.
Open Source Computer Vision Library [Online], https://www.opencv.org.
R. A. Ulichiney, “Dithering with blue noise,” Proc. of the IEEE, vol. 76, no. 1, pp. 56–79 1998.
J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679–698, 1986.
G. Farin, Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide, Elsveir, 1987.
H. Lee and H. J. Kim, “Trajectory tracking control of multirotors from modelling to experiments: a survey,” International Journal of Control, Automation, and Systems, vol. 15, no. 1, pp. 281–292, 2017.
M. Li, T. Zhang, and Y. Chen, “Efficient mini-batch training for stochastic optimization,” Proc. of ACM SIGKDD International Conferecne on Knowledge Discovery and Data Mining, 2014.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Recommended by Editor Kyoung Kwan Ahn. This work was supported partly by ‘Study on Integrated Recognition Technology of Ladle Environment for Slag Removal’ funded by the Reseach Institute of Industrial Science and Technology (RIST) and partly by ‘The Cross-Ministry Giga KOREA Project’ grant funded by the Korea government(MSIT) (No.GK17P0300).
Jeong-Soo Kim received his B.S. degree in computer engineering from Keimyung University in 2016 and his M.S. degree in Electrical Engineering and Computer Science from Kyungpook National University in 2018. His research interests include robot vision, deep learning.
Geon-Tae Ahn received his B.S. degree in Computer Science in 1998, and his M.S. and Ph.D. degrees in Information and Communication Engineering from Ulsan University, in 2001 and 2005, respectively. He has been working with the Research Institute of Industrial Science as senior engineering researcher since 2006. Currently his main research interests are focused on machine learning and its applications in smart factories.
Soon-Yong Park received his B.S. and M.S. degrees in Electronics Engineering from Kyungpook National University, Daegu, Korea, in 1991 and 1999, and his Ph.D. degree in Electrical and Computer Engineering from State University of New York at Stony Brook in 2003. From 1993 to 1999, he was a senior research staff at KAERI, Korea. He is currently a professor in the School of Computer Science and Engineering, Kyungpook National University. His research interests include 3D sensing and modeling, multi-view 3D data processing.
Rights and permissions
About this article
Cite this article
Kim, JS., Ahn, GT. & Park, SY. Estimation of Slag Removal Path using CNN-based Path Probability of Ladle Image Blocks. Int. J. Control Autom. Syst. 18, 791–800 (2020). https://doi.org/10.1007/s12555-019-0019-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-019-0019-3