Abstract
Deep neural networks can perform complex transformations for classification and automatic feature extraction. Their training can be time consuming and require a large number of numerical calculations. Therefore, it is important to choose the good initial learning settings. Results depend, inter alia, on a loss function. The paper proposes a new loss function for multiclass, single-label classification. Experiments were conducted with convolutional neural networks trained on several popular data sets. Tests with multilayer perceptron were also carried out. The obtained results indicate that the proposed loss may be a good alternative to the categorical cross-entropy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Khan, A., Sohail, A., Zahoora, U., Qureshi, A.S.: A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 53(8), 5455–5516 (2020)
Francois, C.: Deep learning with Python. Manning Publications, Shelter Island NY (2017)
Géron, A.: Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O'Reilly Media, Sebastopol (2019)
LeCun, Y., Cortes, C.: MNIST database. https://yann.lecun.com/exdb/mnist/. Accessed 07 Jan 2021
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.077472017
Krizhevsky, A.: CIFAR Dataset, https://www.cs.toronto.edu/~kriz/cifar.html. Accessed 07 Jan 2021
Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images (2009). https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf. Accessed 07 Jan 2021
Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. In: Neural Networks: Tricks of the Trade, pp. 437–478. Springer, Heidelberg (2012)
Bengio, Y., Goodfellow, I., Courville, A.: Deep Learning, vol. 1, MIT Press (2017)
Yazan, E., Talu, M.F.: Comparison of the stochastic gradient descent based optimization techniques. International Artificial Intelligence and Data Processing Symposium (IDAP), pp. 1–5. IEEE (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
UCI Machine learning repository, Covertype Dataset. https://archive.ics.uci.edu/ml/datasets/covertype. Accessed 09 Mar 2021
Blackard, J.A., Dean, D.J.: Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Comput. Electron. Agriculture 24(3), 131–151 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Halawa, K. (2021). New Loss Function for Multiclass, Single-Label Classification. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds) Theory and Engineering of Dependable Computer Systems and Networks. DepCoS-RELCOMEX 2021. Advances in Intelligent Systems and Computing, vol 1389. Springer, Cham. https://doi.org/10.1007/978-3-030-76773-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-76773-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-76772-3
Online ISBN: 978-3-030-76773-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)