Applying Softmax Classifiers to Open Set

  • Darren WebbEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1127)


Many artificial neural network classifiers assume a closed-world, where the composition of classes is fixed and known, however problems where this assumption does not hold are found in nearly every case where multi-class classification is applied. For example, in network traffic classification the actual number of classes often dramatically exceeds the number of classes known or labelled at training time. Various treatments have been proposed to adapt closed-set classifiers for application in open-set scenarios, both formal and informal. To demonstrate the effectiveness of these treatments, we conducted an empirical study using a simple example based on the MNIST digit classification problem. Our results show the various treatments make trade-offs between classification recall and precision. We demonstrate that softmax output equalisation during training can significantly improve performance in open set classification.


Open-set recognition Deep learning Neural network 


  1. 1.
    Aceto, G., Ciuonzo, D., Montieri, A., Pescapè, A.: Mobile encrypted traffic classification using deep learning. In: 2018 Network Traffic Measurement and Analysis Conference (TMA), pp. 1–8 (2018)Google Scholar
  2. 2.
    Bendale, A.: abhijitbendale/OSDN (2016). Accessed 17 June 2019
  3. 3.
    Bendale, A., Boult, T.E.: Towards open set deep networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1563–1572 (2016)Google Scholar
  4. 4.
    Dhamija, A.R., Günther, M., Boult, T.E.: Reducing network agnostophobia. CoRR abs/1811.04110 (2018).
  5. 5.
    Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks (2016)Google Scholar
  6. 6.
    LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010).
  7. 7.
    Li, Y., et al.: Deep content: unveiling video streaming content from encrypted wifi traffic. In: 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA), pp. 1–8 (2018)Google Scholar
  8. 8.
    van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008). Scholar
  9. 9.
    Prakhya, S., Venkataram, V., Kalita, J.: Open set text classification using convolutional neural networks. In: International Conference on Natural Language Processing, December 2017.
  10. 10.
    Rocha, A., Goldenstein, S.: Multi-class from binary - divide to conquer. In: VISAPP (2009)Google Scholar
  11. 11.
    Scheirer, W., Rocha, A., Sapkota, A., Boult, T.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2013). Scholar
  12. 12.
    Schuster, R., Shmatikov, V., Tromer, E.: Beauty and the burst: remote identification of encrypted video streams. In: USENIX Security Symposium (2017)Google Scholar
  13. 13.
    Sensoy, M., Kaplan, L., Kandemir, M.: Evidential deep learning to quantify classification uncertainty. In: Proceedings of the 32Nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 3183–3193. Curran Associates Inc., USA (2018).
  14. 14.
    Sopyła, K.: ksopyla/tensorflow-mnist-convnets (2017). Accessed 1 May 2019Google Scholar
  15. 15.
    Zhang, J., Chen, X., Xiang, Y., Zhou, W., Wu, J.: Robust network traffic classification. IEEE/ACM Trans. Netw. 23, 1 (2014). Scholar
  16. 16.
    Zhou, X.S., Huang, T.S.: Small sample learning during multimedia retrieval using biasmap. In: CVPR (2001)Google Scholar

Copyright information

© Commonwealth of Australia 2019

Authors and Affiliations

  1. 1.Cyber and Electronic Warfare DivisionDefence Science and Technology GroupEdinburghAustralia

Personalised recommendations