Rough Deep Belief Network - Application to Incomplete Handwritten Digits Pattern Classification

  • Wojciech K. Mleczko
  • Tomasz Kapuściński
  • Robert K. NowickiEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 538)


The rough deep belief networks (RDBN) are new modification of well known deep belief networks. Thanks to applied elements from Pawlak’s rough set theory, RDBNs are suitable in processing of incomplete patterns. In this paper we present the results of adaptation of this class of networks for classification of handwritten digits. The samples of the pattern applied in the learning and working processes are randomly corrupted. This allows to study the robustness of classifier for various levels of incompleteness.


Deep belief network Rough set Missing features 


  1. 1.
    Bilski, J.: Momentum modification of the RLS algorithms. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 151–157. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Chu, J.L., Krzyzak, A.: The recognition of partially occluded objects with support vector machines and convolutional neural networks and deep belief networks. J. Artif. Intell. Soft Comput. Res. 4(1), 5–19 (2014)CrossRefGoogle Scholar
  3. 3.
    Cpaka, K., Nowicki, R., Rutkowski, L.: Rough-neuro-fuzzy systems for classification. In: The First IEEE Symposium on Foundations of Computational Intelligence (FOCI 2007) (2007)Google Scholar
  4. 4.
    Dourlens, S., Ramdane-Cherif, A.: Modeling & understanding environment using semantic agents. J. Artif. Intell. Soft Comput. Res. 1(4), 301–314 (2011)Google Scholar
  5. 5.
    Dubois, D., Prade, H.: Rough fuzzy sets and fuzzy rough sets. Int. J. Gen. Syst. 17(2–3), 191–209 (1990)CrossRefGoogle Scholar
  6. 6.
    Dubois, D., Prade, H.: Putting rough sets and fuzzy sets together. In: Słowiński, R. (ed.) Intelligent Decision Support: Handbook of Applications and Advances of the Rough Sets Theory, pp. 203–232. Kluwer, Dordrecht (1992)CrossRefGoogle Scholar
  7. 7.
    Hinton, G.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)CrossRefGoogle Scholar
  8. 8.
    Hinton, G.: A practical guide to training restricted Boltzmann machines. Momentum 9(1), 926 (2010)Google Scholar
  9. 9.
    Hinton, G., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Karpathy, A.: Code for training restricted Boltzmann machines (RBM) and deep belief networks in MATLAB.
  11. 11.
    Karpathy, A.: CPSC 540 project: Restricted Boltzmann machinesGoogle Scholar
  12. 12.
    Korytkowski, M., Nowicki, R., Rutkowski, L., Scherer, R.: AdaBoost ensemble of DCOG rough–neuro–fuzzy systems. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part I. LNCS, vol. 6922, pp. 62–71. Springer, Heidelberg (2011)Google Scholar
  13. 13.
    Korytkowski, M., Nowicki, R., Scherer, R.: Neuro-fuzzy rough classifier ensemble. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part I. LNCS, vol. 5768, pp. 817–823. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Korytkowski, M., Nowicki, R., Scherer, R., Rutkowski, L.: Ensemble of rough-neuro-fuzzy systems for classification with missing features. Proc. World Congr. Comput. Intell. 2008, 1745–1750 (2008)Google Scholar
  15. 15.
    Laskowski, L., Laskowska, M.: Functionalization of sba-15 mesoporous silica by cu-phosphonate units: probing of synthesis route. J. Solid State Chem. 220, 221–226 (2014)CrossRefGoogle Scholar
  16. 16.
    Laskowski, L., Laskowska, M., Balanda, M., Fitta, M., Kwiatkowska, J., Dzilinski, K., Karczmarska, A.: Mesoporous silica sba-15 functionalized by nickel-phosphonic units: raman and magnetic analysis. Microporous Mesoporous Mater. 200, 253–259 (2014)CrossRefGoogle Scholar
  17. 17.
    Laskowski, Ł., Laskowska, M., Jelonkiewicz, J., Boullanger, A.: Spin-glass implementation of a hopfield neural structure. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS, vol. 8467, pp. 89–96. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  18. 18.
    Le Roux, N., Bengio, Y.: Representational power of restricted boltzmann machines and deep belief networks. Neural Comput. 20(6), 1631–1649 (2008)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Little, R., Rubin, D.: Statistical Analysis with Missing Data. Wiley, New York (1987)zbMATHGoogle Scholar
  20. 20.
    Nowak, B.A., Nowicki, R.K., Mleczko, W.K.: A new method of improving classification accuracy of decision tree in case of incomplete samples. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 448–458. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  21. 21.
    Nowicki, R.: Rough-neuro-fuzzy structures for classification with missing data. IEEE Trans. Syst. Man Cybern.-Part B: Cybern 39(6), 1334–1347 (2009)CrossRefGoogle Scholar
  22. 22.
    Nowicki, R.: On combining neuro-fuzzy architectures with the rough set theory to solve classification problems with incomplete data. IEEE Trans. Knowl. Data Eng. 20(9), 1239–1253 (2008)CrossRefGoogle Scholar
  23. 23.
    Nowicki, R.K., Nowak, B.A., Woźniak, M.: Rough k nearest neighbours for classification in the case of missing input data. In: Proceedings of the 9th International Conference on Knowledge, Information and Creativity Support Systems, Limassol, Cyprus, pp. 196–207, November 2014Google Scholar
  24. 24.
    Pawlak, M.: Kernel classification rules from missing data. IEEE Trans. Inf. Theory 39, 979–988 (1993)CrossRefGoogle Scholar
  25. 25.
    Pawlak, Z.: Rough sets. Int. J. Comput. Inform. Sci. 11(5), 341–356 (1982)CrossRefGoogle Scholar
  26. 26.
    Pawlak, Z.: Rough Sets: Theoretical Aspects of Reasoning About Data. Kluwer, Dordrecht (1991)CrossRefGoogle Scholar
  27. 27.
    Smolensky, P.: Information processing in dynamical systems: foundations of harmony theory. In: Rumelhart, D.E., McLelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Vol. 1 Fundations, pp. 194–281. MIT Press, Cambridge (1986)Google Scholar
  28. 28.
    Staff, C.I., Reinders, J.: Parallel Programming and Optimization with Intel\(\textregistered \) Xeon PhiTM Coprocessors: Handbook on the Development and Optimization of Parallel Aplications for Intel\(\textregistered \) Xeon Coprocessors and Intel\(\textregistered \) Xeon PhiTM Coprocessors. Colfax International, Sunnyvale (2013)Google Scholar
  29. 29.
    Szustak, L., Rojek, K., Gepner, P.: Using Intel Xeon Phi coprocessor to accelerate computations in MPDATA algorithm. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 582–592. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  30. 30.
    Zhu, W., Wang, F.Y.: Reduction and axiomization of covering generalized rough sets. Inform. Sci. 152, 217–230 (2003)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Zhu, W., Wang, F.Y.: On three types of covering-based rough sets. IEEE Trans. Knowl. Data Eng. 19(8), 1131–1144 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Wojciech K. Mleczko
    • 1
  • Tomasz Kapuściński
    • 1
  • Robert K. Nowicki
    • 1
    Email author
  1. 1.Institute of Computational IntelligenceCzestochowa University of TechnologyCzestochowaPoland

Personalised recommendations