Three-Way Nonparametric Bayesian Clustering for Handwritten Digit Image Classification

  • Tomonari Masada
  • Atsuhiro Takasu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8228)


This paper proposes a new approach for handwritten digit image classification using a nonparametric Bayesian probabilistic model, called multinomialized subset infinite relational model (MSIRM). MSIRM realizes a three-way clustering, i.e., a simultaneous clustering of digit images, pixel columns, and pixel rows, where the numbers of clusters are adjusted automatically with Chinese restaurant process (CRP). We obtain MSIRM as a modification of subset infinite relational model (SIRM) by Ishiguro et al [4] While this modification is straightforward, our application of MSIRM to handwritten digit image classification leads to an impressive result. To represent a large number of training digit images in a compact form, we cluster the training images and then classify a test image to the class of the cluster most similar to the test image. By extending this line of thought, MSIRM clusters not only digit images but also pixel columns and pixel rows to obtain a more compact representation. With this three-way clustering, we achieved 2.95% and 5.38% test error rates for MNIST and USPS datasets, respectively.


clustering Bayesian nonparametrics handwritten digit recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Proc. of CVPR 2012, 3642–3649 (2012)Google Scholar
  2. 2.
    Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: Proc. of KDD 2001, pp. 269–274 (2001)Google Scholar
  3. 3.
    Halkias, X., Paris, S., Glotin, H.: Sparse penalty in deep belief networks: using the mixed norm constraint. In: Proc. of ICLR 2013 (2013)Google Scholar
  4. 4.
    Ishiguro, K., Ueda, N., Sawada, H.: Subset infinite relational models. JMLR W&CP 22, 547–555 (2012); AISTATS 2012Google Scholar
  5. 5.
    Keglevic, M., Sablatnig, R.: Digit recognition in handwritten weather records. In: Proc. of OAGM/AAPR Workshop (2013)Google Scholar
  6. 6.
    Keysers, D., Dahmen, J., Theiner, T., Ney, H.: Experiments with an extended tangent distance. In: Proc. of ICPR 2000, pp. 38–42 (2000)Google Scholar
  7. 7.
    Maji, S., Malik, J.: Fast and Accurate Digit Classification. Tech. Rep. No. UCB/EECS-2009-159 (2009)Google Scholar
  8. 8.
    Masada, T., Takasu, A.: Trimming prototypes of handwritten digit images with subset infinite relational model. In: Proc. of MUE 2013, pp. 129–134 (2013)Google Scholar
  9. 9.
    Minka, T.P.: Estimating a Dirichlet distribution (2000),
  10. 10.
    Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Statist. 9(2), 249–265 (2000)MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tomonari Masada
    • 1
  • Atsuhiro Takasu
    • 2
  1. 1.Nagasaki UniversityNagasakiJapan
  2. 2.National Institute of InformaticsChiyoda-kuJapan

Personalised recommendations