Skip to main content
SpringerLink
Log in

GPU Accelerated Self-Organizing Map for High Dimensional Data

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The self-organizing map (SOM) model is an effective technique applicable in a wide range of areas, such as pattern recognition and image processing. In the SOM model, the most time-consuming procedure is the training process. It consists of two time-consuming parts. The first part is the calculation of the Euclidean distances between training vectors and codevectors. The second part is the update of the codevectors with the pre-defined neighborhood structure. This paper proposes a graphics processing unit (GPU) algorithm that accelerates these two parts using the graphics rendering ability of GPUs. The distance calculation is implemented in the form of matrix multiplication with compute shader, while the update process is treated as a point-rendering process with vertex shader and fragment shader. Experimental results show that our algorithm runs much faster than previous CUDA implementations, especially for the large neighborhood case. Also, our method can handle the case with large codebook size and high dimensional data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Kohonen T, Oja E, Simula O, Visa A, Kangas J (1996) Engineering applications of the self-organizing map. IEEE Proc 84(10):1358–1384

    Article  Google Scholar 

  2. Nasrabadi N, Feng Y (1988) Vector quantization of images based upon the kohonen self-organizing feature maps. In: Proceedings of the IEEE international conference on neural networks 1988, pp 101–108

  3. Yair E, Zeger K, Gersho A (1992) Competitive learning and soft competition for vector quantizer design. IEEE Trans Signal Process 40(2):294–309

    Article  Google Scholar 

  4. Laha A, Chanda B, Pal NR (2006) Accelerated codebook searching in a som-based vector quantizer. In: Proceedings of the IJCNN 2006, pp 3306–3311

  5. Laha A, Chanda B, Pal NR (2008) Fast codebook searching in a som-based vector quantizer for image compression. Signal Image Video Process 2:39–49

    Article  MATH  Google Scholar 

  6. Amerijckx C, Verleysen M, Thissen P, Legat JD (1998) Image compression by self-organized kohonen map. IEEE Trans Neural Netw 9(3):503–507

    Article  Google Scholar 

  7. Jiang Y, Chen KJ, Zhou ZH (2003) SOM based image segmentation. In: Wang G, Liu Q, Yao Y, Skowron A (eds) Rough sets, fuzzy sets, data mining, and granular computing, vol 2639., Lecture notes in computer scienceSpringer, Berlin, pp 572–572

    Google Scholar 

  8. Neagoe VE, Ropot AD (2002) Concurrent self-organizing maps for pattern classification. In: Proceedings of the first IEEE international conference on cognitive informatics 2002, pp 304–312

  9. Kumar D, Rai C, Kumar S (2005) Face recognition using self-organizing map and principal component analysis. In: Proceedings of the international conference on neural networks and brain (ICNN B ’05), 3, pp 1469–1473

  10. Ciskowski P, Zaton M (2010) Neural pattern recognition with self-organizing maps for efficient processing of forex market data streams. In: Rutkowski L, Scherer R, Tadeusiewicz R, Zadeh L, Zurada J (eds) Artificial intelligence and soft computing, vol 6113., Lecture notes in computer scienceSpringer, Berlin, pp 307–314

    Chapter  Google Scholar 

  11. Vesanto J (1999) SOM-based data visualization methods. Intel Data Anal 3(2):111–126

    Article  MATH  Google Scholar 

  12. Jin H, Shum WH, Leung KS, Wong ML (2004) Expanding self-organizing map for data visualization and cluster analysis. Inf Sci 163(1–3):157–173

    Article  MathSciNet  Google Scholar 

  13. Nikkilä J, Törönen P, Kaski S, Venna J, Castrén E, Wong G (2002) Analysis and visualization of gene expression data using self-organizing maps. Neural Netw 15(8–9):953–966

    Article  Google Scholar 

  14. Leung CS, Chan L (1997) Transmission of vector quantized data over a noisy channel. IEEE Trans Neural Netw 8(3):582–589

    Article  Google Scholar 

  15. Laaksonen J, Koskela M, Oja E (2002) Picsom-self-organizing image retrieval with MPEG-7 content descriptors. IEEE Trans Neural Netw 13(4):841–853

    Article  Google Scholar 

  16. D’haeseleer P (2005) How does gene expression clustering work? Nat Biotechnol 23:1499–1501

    Article  Google Scholar 

  17. Dhillon IS, Modha DS (2001) Concept decompositions for large sparse text data using clustering. Mach Learn 42(1–2):143–175

    Article  MATH  Google Scholar 

  18. Tan PN, Kumar V, Srivastava J (2004) Selecting the right objective measure for association analysis. Inf Syst 29:293–313

    Article  Google Scholar 

  19. Mark WR, Glanville RS, Akeley K, Kilgard MJ (2003) Cg: A system for programing graphics hardware in a C-like language. ACM Trans Graph 22(3):896–907

    Article  Google Scholar 

  20. Bolz J, Farmer I, Grinspun E, Schreoder P (2003) Sparse matrix solvers on the GPU: conjugate gradients and multigrid. ACM Trans Graph 22(3):917–924

    Article  Google Scholar 

  21. Leung CS, Wong TT, Lam PM, Choy KH (2006) An RBF-based image compression method for image-based rendering. IEEE Trans Image Process 15(1):1031–1041

    Article  Google Scholar 

  22. Garcia A, Shen HW (2005) GPU-based 3D wavelet reconstruction with tileboarding. Vis Comput 21(8–10):755–763

    Article  Google Scholar 

  23. Wong TT, Leung CS, Heng PA, Wang J (2007) Discrete wavelet transform on consumer-level graphics hardware. IEEE Trans Multimed 9(3):668–673

    Article  Google Scholar 

  24. Ho TY, Lam PM, Leung CS (2008) Parallelization of cellular neural networks on GPU. Pattern Recogn 41(8):2684–2692

    Article  MATH  Google Scholar 

  25. Bohn C (1998) Kohonen feature mapping through graphics hardware. In: Proceedings of the 3rd international conference on computational intelligence and neurosciences, pp 64–67

  26. Takizawa H, Kobayashi H (2006) Hierarchical parallel processing of large scale data clustering on a PC cluster with GPU co-processing. J Supercomput 36(3):219–234

    Article  Google Scholar 

  27. Xiao Y, Leung CS, Ho TY, Lam PM (2011) A GPU implementation for LBG and SOM training. Neural Comput Appl 20(7):1035–1042

    Article  Google Scholar 

  28. Situ W, Lam YK, Xiao Y, Tsang P, Leung CS (2011) GPU accelerated pk-means algorithm for gene clustering. In: Proceedings of the 2011 international conference on bioinformatics & computational biology

  29. Takatsuka M, Bui M (2010) Parallel batch training of the self-organizing map using OpenCL. In: Proceedings of the ICONIP (2), vol. 6444, Lecture Notes in Computer Science, pp 470–476

  30. Wittek P, Darnyi S (2012) A GPU-accelerated algorithm for self-organizing maps in a distributed environment. In: Proceedings of the 20th European symposium on artificial neural networks, computational intelligence and machine learning 2012. Bruges

  31. Moraes F, Botelho S, Filho N, Gaya J (2012) Parallel high dimensional self organizing maps using CUDA. In: Proceedings of the Brazilian robotics symposium and Latin American robotics symposium (SBR-LARS) 2012, pp 302–306

  32. McConnell S, Sturgeon R, Henry G, Mayne A, Hurley R (2012) Scalability of self-organizing maps on a GPU cluster using OpenCl and CUDA. J Phys 341(1):012–018

    Google Scholar 

  33. Kohonen T (1997) Self-organizing maps. Springer-Verlag, New York

    Book  MATH  Google Scholar 

  34. Togneri R, Lai E, Attikiouzel Y (1990) Kohonen’s algorithm for the numerical parametrisation of manifolds. Pattern Recogn Lett 11(5):313–319

    Article  MATH  Google Scholar 

  35. Leung CS, Chan LW (1999) Design of trellis vetor quantizers using kohonen maps. Neural Netw 12(6):907–914

    Article  Google Scholar 

  36. Harris M (2007) Optimizing parallel reduction in CUDA. NVIDIA

  37. Cui X, Chen Y, Mei H (2009) Improving performance of matrix multiplication and FFT on GPU. In: Proceedings of the international conference on parallel and distributed systems (ICPADS 2009), pp 42–48

Download references

Acknowledgments

The work was supported by RGC General Research Fund from Hong Kong (Project No.: CityU 115612).

Author information

Authors and Affiliations

  1. College of Computer Science and Electronic Engineering, Hunan University, Changsha, China

    Yi Xiao

  2. Department of Electronic Engineering, City University of Hong Kong, Kowloon Tong, Hong Kong

    Yi Xiao, Rui-Bin Feng, Zi-Fa Han & Chi-Sing Leung

Authors
  1. Yi Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui-Bin Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zi-Fa Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi-Sing Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Xiao.

Appendix

Appendix

Suppose the largest texture size supported by customer’s GPU is \(T_x \times T_y\), the codebook size is \(K=K_x\times K_y\), where \(x, y\) denote the size in horizontal and vertical direction. Then \(L=L_x \times L_y\) (all integers) can be easily chosen from the following inequality array

$$\begin{aligned} \begin{array}{l} K_x L_x \le T_x \\ K_y L_y \le T_y \\ M L_x L_y \ge \left\lceil \frac{d_1}{4} \right\rceil , \end{array} \end{aligned}$$
(2)

where \(M\) and \(d_1\) are the same as aforementioned in the paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, Y., Feng, RB., Han, ZF. et al. GPU Accelerated Self-Organizing Map for High Dimensional Data. Neural Process Lett 41, 341–355 (2015). https://doi.org/10.1007/s11063-014-9383-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-014-9383-4

Keywords

Search

Navigation