KI - Künstliche Intelligenz

, Volume 26, Issue 4, pp 373–380 | Cite as

Neural Networks for Complex Data

  • Marie Cottrell
  • Madalina Olteanu
  • Fabrice Rossi
  • Joseph Rynkiewicz
  • Nathalie Villa-Vialaneix
Fachbeitrag

Abstract

Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world problems, ranging from time evolving data to sophisticated data structures such as graphs and functions. This paper summarizes advances on those themes from the last decade, with a focus on results obtained by members of the SAMM team of Université Paris 1.

References

  1. 1.
    Abbott A, Tsay A (2000) Sequence analysis and optimal matching methods in sociology. Sociol Methods Res 29(1):3–33 CrossRefGoogle Scholar
  2. 2.
    Andras P (2002) Kernel-Kohonen networks. Int J Neural Syst 12:117–135 Google Scholar
  3. 3.
    Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 68(3):337–404 MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Besse P, Cardot H, Stephenson D (2000) Autoregressive forecasting of some functional climatic variations. Scand J Stat 4:673–688 CrossRefGoogle Scholar
  5. 5.
    Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, New York Google Scholar
  6. 6.
    Boulet R, Jouve B, Rossi F, Villa N (2008) Batch kernel SOM and related Laplacian methods for social network analysis. Neurocomputing 71(7–9):1257–1273 CrossRefGoogle Scholar
  7. 7.
    Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge MATHGoogle Scholar
  8. 8.
    Bunke H (2003) Graph-based tools for data mining and machine learning. In: Proc of the international conference on machine learning and data mining in pattern recognition. Springer, Berlin, pp 7–19 CrossRefGoogle Scholar
  9. 9.
    Conan-Guez B, Rossi F, El Golli A (2006) Fast algorithm and implementation of dissimilarity self-organizing maps. Neural Netw 19(6–7):855–863 MATHCrossRefGoogle Scholar
  10. 10.
    Cottrell M, Letrémy P (2005) How to use the Kohonen algorithm to simultaneously analyse individuals in a survey. Neurocomputing 63:193–207 CrossRefGoogle Scholar
  11. 11.
    Cottrell M, Girard B, Girard Y, Mangeas M, Muller C (1995) Neural modeling for time series: a statistical stepwise method for weight elimination. IEEE Trans Neural Netw 6(6):1355–1364 CrossRefGoogle Scholar
  12. 12.
    Cottrell M, Girard B, Rousset P (1998) Forecasting of curves using Kohonen classification. J Forecast 17(5–6):429–439 CrossRefGoogle Scholar
  13. 13.
    Cottrell M, Ibbou S, Letrémy P (2004) SOM-based algorithms for qualitative variables. Neural Netw 17:1149–1167 MATHCrossRefGoogle Scholar
  14. 14.
    Dutot AL, Rynkiewicz J, Steiner F, Rude J (2007) A 24-h forecast of ozone peaks and exceedance levels using neural classifiers and weather predictions. Environ Model Softw 22(9):1261–1269 CrossRefGoogle Scholar
  15. 15.
    El Golli A, Rossi F, Conan-Guez B, Lechevallier Y (2006) Une adaptation des cartes auto-organisatrices pour des données décrites par un tableau de dissimilarités. Rev Stat Appl LIV(3):33–64 Google Scholar
  16. 16.
    Gärtner T (2008) Kernel for structured data. World Scientific, Singapore Google Scholar
  17. 17.
    Gärtner T, Flach A, Wrobel S (2003) On graph kernels: hardness a results and efficient alternatives. In: Proc of the annual conference on computational learning theory, pp 129–143 Google Scholar
  18. 18.
    Graepel T, Burger M, Obermayer K (1998) Self-organizing maps: generalizations and new optimization techniques. Neurocomputing 21:173–190 MATHCrossRefGoogle Scholar
  19. 19.
    Hagenbuchner M, Sperduti A, Tsoi AC (2009) Graph self-organizing maps for cyclic and unbounded graphs. Neurocomputing 72(7–9):1419–1430 CrossRefGoogle Scholar
  20. 20.
    Hammer B, Hasenfuss A (2010) Topographic mapping of large dissimilarity data sets. Neural Comput 22(9):2229–2284 MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Hammer B, Jain BJ (2004) Neural methods for non-standard data. In: Proc of XIIth European symposium on artificial neural networks (ESANN 2004), Bruges, Belgium, pp 281–292 Google Scholar
  22. 22.
    Hammer B, Micheli A, Strickert M, Sperduti A (2004) A general framework for unsupervised processing of structured data. Neurocomputing 57:3–35 CrossRefGoogle Scholar
  23. 23.
    Hammer B, Hasenfuss A, Rossi F, Strickert M (2007) Topographic processing of relational data. In: Proc of the 6th workshop on self-organizing maps (WSOM 07), Bielefeld, Germany Google Scholar
  24. 24.
    Hassibi B, Stork DG, Wolff GJ (1993) Optimal brain surgeon and general network pruning. In: Proc of the international conference on neural networks. IEEE Press, New York, pp 293–299 CrossRefGoogle Scholar
  25. 25.
    Hébrail G, Hugueney B, Lechevallier Y, Rossi F (2010) Exploratory analysis of functional data via clustering and optimal segmentation. Neurocomputing 73(7–9):1125–1141 CrossRefGoogle Scholar
  26. 26.
    Hornik K, Stinchcombe M, White H (1989) Multilayer feed-forward networks are universal approximators. Neural Netw 2:359–366 CrossRefGoogle Scholar
  27. 27.
    Kaufman L, Rousseeuw P (1987) Clustering by means of medoids. In: Dodge Y (ed) Statistical data analysis based on the L1-norm and related methods. North-Holland, Amsterdam, pp 405–416 Google Scholar
  28. 28.
    Kohohen T, Somervuo P (1998) Self-organizing maps of symbol strings. Neurocomputing 21:19–30 CrossRefGoogle Scholar
  29. 29.
    Kohonen T (2001) Self-organizing maps, 3rd edn. Springer series in information sciences, vol 30. Springer, Berlin MATHCrossRefGoogle Scholar
  30. 30.
    Kondor R, Lafferty J (2002) Diffusion kernels on graphs and other discrete structures. In: Proc of the 19th international conference on machine learning, pp 315–322 Google Scholar
  31. 31.
    Lapedes A, Farber R (1987) Nonlinear signal processing using neural networks: prediction and system modeling. Signal Process Google Scholar
  32. 32.
    Le Cun Y, Denker J, Solla S (1990) Optimal brain damage. In: Touretzky DS (ed) Advances in neural information processing systems (NIPS 1989), vol 2. Morgan Kaufmann, San Mateo, pp 598–605 Google Scholar
  33. 33.
    Le Roux B, Rouanet H (2004) Geometric data analysis: from correspondence analysis to structured data analysis. Springer, Dordrecht MATHGoogle Scholar
  34. 34.
    Levenshtein VI (1966) Binary codes capable of correcting deletions, insertions and reversals. Sov Phys Dokl 6:707–710 MathSciNetGoogle Scholar
  35. 35.
    Lodhi H, Saunders C, Shawe-Taylor J, Cristianini N, Watkins C (2002) Text classification using string kernels. J. Mach. Learn. Res. 444 Google Scholar
  36. 36.
    Mac Donald D, Fyfe C (2000) The kernel self organising map. In: Proc of 4th international conference on knowledge-based intelligence engineering systems and applied technologies, pp 317–320 Google Scholar
  37. 37.
    Maillet B, Olteanu M, Rynkiewicz J (2004) Nonlinear analysis of shocks when financial markets are subject to changes in regime. In: Proc of XIIth European symposium on artificial neural networks (ESANN 2004), pp 87–92 Google Scholar
  38. 38.
    Mangeas M (1997) Neural model selection: how to determine the fittest criterion. In: Proc of ICANN 1997 (artificial neural networks), Lausanne, Switzerland. Lecture notes in computer science, vol 1327. Springer, Berlin, pp 987–992 CrossRefGoogle Scholar
  39. 39.
    Massoni S, Olteanu M, Rynkiewicz J (2009) Career-path analysis using optimal matching and self-organizing maps. In: Advances in self-organizing maps: 7th international workshop (WSOM 2009). Lecture notes in computer science. Springer, Berlin, pp 154–162 CrossRefGoogle Scholar
  40. 40.
    Newman MEJ (2003) The structure and function of complex networks. SIAM Rev 45:167–256 MathSciNetMATHCrossRefGoogle Scholar
  41. 41.
    Olteanu M, Rynkiewicz J (2008) Estimating the number of components in a mixture of multilayer perceptrons. Neurocomputing 71(7–9):1321–1329 CrossRefGoogle Scholar
  42. 42.
    Olteanu M, Rynkiewicz J (2011) Asymptotic properties of mixture-of-experts models. Neurocomputing 74(9):1444–1449 CrossRefGoogle Scholar
  43. 43.
    Osowski S, Siwek K, Markiewicz T (2004) Mlp and svm networks—a comparative study. In: Proceedings of the 6th nordic signal processing symposium (NSPS 2004), pp 37–40 Google Scholar
  44. 44.
    Ramon J, Gärtner T (2003) Expressivity versus efficiency of graph kernels. In: Proc of first international workshop on mining graphs, trees and sequences (held with ECML/PKDD’03) Google Scholar
  45. 45.
    Ramsay J, Silverman B (1997) Functional data analysis. Springer, New York MATHGoogle Scholar
  46. 46.
    Rossi F, Conan-Guez B (2005) Functional multi-layer perceptron: a nonlinear tool for functional data analysis. Neural Netw 18(1):45–60 MATHCrossRefGoogle Scholar
  47. 47.
    Rossi F, Conan-Guez B (2006) Theoretical properties of projection based multilayer perceptrons with functional inputs. Neural Process Lett 23(1):55–70 CrossRefGoogle Scholar
  48. 48.
    Rossi F, Conan-Guez B, Fleuret F (2002) Functional data analysis with multi layer perceptrons. In: Proceedings IJCNN 2002 (part of WCCI), pp 2843–2848 Google Scholar
  49. 49.
    Rossi F, Conan-Guez B, El Golli A (2004) Clustering functional data with the SOM algorithm. In: Proc of XIIth European symposium on artificial neural networks (ESANN 2004), Bruges, Belgium, pp 305–312 Google Scholar
  50. 50.
    Rynkiewicz J (1999) Hybrid hmm/mlp models for time series prediction. In: Proc of VIIth European symposium on artificial neural networks (ESANN 1999), Bruges, Belgium, pp 455–462 Google Scholar
  51. 51.
    Rynkiewicz J (2001) Estimation of hybrid hmm/mlp models. In: Proc of IXth European symposium on artificial neural networks (ESANN 2001), Bruges, Belgium, pp 383–390 Google Scholar
  52. 52.
    Rynkiewicz J (2006) Consistent estimation of the architecture of multilayer perceptrons. In: Proc of the 14th European symposium on artificial neural networks (ESANN 2006), pp 149–154 Google Scholar
  53. 53.
    Rynkiewicz J (2008) Asymptotic law of likelihood ratio for multilayer perceptron models. In: Advances in neural networks, 5th international symposium on neural networks (ISNN 2008), Beijing, China. Lecture notes in computer science, vol 5263. Springer, Berlin, pp 186–195 Google Scholar
  54. 54.
    Rynkiewicz J (2012) General bound of overfitting for mlp regression models. Neurocomputing 90:106–110 CrossRefGoogle Scholar
  55. 55.
    Rynkiewicz J, Cottrell M, Mangeas M, Yao J (2001) Modèles de réseaux de neurones pour l’analyse des séries temporelles ou la régression. Rev Intell Artif 15(3–4):317–332 Google Scholar
  56. 56.
    Sandberg I (1996) Notes on weighted norms and network approximation of functionals. IEEE Trans Circuits Syst I, Fundam Theory Appl 43(7):600–601 MathSciNetCrossRefGoogle Scholar
  57. 57.
    Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge CrossRefGoogle Scholar
  58. 58.
    Smola A, Kondor R (2003) Kernels and regularization on graphs. In: Warmuth M, Schölkopf B (eds) Proc of the conference on learning theory (COLT) and kernel workshop. Lecture notes in computer science, pp 144–158 CrossRefGoogle Scholar
  59. 59.
    Somervuo P (2004) Online algorithm for the self-organizing map of symbol strings. Neural Netw 17:1231–1239 CrossRefGoogle Scholar
  60. 60.
    Vesanto J (1999) Som-based data visualization methods. Intell Data Anal 3(2):111–126 MATHCrossRefGoogle Scholar
  61. 61.
    Villmann T (2007) Sobolev metrics for learning of functional data—mathematical and theoretical aspects. Machine learning reports 1 (MLR–03-2007), pp 1–15. ISSN:1865-3960 Google Scholar
  62. 62.
    Werbos PJ (1974) Beyond regression: new tools for prediction and analysis in the behavior sciences. Ph.D. thesis, Harvard University, Cambridge, MA Google Scholar
  63. 63.
    White H (1990) Connectionist nonparametric regression: mutilayer feedforward networks can learn arbitrary mappings. Neural Netw 3:535–549 CrossRefGoogle Scholar
  64. 64.
    Yao J (2000) On least square estimation for stable nonlinear ar processes. Ann Inst Math Stat 52:316–331 MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Marie Cottrell
    • 1
  • Madalina Olteanu
    • 1
  • Fabrice Rossi
    • 1
  • Joseph Rynkiewicz
    • 1
  • Nathalie Villa-Vialaneix
    • 1
  1. 1.Équipe SAMM, EA 4543Université Paris I Panthéon-SorbonneParis Cedex 13France

Personalised recommendations