Neural Processing Letters

, Volume 3, Issue 2, pp 101–106 | Cite as

Use of modular architectures for time series prediction

  • Samy Bengio
  • Françoise Fessant
  • Daniel Collobert


Recently, a lot of papers have been published in the field of time series prediction using connectionist models. Nevertheless we think that one of the major problem with is rarely treated in the literature is related to the choice of input parameters (embedding dimension and delay). In this paper, we propose two modular approaches to this problem and apply them to a sunspot-related time series. Experimental results are then compared to a single multi-layer perceptron in order to estimate performances of these models.

Key words

embedding dimension mixtures of experts modular architectures time-series prediction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    F.Takens, “Detecting strange attractors in turbulence”, in D.A.Rand and L.-S.Young (eds) Dynamical Systems and Turbulence, Vol. 898 of Lecture Notes in Mathematics, pp. 366–381, Warwick 1980, Springer-Verlag: Berlin, 1981.Google Scholar
  2. 2.
    D.E.Rumelhart, G.E.Hinton and R.J.Williams, “Learning internal representations by error propagation”, in D.E.Rummelhart and J.L.McClelland (eds) Parallel Distributed Processing, Vol. 1, MIT Press: Cambridge, MA, 1986.Google Scholar
  3. 3.
    M. Hu, “Application of the adaline system to weather forecasting”, Electrical Engineering Degree Thesis Technical Report 6775-1, Stanford Electronic Laboratory, 1964.Google Scholar
  4. 4.
    A. Lapedes and R. Farber, “Nonlinear signal processing using neural networks: prediction and system modelling”, Tech. Rep. LA-UR-87-2662, Los Alamos National Laboratory, Theoretical Division, 1987.Google Scholar
  5. 5.
    A.S.Weigend and N.A.Gershenfeld (eds), Time Series Prediction: Forecasting the Future and Understanding the Past, Addison-Wesley: Redwood City, CA, 1993.Google Scholar
  6. 6.
    H.Abarbanel, R.Brown, J.Sidorowich and L.Tsimring, “The analysis of observed chaotic data in physical systems”, Reviews of Modern Physics, Vol. 65, pp. 1331–1392, 1993.Google Scholar
  7. 7.
    A.Izenman, “J.R. Wolf and the Zürich sunspot relative numbers”, The Mathematical Intelligencer, Vol. 7, pp. 27–33, 1985.Google Scholar
  8. 8.
    A.S.Weigend, B.A.Huberman and D.E.Rumelhart, “Predicting sunspots and exchange rates with connectionist networks”, in M.Casdagli and S.Eubank (eds), Nonlinear Modeling and Forecasting, pp. 395–431, Addison Wesley: Reading, MA, 1992.Google Scholar
  9. 9.
    F. FogelmanSoulié, “Neural network architectures for pattern recognition”, in V.Cherkassky, J.Friedman and H.Wechster (eds) From Statistics to Neural Networks, Theory and Pattern Recognition Applications, pp. 243–262, Springer Verlag: Berlin, 1994.Google Scholar
  10. 10.
    R.A.Jacobs, M.I.Jordan, S.J.Nowlan and G.E.Hinton, “Adaptive mixtures of local experts”, Neural Computation, Vol. 3, pp. 79–87, 1991.Google Scholar
  11. 11.
    T.Meyer and N.Packard “Local forecasting of high-dimensional chaotic dynamics”, in M.Casdagli and S.Eubank (eds) Nonlinear Modeling and Forecasting, pp. 249–263, Addison Wesley: Reading, MA, 1992.Google Scholar
  12. 12.
    L.Xu, “Signal segmentation by finite mixture model and EM algorithm”in International Symposium on Artificial Neural Networks, pp. 453–458, IEEE: Taiwan, 1994.Google Scholar
  13. 13.
    L.Xu, “Channel equalization by finite mixtures and the EM algorithm”, in V.F.Girosi, J.Makhoul, E.Manolakos and E.Wilson (eds) Neural Networks for Signal Processing, pp. 603–612, IEEE Press: New York, 1995.Google Scholar
  14. 14.
    J.Hamilton,“Analysis of time series subject to changes in regime”, Journal of Econometrics, Vol. 45, pp. 39–79, 1990.Google Scholar
  15. 15.
    H.Tong, Non-Linear Time Series Analysis: A Dynamical Systems Approach, Clarendon Press: Oxford, 1990.Google Scholar
  16. 16.
    Y.Bengio and P.Frasconi, “An input output HMM architecture”, in G.Tesauro, D.S.Touretzky and T.K.Leen (eds) Advances in Neural Information Processing Systems: Proceedings of the 1994 Conference, MIT Press: Cambridge, MA, 1995.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Samy Bengio
    • 1
  • Françoise Fessant
    • 2
  • Daniel Collobert
    • 3
  1. 1.Dept. IROUniversité de MontréalMontréalCanada
  2. 2.LEIBNIZ-IMAGGrenoble CédexFrance
  3. 3.LAB/RIO/TNTFrance Télécom CNETLannion CédexFrance

Personalised recommendations