Adaptive online learning for nonstationary problems
An adaptation algorithm for online training is examined. For stationary tasks it can reduce the learning rate to reach the best convergence. Instead of simple annealing, it keeps the learning rate flexible, such that it can also adapt to nonstationary tasks. Different tasks, abrupt or gradual changes, and different guidance measures are discussed.
Unable to display preview. Download preview PDF.
- 1.Amari S. (1997), in NIPS 9, MIT Press, in press.Google Scholar
- 2.Opper M. (1996), Phys. Rev. Lett. 77, p. 4671–4674.Google Scholar
- 3.Sompolinsky H., Barkai N. & Seung H.S. (1995), in Neural Networks: The Statistical Mechanics Perspective, World Scientific, p. 105–130.Google Scholar
- 4.Bös S. (1995), in ICANN'95, p. 111-116, and to be submitted.Google Scholar
- 5.Bös S. (1996), in ICANN'96, Springer LNCS 1112, p. 89–94.Google Scholar
- 6.Bös S., Murata N., Amari S., & Müller K.-R. (1997), submitted.Google Scholar
- 7.Murata N., Bös S., Amari S., & Müller K.-R. (1997), in preparation.Google Scholar
- 8.Murata N., Müller K.-R., Ziehe A., & Amari S. (1997), in NIPS 9, MIT Press, in press.Google Scholar