Learning at Multiple Resolutions: Wavelets as Basis Functions in Artificial Neural Networks, and Inductive Decision Trees

Abstract

Learning at multiple resolutions provides a fast, hierarchical and efficient technique for extracting models from empirical data. In this chapter we describe the application of wavelets for multi-resolution learning in artificial neural networks and inductive decision trees, and show how wavelets may provide a unifying framework for various supervised learning techniques. A Wave-Net is an artificial neural network with activation functions derived from the class of wavelets. Wave- Nets combine the mathematically rigorous, multi-resolution character of wavelets with the adaptive learning of artificial neural networks. Learning with Wave-Nets is efficient, and is explicitly based on the local or global error of approximation. The advantages of Wave-Net learning over other artificial neural learning techniques are highlighted, and learning methods for minimizing the L² or L∞ norms are described. The reduced black box character of Wave-Nets is demonstrated by the explicit relationship between Wave-Net parameters and the quality of learning, and by the ability to extract if-then rules from a Haar Wave-Net. The relationship between Haar Wave-Nets and other rule-extraction techniques such as decision trees is described.