A Dynamic Pruning Strategy for Incremental Learning on a Budget

Abstract

Several kernel-based perceptron learning methods on a budget have been proposed. In the early steps of learning, such methods record a new instance by allocating it a new kernel. In the later steps, however, useless memory must be forgotten to make space for recording important and new instances once the number of kernels reaches an upper bound. In such cases, it is important to find a way to determine what memory should be forgotten. This is an important process for yielding a high generalization capability. In this paper, we propose a new method that selects between one of two forgetting strategies, depending on the redundancy of the memory in the learning machine. If there is redundant memory, the learner replaces the most redundant memory with a new instance. If there is less redundant memory, the learner replaces the least recently used / least frequently used memory. Experimental results suggest that this proposed method is superior to existing learning methods on a budget.

This research was supported by JST Adaptable and Seamless Technology Transfer Program through Target-driven R&D Exploratory Research AS221Z01499A.