Information-Geometric Optimization: The Interest of Information Theory for Discrete and Continuous Optimization

Abstract

Black box optimization is the problem of searching for the minimum of a function on a given space (discrete or continuous), without any prior knowledge about the function. Information geometry provides a systematic method, IGO (information-geometric optimization) to easily build optimization algorithms having nice properties; in particular it minimizes the influence of arbitrary choices such as how the space of solutions is represented. In some situations IGO recovers known and widely used algorithms, thus providing theoretical justification for them. Specific properties of information geometry and the Kullback–Leibler divergence guarantee, at each step, minimal diversity loss in the exploration of possible solutions; this suggests IGO algorithms automatically tune the simultaneous exploration of different regions.