A competitive functional link artificial neural network as a universal approximator

Abstract

In this article, a competitive functional link artificial neural network (C-FLANN) is proposed for function approximation and classification problems. In contrast to the traditional functional link artificial neural networks (FLANNs), the novel structure is a universal approximator and can be used for various applications. C-FLANN is a single-layered feed-forward neural network that enjoys from the concepts of expanded inputs, information capacity units (ICUs) and a winner-take-all competition among the ICUs. These features increase the information capacity of the model without adding the hidden neurons. In the experimental studies, the proposed method is tested on function approximation problems as well as classification applications. Various comparisons with related algorithms such as improved swarm optimization-based FLANN, random vector FLANN and a multilayer perceptron indicate the superiority of the approach in terms of higher accuracy.

Appendix 1

1.1 Universal Approximation Theorem (Cybenko 1989; Hornik 1991; Hassoun 1995)

Let $f(\cdot )$ be a nonconstant, bounded and monotonically increasing continuous function. Let denote the n-dimensional unit hypercube $[0,1]^{m}$. The space of continuous functions on $I^{m}$ is denoted by $C(I^{m})$. Then, given any function $g\in C(I^{m})$ and $\epsilon > 0$, there exist an integer k and real constants $\rho _i$, $\theta _i \in \mathfrak {R}$, $\zeta _i \in \mathfrak {R}^{m}$, where $i=1\ldots m$ such that we may define:

$$\begin{aligned} G(\vec {X})=\sum _{i=1}^k {\rho _i f(\zeta _i \vec {X}+\theta _i )} \end{aligned}$$

(a.1)

as an approximate realization of the function g, where G is independent of f, that is,

$$\begin{aligned} |G(\vec {X})-g(\vec {X})|<\varepsilon , \end{aligned}$$

(a.2)

for all $\vec {X}\in I^{m}$. In other words, functions of the form $G(\vec {X})$ are dense in $C(I^{m})$.