Relating an Adaptive Network’s Structure to Its Emerging Behaviour for Hebbian Learning

Abstract

In this paper it is analysed how emerging behaviour of an adaptive network can be related to characteristics of the adaptive network’s structure (which includes the adaptation structure). In particular, this is addressed for mental networks based on Hebbian learning. To this end relevant properties of the network and the adaptation that have been identified are discussed. As a result it has been found that in an achieved equilibrium state the value of a connection weight has a functional relation to the values of the connected states.

Appendix Proofs of Propositions 1 and 2

Proof of Proposition 1.

(a) Consider μ < 1. Then by Definition 2 (b) the function W → c(V₁, V₂, W) - μW is monotonically decreasing in W, and since μ − 1 < 0 the function W → (μ − 1)W is strictly monotonically decreasing in W. Therefore the sum of them is also strictly monotonically decreasing in W. Now this sum is

$$ {\text{c}}(V_ 1, \, V_ 2,W) - \upmu W + \, (\upmu - 1)W = {\text{ c}}(V_ 1, \, V_ 2,W) - \, W $$

So, the function W → c(V₁, V₂, W) - W is strictly monotonically decreasing in W; by Definition 2(d) it holds c(V₁, V₂, 1) −1 = μ − 1 < 0, and by Definition 2(c) c(V₁, V₂, 0) – 0 ≥ 0. Therefore c(V₁, V₂, W) - W has exactly 1 point with c(V₁, V₂, W) − W = 0; so for each V₁, V₂ the equation c(V₁, V₂, W) – W = 0 has exactly one solution W, indicated by f_μ(V₁, V₂); this provides a unique function f_μ: [0, 1] x [0, 1] → [0, 1] implicitly defined by c(V₁, V₂, f_μ(V₁, V₂)) = f_μ(V₁, V₂). To prove that f_μ is monotonically increasing, the following. Suppose V₁ ≤ $V^{\prime}_{1}$ and V₂ ≤ $V^{\prime}_{2}$, then by monotonicity of V₁, V₂ → c(V₁, V₂, W) in Definition 2(a) it holds

$$ 0 \, = {\text{ c}}(V_{ 1} , \, V_{ 2} ,{\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} )) \, - {\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} ) \le {\text{c}}(V^{\prime}_{1} , \, V^{\prime}_{ 2} ,{\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} )) \, - {\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} ) $$

So c($V^{\prime}_{1}$, $V^{\prime}_{2}$, f_μ(V₁, V₂)) − f_μ(V₁, V₂) ≥ 0 whereas c($V^{\prime}_{1}$, $V^{\prime}_{2}$, f_μ($V^{\prime}_{1}$, $V^{\prime}_{2}$)) − f_μ($V^{\prime}_{1}$, $V^{\prime}_{2}$) = 0

and therefore

$$ {\text{c}}(V^{\prime}_{ 1} , \, V^{\prime}_{ 2} ,{\text{ f}}_{\upmu} (V^{\prime}_{ 1} , \, V^{\prime}_{ 2} )) \, - {\text{ f}}_{\upmu} (V^{\prime}_{ 1} , \, V^{\prime}_{ 2} ) \le {\text{c}}(V^{\prime}_{ 1} , \, V^{\prime}_{ 2} ,{\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} )) \, - {\text{ f}}_{\upmu} (V_{ 1} , \, V_{ 2} ) $$

By strict decreasing monotonicity of W → c(V₁, V₂, W) - W it follows that f_μ(V₁, V₂) > f_μ($V^{\prime}_{ 1}$, $V^{\prime}_{ 2}$) cannot hold, so f_μ(V₁, V₂) ≤ f_μ($V^{\prime}_{ 1}$, $V^{\prime}_{ 2}$). This proves that f_μ is monotonically increasing. From this monotonicity of f_μ(..) it follows that f_μ(1, 1) is the maximal value and f_μ(0, 0) the minimal value. Now by Definition 1(d) it follows that f_μ(0, 0) = c(0, 0, f_μ(0, 0)) = μ f_μ(0, 0) so f_μ(0, 0) = μ f_μ(0, 0), and as μ < 1 this implies f_μ(0, 0) = 0.

(b) Consider μ = 1. When both V₁, V₂ are > 0, and c(V₁, V₂, W) = W, then W = 1, by Definition 1(d). This defines a function f₁(V₁, V₂) of V₁, V₂ ∈ (0, 1], this time f₁(V₁, V₂) = 1 for all V₁, V₂ > 0. When one of V₁, V₂ is 0 and μ = 1, then also by Definition 1(d) always c(V₁, V₂, W) = W, so in this case multiple solutions for W are possible: every W is a solution, and therefore no unique function value for f₁(V₁, V₂) can be defined then.

Proof of Proposition 2

• (a) From cc(W) monotonically decreasing in W it follows that W→1/cc(W) is monotonically increasing on [0, 1). Moreover, the function W is strictly monotonically increasing; therefore for μ < 1 the function h_μ(W) = (1 − μ)W/cc(W) is strictly monotonically increasing. Therefore h_μ is injective and has an inverse function g_μ on the range of h_μ: a function g_μ with g_μ(h_μ(W)) = W for all W ∈ [0, 1).

• (b) Suppose μ < 1 and c(V₁, V₂, W) = W, then from Definition 2(d) it follows that W = 1 is excluded, since from both c(V₁, V₂, W) = W and c(V₁, V₂, W) = μW it would follow μ = 1, which is not the case. Therefore W < 1, and the following hold

  $$ \eqalign{ & {\text{cs}}(V_ 1, \, V_ 2){\text{ cc}}\left( W \right) \, + \upmu W = \, W \cr & {\text{cs}}(V_ 1, \, V_ 2){\text{ cc}}\left( W \right) \, = \, ( 1- \upmu )W \cr & {\text{cs}}(V_ 1, \, V_ 2) \, = \, ( 1- \upmu )W/{\text{cc}}\left( W \right) \, = \, {{\text{h}}_\upmu }\left( W \right) \cr} $$

  So, h_μ(W) = cs(V₁, V₂). Applying the inverse g_μ yields W = g_μ(h_μ(W)) = g_μ(cs(V₁, V₂)).

  Therefore in this case for the function f_μ from Theorem 1 it holds:

  $$ {\text{f}}_{\upmu} (V_{ 1} , \, V_{ 2} ) \, = W = {\text{ g}}_{\upmu} ({\text{cs}}(V_{ 1} , \, V_{ 2} )) \, < { 1} $$

  so f_μ is the composition of cs(..) followed by g_μ.

• (c) For μ = 1 the equation c(V₁, V₂, W) = W becomes cs(V₁, V₂) cc(W) = 0 and this is equivalent to cs(V₁, V₂) = 0 or cc(W) = 0. From the definition of separation of variables it follows that this is equivalent to V₁ = 0 or V₂ = 0 or W = 1.

• (d) Suppose μ < 1 and c(V₁, V₂, W) = W, then because cs(..) and g_μ are both monotonically increasing, the maximal W is g_μ(cs(1, 1)), and the minimal W is g_μ(cs(0, 0)). For μ = 1 these values are 1 always when V₁, V₂ > 0, and any value in [0, 1] (including 0) when one of V₁, V₂ is 0.