Ideas Worth Spreading: A Free Energy Proposal for Cumulative Cultural Dynamics

Abstract

While there is a fast growing body of theoretical work on characterizing cumulative culture, quantifiable models underlining its dynamics remain scarce. This paper provides an active-inference formalization and accompanying simulations of cumulative culture in two steps: Firstly, we cast cultural transmission as a bi-directional process of communication that induces a generalized synchrony (operationalized as a particular convergence) between the internal states of interlocutors. Secondly, we cast cumulative culture as the emergence of accumulated modifications to cultural beliefs from the local efforts of agents to converge on a shared narrative.

N. Kastel and C. Hesp--Private authors.

Appendix A Generative Model Architecture, Factors and Parameters

1.1 A.1 Higher Level Hidden State Factors

1.2 A.2 Lower Level Hidden State Factors (Specify Events on a Given ‘Day’)

A.3 Higher level generative model

1.3 A.4 Lower Level Generative Model For Action

Action model for meeting selection

In our simulations, we have incorporated psychological biases in agents’ preferences for meeting similar (i.e., belief compatible) or unknown agents. Note that while agents biased toward confirming beliefs would tend toward individuals with similar beliefs to their own, novelty seekers would not look for the opposite of this (i.e. look for individuals of divergent beliefs to their own), but rather have a preference for individuals of yet unknown beliefs.

In active inference, action selection is guided by the expected free energy [G], which entails maximising the expected benefit or utility of an action (known as pragmatic value), while also maximising the potential information gain of future actions by reducing uncertainty about the causes of valuable outcomes (known as epistemic value). These constraints to action selection could be interpreted as formalising the exploration–exploitation trade-off in learning systems. Epistemic value (exploration) refers to the benefit related to searching over a sample space in order to get a better estimation of promising areas that will maximise pragmatic value (exploitation). Active-inference agents would therefore maximise epistemic value until information gain is low, after which the maximisation of pragmatic value and exploitation are assured (Friston et al. 2015).

In our model, agents’ choice in meeting interlocutors with known and similar beliefs versus those with unknown beliefs can be cast in terms of a tradeoff between pragmatic and epistemic value. On the one hand, a confirmation bias emerges from the maximisation of expected utility, increasing synchronisation between interlocutors’ internal models, thus allowing for the emergence of shared expectations (Hesp et al. 2019). On the other hand, novelty seeking emerges from the maximisation of information gain, allowing for the exploration of the sample space. Also understood as intrinsically motivated curious behaviour (Friston et al. 2017), maximisation of epistemic value allows individuals to better predict the consequences of their actions (e.g., which agent to meet) through greater certainty about the hidden states of their environment (e.g., the beliefs of other agents).

From the point of view of agents in our simulations, increasing pragmatic value translates into selecting to meet interlocutors with similar beliefs, while increasing epistemic value translates into selecting agents whose beliefs are unknown or highly uncertain (This way, a meeting increases information gain). From this point of view, it is clear the two values constrain each other and maximizing both simultaneously is partially (but not entirely) paradoxical. While maximising pragmatic value requires agents to choose to meet with an interlocutor they know is similar to them, maximising epistemic value necessitates they meet with one they do not know at all.

$$ \begin{gathered} \,\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;P\left( {u_{loc} } \right) =\upsigma \left( { -\upgamma _{G,loc} G_{loc} +\upgamma _{E,loc} E_{loc} } \right) \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;G_{loc} = o_{u,belief} \cdot \left( {{\text{ln}}\,o_{u,belief} - C_{belief} } \right) + H \cdot x_{u,2,visit} \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;x_{u,2}^{\left( 1 \right)} = B_{u}^{\left( 1 \right)} x_{1}^{\left( 1 \right)} \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\,o_{u,belief} = A_{belief}^{\left( 1 \right)} x_{u,2}^{\left( 1 \right)} \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;c_{beleif} = {\text{In}}\left( {A_{C}^{\left( 2 \right)} x_{core}^{\left( 2 \right)} } \right) \hfill \\ \end{gathered} $$

$$ \begin{gathered} \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;if\,x_{visit,j} = 1; \hfill \\ {\text{[equal}}\,{1}\,{\text{if}}\,{\text{agent}}\,{\text{visited}}\,{\text{a}}\,{\text{particular}}\,{\text{agent}}\,j{]} \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\,\;\;\;\;\;\;\;\;H_{j} = 0 \hfill \\ {\text{[ambiguity}}\,{\text{is}}\,{\text{zero}}\,{\text{if}}\,{\text{agent}}\,{\text{visited}}\,{\text{this}}\,{\text{agent}}\,j\,{\text{already]}}\;else: \hfill \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;H_{j} = 0.1 \hfill \\ {\text{[ambiguity}}\,{\text{is}}\,{\text{non - zero}}\,{\text{if}}\,{\text{agent}}\,{\text{has}}\,{\text{not}}\,{\text{visited}}\,{\text{agent}}\,j\,{\text{yet]}} \hfill \\ \end{gathered} $$

1.4 A6. Generative Process

Generative process for meeting selection

$$ u_{loc} \sim P\left( {u_{loc} } \right)[actual\,meeting\,u_{loc} \,is\,sampled\,from\,meeting\,selection\,prior\,P\left( {u_{loc} } \right)] $$

Generative process for belief expression and EV (satisfaction) of each agent

At a high level of cognitive control, agents incorporate a series of processes underlying the selection of a particular belief for expression (u2). Other than the partial reliance on a low level habitual factor [E], this action involves multiple higher order considerations.

First, an agent considers their core belief state (x), and the way this state apriori maps on to one of two discrete emotional valence states (s2) via an initial likelihood mapping [A2] Emotional Valence (EV) is defined as the extent to which an emotion is positive or negative (Feldman Barrett and Russell 1999), such that agents’ core beliefs are apriori associated with either positive emotional valence or negative emotional valence (with some probability). As a minimal form of vicarious learning, the initial mapping is further updated based on associations agents observe between their interlocutors’ expressed belief state and EV value The initial mapping therefore involves minimal precision for the expected EV for belief 2, since agents are first introduced to this belief (and associated EV) during the simulations. For this reason, the initial likelihood mapping between states is updated throughout our simulation via a crucial concentration parameter (α).

EV states are generated from core belief states, using a (learnable) likelihood mapping:

$$ x_{sat}^{\left( 1 \right)} = A_{sat}^{\left( 2 \right)} x_{core}^{\left( 2 \right)} $$

Confidence of belief expression is generated using a Gamma distribution, where the rate parameter expris the Bayesian model average of (+, –) values associated with high and low satisfaction:

$$ \begin{gathered} \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;P\left( {\upgamma _{expr} } \right) \approx \Gamma \left( {1,\upbeta _{expr} } \right) \hfill \\\upbeta _{expr} =\upbeta ^{{\left( { + , - } \right)}} \cdot x_{sat}^{\left( 1 \right)} ,\;\;\;\;\;\;\;\;\;\;\;\;\;\upbeta ^{{\left( { + , - } \right)}} = [0.25,2.0] \hfill \\ \end{gathered} $$

The expression of beliefs is guided by current core beliefs (scaled with satisfaction-dependent expr) and by habitual belief expression Eexpr (scaled with a fixed parameter E,expr):

$$ P\left( {u_{expr} \left| {\upgamma _{expr} } \right.} \right) = \sigma \left( { -\upgamma _{expr} {\text{ln}}\,x_{core}^{\left( 2 \right)} +\upgamma _{E,expr} E_{expr} } \right) $$

The intrinsically stochastic and itinerant nature of the generative process of communication is modeled by using a two-dimensional Dirichlet distribution to generate observed expressions on the range [0,1], where each agent’s belief expression prior Puexpr|expr is used to specify their concentration parameters (multiplied by 12 to reduce variance):

$$ o_{expr} = Dir\left( {12u_{expr} } \right) $$

Generative process for emotional valence expressed by each agent

$$ \begin{gathered} \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;o_{sat} = A_{sat}^{\left( 1 \right)} x_{sat}^{\left( 1 \right)} \hfill \\ [satisfication\,observed\,by\,interaction\,partner\,corresponds\,to\,actual\,satisfaction] \hfill \\ \end{gathered} $$

The EV state predicted is then used to generate an action confidence value (γ) such that positive EV generates high confidence in a certain expression of the belief state (u1) and negative EV generates low confidence values. Higher confidence values produce higher precision on the expected free energy (G) for one’s belief expressed in the current conversation.

1.5 A7. Perception

Updating beliefs about the other agent’s belief based on their expression

$$ Q\left( {x_{belief}^{\left( 1 \right)} } \right) = o_{expr} $$

Updating of core belief based on beliefs expressed by other agents

$$ Q\left( {x_{core}^{\left( 2 \right)} } \right) =\upsigma \left( {{\text{ln}}\,ln\,x_{core}^{\left( 2 \right)} +\upgamma _{A,self}^{\left( 2 \right)} \,{\text{ln}}\,ln\,o_{expr,self} +\upgamma _{A,other}^{\left( 2 \right)} \,{\text{ln}}\,ln\,o_{expr,other} } \right) $$

1.6 A8. Learning

Habit learning for meeting selection

$$ \begin{gathered} \;\;\;\;\;P\left( {E_{loc} } \right) = Dir\left( {e_{loc} } \right) \hfill \\ Q\left( {E_{loc} } \right) = Dir\left( {e_{loc} + 0.05u_{loc} } \right) \hfill \\ \end{gathered} $$

Habit learning for belief expression

$$ \begin{gathered} \;\;\;\;P\left( {E_{expr} } \right) = Dir\left( {e_{expr} } \right) \hfill \\ Q\left( {E_{expr} } \right) = Dir\left( {e_{expr} } \right) + 0.1o_{expr} \hfill \\ \end{gathered} $$

Perceptual learning for the mapping between satisfaction and core beliefs, based on the expressions of other agents

$$ \begin{gathered} \,\,\,\;\;\;\;\;P\left( {A_{sat}^{\left( 2 \right)} } \right) = Dir\left( {a_{sat}^{\left( 2 \right)} } \right) \hfill \\ Q\left( {A_{sat}^{\left( 2 \right)} } \right) = Dir\left( {a_{sat}^{\left( 2 \right)} +\upgamma _{A}^{\left( 2 \right)} o_{expr} \,{\text{ln}}\,ln\,x_{sat}^{\left( 1 \right)} } \right) \hfill \\ \end{gathered} $$

1.7 A9. Initialisation of Parameters for Each Agent

$$ \begin{aligned} & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{A,belief}}^{{(2)}} \sim \Gamma (5,6) \\ & [regulates\,the\,integration\,of\,beliefs\,of\,other\,agents\,in\,one's\,own\,core\,belief] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{A,sat}}^{{(2)}} \sim \Gamma (10,1) \\ & \begin{array}{*{20}l} {\left[ {regulates\,learning\,rate\,of\,mappings\,between\,satisfaction\,and\,core\,belief\,based\,on\,observed} \right.} \hfill \\ {\left. {correspondences\,in\,other\,agents} \right]} \hfill \\ \end{array} \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{G,loc}} \sim \Gamma (1,1) \\ & [regulates\,reliance\,on\,action\,model\,in\,selecting\,agent\,to\,meet] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{E,loc}} \sim \Gamma (1,1) \\ & [regulates\,reliance\,on\,habitual\,prior\,in\,selecting\,agent\,to\,meet] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \rm{\gamma }_{{E,expr}} \sim N\left( {\frac{{\rm{\gamma }_{{E.scc}} }}{{10}},\frac{{\rm{\gamma }_{{E.loc}} }}{{200}}} \right) \\ & [regulates\,reliance\,on\,habitual\,prior\,in\,expressing\,action,which\,correlates\,with\,\left. {\gamma _{{E,loc}} } \right] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{B,core}}^{{(2)}} \sim \Gamma (4,.5) \\ & [regulates\,stability\,of\,core\,beliefs\,across\,days] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \gamma _{{B,habits}}^{{(2)}} \sim \Gamma (.5,1) \\ & [regulates\,stability\,of\,expression\,habits\,across\,days] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad B_{0}^{{(2)}} = [[.75,.25],[.25,.75]] \\ & [specifies\,baseline\,transition\,probabilities] \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad B^{{(2)}} = \rm{\sigma }\left( {\rm{\gamma }_{B}^{{(2)}} \ln B_{0}^{{(2)}} } \right) \\ & [corrects\,B_{0}^{{(2)}} \,using\,the\,agent - specific\,\rm{\gamma }_{B}^{{(2)}} \,values] \\ \end{aligned} $$

$$ \begin{aligned} & Agents\,with\,relatively\,weak\,confirmation\,bias: \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad A_{C}^{{(2)}} \sim Dir(6,4) \\ & [induces\,weak\,reliance\,on\,core\,beliefs\,for\,specifying\,lower{\text{-}}level\,preferences] \\ & Agents\,with\,relatively\,strong\,confirmation\,bias: \\ & \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad A_{{C,1}}^{{(2)}} \sim Dir(999,1) \\ & [induces\,strong\,reliance\,on\,core\,beliefs\,for\,specifying\,lower{\text{-}}level\,preferences] \\ \end{aligned} $$