Deciding to Disclose: A Decision Theoretic Agent Model of Pregnancy and Alcohol Misuse

Abstract

We draw together methodologies from game theory, agent based modelling, decision theory, and uncertainty analysis to explore the process of decision making in the context of pregnant women disclosing their drinking behaviour to their midwives. We employ a game theoretic framework to define a signalling game. The game represents a scenario where pregnant women decide the extent to which they disclose their drinking behaviours to their midwives, and midwives employ the information provided to decide whether to refer their patients for costly specialist treatment. This game is then recast as two games played against “nature”, to permit the use of a decision theoretic approach where both classes of agent use simple rules to decide their moves. Four decision rules are explored – a lexicographic heuristic which considers only the link between moves and payoffs, a Bayesian risk minimisation agent that uses the same information, a more complex Bayesian risk minimiser with full access to the structure of the decision problem, and a Cumulative Prospect Theory (CPT) rule. In simulation, we recreate two key qualitative trends described in the midwifery literature for all the decision models, and investigate the impact of introducing a simple form of social learning within agent groups. Finally a global sensitivity analysis using Gaussian Emulation Machines (GEMs) is conducted, to compare the response surfaces of the different decision rules in the game.

Appendix

1.1 A.1 Disclosure Game Model Development

This appendix provides a more in depth exploration of the model development process, beginning by deriving a game to serve as the basis for the model and decision problems.

A game, in the game theoretic sense, can be any interaction where the result for one person is dependent on the actions of another. In this scenario, the result for the woman would seem dependent on whether the midwife chooses to refer her for specialist support (although naturally the reality can only be thought of in terms of risk mitigation), and conversely, the right choice for the midwife is somewhat contingent on what the woman is willing to tell them.

A very simple way to represent this would be a game with two players, who both have two possible moves – ask for help, or not; and refer, or not (Fig. 11.6). Since both parties are invested in the outcome of the pregnancy, we might allow them to share the same payoff if everything ends well.

Fig. 11.6

A very simple two player game. The only time things in this very restricted world obviously end poorly is if the woman asks for help but does not get any. This implies that a rational player would always refer if asked for help, and is indifferent otherwise – in other words, there are three possible Nash equilibriums (a Nash equilibrium is a solution to a game between two or more players, where no player can gain from changing their move)

The first complication, is that there should be differentiation between referring, and doing nothing because specialist treatment incurs a cost. We can modify the payoffs to reflect this, by reducing the midwife’s payoffs when they refer. If the cost of referring is less than the value of a good outcome, then the effect of this is to make the only rational choice when not asked for help is to do nothing.

This simple game is however not very informative, and clearly neglects much of the nuance of the scenario. The wider difficulty here is that the real outcome depends on an attribute of one of the players, rather than their moves. In this case, we would expect the right choices to depend on the alcohol consumption of the woman, rather than entirely on what she has claimed about it. To reflect this, we would need different variations on the same game to reflect this attribute.

To resolve this, we can do exactly that and cast it as a signalling game (Fig. 11.7), with three types of player corresponding to categories of drinking behaviour (light, moderate, and heavy). Each of these types of player, will play a different game. This also introduces a third player, who we will call nature. Nature takes the first move, and decides the type of the woman according to some probability distribution; in this case we will allow the probability of types to be uniform. This changes the dynamics of play substantially, since the midwife can no longer be certain of which game they are playing, and hence which move yields the best outcome. We must also amend the moves, and payoffs slightly. The woman now claims to be one of the types, and may send a signal to say, for example, that she a heavy drinker. We will also modify the common payoffs to allow light drinkers to get the best outcome no matter what, and moderate and heavy types to get the best outcome only if referred. We can also differentiate between the consequences of not getting help for these types by letting heavy drinkers have a very negative outcome, and moderate drinkers a slight one.

Fig. 11.7

A less simple two player signalling game

At this point, the game becomes challenging to analyse from a Nash equilibrium perspective (there are several hundred). But, having raised to issue of stigma, we would also like to incorporate this in the game. A possible approach to this is similar to the drinking behaviour of the women, and lets midwives have a type as well, corresponding to how judgemental they are when receiving signals: non-judgemental, moderately judgemental, and harshly judgemental. The expression of this judgement is not a matter of choice on their part, and is assumed to have no impact on their clinical response. Nature now has an additional move, to choose the type of the midwife, and we add costs for sending moderate and heavy signals. A heavy signal to a harshly judgemental midwife adds a heavy cost, and a moderate cost from a moderate midwife. The resulting game might reasonably be said to be intractable.

At this juncture, we do not gain much further from the game representation, and instead separate it into multiple decision problems.

Breaking the game down into separate decision problems can be achieved by treating the moves of the other players as a chance node, and omitting moves by nature that are known to them. For women, there are two such nodes, corresponding to the move by nature determining the type of midwife they play against, and the midwife’s action. Midwives have a simpler problem with only a single chance node, because the woman’s move is known to them. Figure 11.8 shows the structure of the resulting decision problems. Note that there are in fact three distinct decision problems for the three types of woman, since the move by nature determining their type is known to them.

Fig. 11.8

Influence diagrams, showing the game broken into two decision problems. Squares indicate a decision node, while circles are (from the perspective of the agent) chance nodes (a ) Women (heavy drinkers). (b ) Midwives

The precise structure of the decision problem is to some extent dependent on the decision rule in use, for example the Lexicographic heuristic rule is concerned only with a direct relationship between action and consequence. However, the literal translation from game to decision problem for women yields two chance nodes. As a result, solving this using the heuristic approach requires that the nodes be combined. By the same token, an arbitrarily complex problem could be resolved by rules without this limitation. This is significant, in that the decision problem is an individual agent’s model of the situation, which might not be expected to correspond perfectly with the true sequence of events.

From this position, simulating play and augmenting the basic conjecture is easily achievable, since together the game and the decision rules specify the basis for a simulation model. In the disclosure game case, we make additional stipulations on how many games agents play, order of play, the circumstances under which agents observe true types, and the structure of agent populations amongst others.

1.2 A.2 Simulation Schedule

In this section we give the step by step process for a single run of the disclosure game simulation.

• 1. Generate 1000 women, and place them in a queue

• 2. Generate 100 midwives.

• 3. For each round of the game

  • a. Take 100 women from the queue

  • b. Pair each one with a random midwife

  • c. For each pair

    • i. The woman sends a signal

    • ii. The midwife refers or not based on the signal

    • iii. The woman is informed of her payoff, the midwife’s type, and whether she is referred

    • iv. The woman updates her beliefs

    • v. The midwife stores the game in their memory

    • vi. If the woman is referred

      • A. The midwife is informed of the woman’s true type

      • B. The midwife retrospectively updates their beliefs using the true type, and memories of any games with this woman

      • C. The midwife is now eligible to share their memories of the games played with this woman

  • d. Women who have not been referred or had their baby, join the back of the queue

  • e. New women are generated to replace those referred or delivered

  • f. The new women are added to the back of the queue

  • g. For each referred or birthed woman

    • i. With probability p, her memory of games is shared with the active women

    • ii. She is removed from simulation

  • h. The active women update their beliefs

  • i. For each midwife with information to share

    • i. With probability p, their memory of games with the referred woman is shared

    • ii. The memory is no longer eligible to be shared

  • j. The midwives update their beliefs

1.3 A.3 Agent Examples

This section provides a worked example for the learning and decision process of each agent model, focusing on the behaviour of the signalling agent.

1.3.1 A.3.1 Lexicographic Heuristic

As an example, take a light drinker who has played three rounds with a succession of particularly judgemental midwives, signalling honestly in two and claiming to be a moderate drinker in one. The most common outcome of the honest signal was a payoff of 10, which is clearly preferable to the 9 gained by claiming to be moderate. On that basis, they choose to signal honestly.

1.3.2 A.3.2 Bayesian Payoff

We take our light drinker from the lexicographic case and assume that they began with an uninformative prior. The 6 possible signal-payoffs pairings are then $[(l,10),(m,10),(h,10),(m,9),(h,9),(h,8)]$, with $\alpha _{i} = 1$ for all $i$. After playing the three rounds, $n_{l,10} = 2$, and $n_{m,9} = 1$.

The agent then evaluates $R_{w}$ for each signal, e.g. for the light signal:

$$\displaystyle{ \begin{array}{rl} X & =\{ 10\} \\ R_{w}(l)& =\sum _{x\in X} - xp(x\vert l) = -10p(10\vert l) \\ R_{w}(l)& = -10(\frac{\alpha _{l,10}+n_{l,10}} {\sum _{j}(\alpha _{j}+n_{j})} ) = -10(\frac{1+2} {1+2}) \\ R_{w}(l)& = -10(\frac{3} {3}) = -10\end{array} }$$

and by the same method, $R_{w}(m) = -9\frac{1} {3}$, and $R_{w}(h) = -9$, concluding that signalling honestly is the best move.

1.3.3 A.3.3 Bayesian Risk Minimisation

Returning to our example agent, under this model the type of the midwife becomes salient, hence $n_{h} = 3$, and $n_{l,n} = 2$, $n_{m,n} = 1$. Their prior beliefs remain uninformative, i.e. $\alpha _{j} = 1,j \in \{ l,m,h\}$, $\alpha _{i,j} = 1,i \in \{ r,n\},j \in \{ l,m,h\}$. As before, the agent evaluates $R_{w}$ for the three signals, and the process for the light signal is given below:

$$\displaystyle{ \begin{array}{rl} R_{w}(l,l)& =\sum _{i\in A_{m}}\sum _{j\in \varTheta }- u_{w}(l,i,l,j)p(j)p(i\vert l) \\ R_{w}(l,l)& = -u_{w}(l,r,l,l)p(l)p(r\vert l) - u_{w}(l,n,l,l)p(l)p(n\vert l) \\ {}&\phantom{ = 1} - u_{w}(l,r,l,m)p(m)p(r\vert l) - u_{w}(l,n,l,m)p(m)p(n\vert l) \\ {}&\phantom{ = 1} - u_{w}(l,r,l,h)p(h)p(r\vert l) - u_{w}(l,n,l,h)p(h)p(n\vert l) \\ u_{w}(l,i,l,j)& = 10 \\ R_{w}(l,l)& = -10p(l)p(r\vert l) - 10p(l)p(n\vert l) - 10p(m)p(r\vert l) - 10p(m)p(n\vert l) \\ {}&\phantom{ = 1} - 10p(h)p(r\vert l) - 10p(h)p(n\vert l) \\ p(l)& = \frac{1 + 0} {1 + 1 + 1 + 3} = \frac{1} {6} \\ p(m)& = \frac{1 + 0} {1 + 1 + 1 + 3} = \frac{1} {6} \\ p(h)& = \frac{1 + 3} {1 + 1 + 1 + 3} = \frac{2} {3} \\ p(r\vert l)& = \frac{1 + 0} {1 + 1 + 2} = \frac{1} {4} \\ p(n\vert l)& = \frac{1 + 2} {1 + 1 + 2} = \frac{3} {4} \\ R_{w}(l,l)& = -10 \cdot \frac{1} {6} \cdot \frac{1} {4} - 10 \cdot \frac{1} {6} \cdot \frac{3} {4} - 10 \cdot \frac{1} {6} \cdot \frac{1} {4} - 10 \cdot \frac{1} {6} \cdot \frac{3} {4} - 10 \cdot \frac{2} {3} \cdot \frac{1} {4} \\ {}&\quad \, - 10 \cdot \frac{2} {3} \cdot \frac{3} {4}\\ {}&= -10\end{array} }$$

and similarly for moderate ($R_{w}(m,l) = -9\frac{1} {3}$), and heavy ($R_{w}(h,l) = -8\frac{1} {2}$) signals, once again concluding that honesty is the better option.

1.3.4 A.3.4 Descriptive Decision Theory

Once again, we return to the light drinker example. The inferential aspects are identical with the more complex Bayesian risk minimisation algorithm, hence $p(j)p(i\vert l)$, and $u_{w}(l,i,l,j)$ remain the same, but the agent additionally calculates v(u _w (l, i, l, j))w ⁺(p(j))w ⁺(p(i | l)). For the CPT parameters, the values are those originally given by Tversky and Kahneman (1992) and used in the actual simulations which are given in Table 11.4.

Table 11.4 CPT parameters

$$\displaystyle{\begin{array}{rl} \alpha & = 0.88\\ \gamma & = 0.61 \\ p(l)& = \frac{1} {6} \\ p(m)& = \frac{1} {6} \\ p(h)& = \frac{2} {3} \\ p(r\vert l)& = \frac{1} {4} \\ p(n\vert l)& = \frac{3} {4} \\ u_{w}(l,i,l,j)& = 10 \\ f & = (10; \frac{1} {24},10; \frac{1} {8},10; \frac{1} {24},10; \frac{1} {8},10; \frac{1} {6},10; \frac{1} {2}) \\ f^{+} & = f,f^{-} = () \\ n& = 5 \\ v(u_{w})& = f(u_{w}) = u_{w}^{\alpha } \\ v(u_{w})& = 10^{0.88} = 7.59 \\ \pi _{0}^{+} & = w^{+}( \frac{1} {24} + \frac{1} {8} + \frac{1} {24} + \frac{1} {8} + \frac{1} {6} + \frac{1} {2}) - w^{+}(\frac{1} {8} + \frac{1} {24} + \frac{1} {8} + \frac{1} {6} + \frac{1} {2}) \\ & = w^{+}(1) - w^{+}(\frac{23} {24})\\ & = 0.19 \\ \pi _{1}^{+} & = w^{+}(\frac{1} {8} + \frac{1} {24} + \frac{1} {8} + \frac{1} {6} + \frac{1} {2}) - w^{+}( \frac{1} {24} + \frac{1} {8} + \frac{1} {6} + \frac{1} {2}) \\ & = w^{+}(\frac{23} {24}) - w^{+}(\frac{5} {6})\\ & = 0.17 \\ \pi _{2}^{+} & = w^{+}( \frac{1} {24} + \frac{1} {8} + \frac{1} {6} + \frac{1} {2}) - w^{+}(\frac{1} {8} + \frac{1} {6} + \frac{1} {2}) = w^{+}(\frac{5} {6}) - w^{+}(\frac{19} {24})\\ & = 0.04 \\ \pi _{3}^{+} & = w^{+}(\frac{1} {8} + \frac{1} {6} + \frac{1} {2}) - w^{+}(\frac{1} {6} + \frac{1} {2}) = w^{+}(\frac{19} {24}) - w^{+}(\frac{2} {3})\\ & = 0.09 \\ \pi _{4}^{+} & = w^{+}(\frac{1} {6} + \frac{1} {2}) - w^{+}(\frac{1} {2}) = w^{+}(\frac{2} {3}) - w^{+}(\frac{1} {2})\\ & = 0.09 \\ \pi _{5}^{+} & = w^{+}(\frac{1} {2})\\ & = 0.42 \\ V (f)& = V (f^{+}) + V (f^{-}) = V (f^{+}) + 0 \\ V (f^{+})& =\sum _{ i}^{n}\pi _{i}^{+}(f^{+})v_{i}^{+}(f^{+}) = 7.59 \end{array} }$$

And as before, following the same process for moderate and heavy signals, which yields respectively 7.14, and 6.22, the agent chooses the higher valued action and sends an honest signal.

1.4 A.4 Supplementary Figures

Fig. 11.9

Average fraction of population referred by each appointment, after 1000 rounds, mean with 95 % confidence limit over 1000 runs. Note that the large number of runs leads to very tight confidence intervals

1.5 A.5 Sensitivity Analysis

This section provides complete variance based sensitivity analysis results for the disclosure game model. Each subsection gives results for one simulation output under all four decision rules, with tables providing the percentage of overall variance attributable to the individual parameters, emulator quality statistics, and the five most significant interaction contributions to variance in the output.

1.5.1 A.5.1 Median Moderate Drinker Signalling

Table 11.5 Median moderate drinker signalling parameter sensitivity

Table 11.6 Median moderate drinker signalling emulator statistics

Table 11.7 Top 5 interaction terms for CPT decision rule

Table 11.8 Top 5 interaction terms for Bayesian decision rule

Table 11.9 Top 5 interaction terms for lexicographic decision rule

Table 11.10 Top 5 interaction terms for Bayesian payoff decision rule

1.5.2 A.5.2 Median Between Groups IQR

Table 11.11 Median between groups IQR parameter sensitivity

Table 11.12 Median between groups IQR emulator statistics

Table 11.13 Top 5 interaction terms for CPT decision rule

Table 11.14 Top 5 interaction terms for Bayesian decision rule

Table 11.15 Top 5 interaction terms for lexicographic decision rule

Table 11.16 Top 5 interaction terms for Bayesian payoff decision rule

1.5.3 A.5.3 Median Moderate Drinker Signalling IQR

Table 11.17 IQR of median moderate drinker signalling parameter sensitivity

Table 11.18 IQR of median between groups IQR emulator statistics

Table 11.19 Top 5 interaction terms for CPT decision rule

Table 11.20 Top 5 interaction terms for Bayesian decision rule

Table 11.21 Top 5 interaction terms for lexicographic decision rule

Table 11.22 Top 5 interaction terms for Bayesian payoff decision rule

1.5.4 A.5.4 IQR of Between Groups IQR

Table 11.23 IQR of median between groups IQR parameter sensitivity

Table 11.24 IQR of median between groups IQR emulator statistics

Table 11.25 Top 5 interaction terms for CPT decision rule

Table 11.26 Top 5 interaction terms for Bayesian decision rule

Table 11.27 Top 5 interaction terms for lexicographic decision rule

Table 11.28 Top 5 interaction terms for Bayesian payoff decision rule