Strategy and the pursuit of truth

Abstract

Science is a social epistemic enterprise. The complexity of research requires the division of cognitive labor. As a consequence, scientists have to present results and incorporate the results of others into their body of knowledge. This creates the possibility of strategic behavior, leading to phenomena such as publication bias. To analyze the dynamics of strategic behavior in epistemic communities, agent-based modeling suggests itself as a method. The phenomena generated by the developed agent-based simulation model reveal a diverse set of possible dynamics in strategically heterogeneous groups and support the claim that there is a trade-off between a behavioral rule’s efficacy to generate accurate beliefs under optimal conditions and its robustness to variation in the composition of the epistemic environment.

Appendix: Sensitivity Analysis

The deliberation game model has, in particular due to the vast combinatorial space of agent populations, too large a parameter space to provide a full parameters sweep. However, it is worth looking at some important alternatives to the configurations utilized to support the arguments in this paper. The sensitivity checks will focus on three questions:

• 1. How does variation in the communication schedule change model dynamics?

• 2. How do the accuracy results scale with increasing numbers of participating agents?

• 3. What is the accuracy impact of an increasing number of strategic exaggerators?

1.1 Sequential dynamics

Figures 3 and 4 depict runs that are configured analogously to the runs chosen in the exploration of model dynamics in Sect. 3, with one key difference: Communication is not synchronous, but agents update their expressed opinion in randomized sequential order.

Fig. 3

Group of naive learners without exaggerator (left) and with a single exaggerator (right); updating is performed in randomized sequential order

Fig. 4

Group of cautious learners without exaggerator (left) and with a single exaggerator (right); updating is performed in randomized sequential order

The key results inferred from the synchronous case still hold, though with some variation:

• Exaggerators are still able to exert their influence, fully steering naive learners and partially influencing cautious learners.

• Cautious learners still fail to fully merge.

• Oscillations caused by oversteering from the exaggerator can still be observed, but it is substantially diminished.

In particular with respect to the last result, this scheduling algorithm may produce results that are more realistic for particular domains of applications. While contributions to a scientific conference are often written in parallel and with little to no knowledge what beliefs the other participants will be expressing, political discussions, to name just an obvious example, often have a more sequential, reactive structure.

This partial sensitivity result is supported by the theoretical background of the model in game theory: Sequential and synchronous game structures can differ substantially in their behavioral predictions, and the bounded rationality of agents only increases the difference.

1.2 Scaling, accuracy and communities of exaggeration

To explore both scaling up the model and evaluating the impact of an increased percentage of exaggerators, the following simulation experiment has been performed: Populations of 20 and 50 agents respectively have been simulated playing the simulation game. The groups were populated with strategic exaggerators and either naive or cautious learners. For each configuration, all possible proportions of exaggerators and learners where run. Updating of beliefs was synchronous and the initial step featured again a revelation of the received signal, again unknown to the agents participating. Each of the resulting model configurations has been run for 80 timesteps to remain comparable with the results presented in the main analysis, though convergence generally takes place substantially earlier. Each configuration ran 100 times, and the results are averaged over those 100 runs. Mean squared error, squared error of mean belief and differences between naive and cautious learners are depicted in Figs. 5, 6, 7 and 8 (Note, that the viewpoints for the plots of accuracy differences are slightly different, to make the graph more readable. In the same vein, the accuracy scales in the various plots differ in their range, as the same range for all plots would have rendered them harder to read.)

Fig. 5

20 agents total, mean of squared error of belief, naive learners (left), cautious learners (middle) and difference between the two population types (right). The plane in the difference plot signifies 0, and values above 0 signify a difference in favor of cautious learners

Fig. 6

50 agents total, mean of squared error of belief, naive learners (left), cautious learners (right) and difference between the two population types (right)

Fig. 7

20 agents total, squared error of mean belief, naive learners (left), cautious learners (middle) and difference between the two population types (right)

Fig. 8

50 agents total, squared error of mean belief, naive learners (left), cautious learners (middle) and difference between the two population types (right)

These plots contain an immense amount of information, but a few key observations are sufficient to understand their general implication. First, the differential advantage of cautious and naive learners appears most pronounced in a comparison of the populations with 0 and 1 exaggerator respectively. When the number of exaggerators increases, accuracy first improves and then worsens substantially. This is in line with the trade-off between cautious and naive social learning argued before.

There is also a pattern that may seem strange at first, namely that populations with a particular large proportion of exaggerators suddenly become way worse, in particular in terms of the accuracy of mean belief. One can also observe, more visible in the graphs depicting the mean of belief inaccuracy, that populations seem to be split between a few distinct levels of accuracy and jump around between these levels in very short time spans and without fully converging at any point during the simulation. The underlying dynamic for these patterns is a population wide oscillation between 0 and 1; without a sufficient amount of learners mediating between the exaggerators by taking averages of their expressed belief, subgroups of exaggerators emerge that jump between the two extremes, forcing each other to reverse that jump to the other extreme on the next time step. With respect to the intended application of scientific exchange, this behavior can only be interpreted as a model artifact due to a move out of the proper parameter space (where not almost everyone is a strategic exaggerator). In cases where such population compositions are plausible, a more subtle update mechanism may be required to avoid this extreme pattern.

The figures also show that the sheer number of agents has little impact on the core results; what is relevant is whether any exaggerators are present and how much of the population they actually comprise.

Finally, it is worth noting that this additional data suggests a limitation to the advantage of competing exaggerators, as speculated in the model analysis. Moving from a single exaggerator to a competitive environment improves accuracy, but further additions quickly start to actually worsen the results again—though not beyond the negative impact of a single exaggerator except for the extreme population-wide oscillation patterns mentioned above. This insight does not contradict the speculation on the advantages of competition, as those are contingent on the presence of a substantial population of learners, and not necessarily imply good accuracy for the competing opinion leaders themselves.