The experts found determining the priors mentally exhausting, as the phrasing of the questions were the same for each health state. It took approximately one hour for each expert to complete the two transition probability matrices (for the first 6 weeks and thereafter). The experts found the feedback loops in the application to be very helpful, as reviewing the results of their estimations helped them correct their initial judgements when they felt it was necessary. The estimates of experts 1 and 2 showed considerable consistency (Fig. 2), whereas expert 3 consistently estimated higher probabilities for the patients staying in the state where they were than did the other two experts. Table 1 shows the mean values of the estimates of the three experts.
In Table 2, the observed relative frequencies of the weekly transitions in the first four weeks of the pivotal clinical trial are shown for both treatment arms . There were quite a few transitions that were not observed, and there were no patients in states 7 and 8 (the two most severe Mohr-Lenert health states) in the first four weeks of the trial. In contrast to this observation, the experts generally believed that any of the possible transitions could happen, but some of these transitions were rather unlikely according to them.
The average estimates of the experts showed reasonable consistency with the trial data, with some attenuation of the extreme observed values in the trial (Tables 1 and 2). For example, both patients who were in the risperidone arm who were in state 3 at the beginning of the trial stayed in state 3, resulting in a 100% transition probability, whereas the mean of the expert estimates was 42.9%. Note that if one of the two patients had moved to a different state, the trial transition probability estimate would have dropped to 50%. Similarly, the estimates for the transitions from state 1 to state 1 were 100% in both treatment arms based on the trial data, while the estimate was 82.39% by the experts. The experts did not exclude the possibility of any particular transition; thus, there were no zero estimates in their estimated transition probability matrix.
The estimates of the two chains converged on the trace plot, and the Brooks-Gelman-Rubin diagnostics did not show evidence for non-convergence either.
As expected, the posterior values of the transition probability matrix were very similar to the trial data relative frequencies in cells where there were a reasonably large number of events (states 2, 4, and 6) (Table 3). For example, in the clinical trial, the proportion of patients who stayed in state 2 was 80.99% in the risperidone arm, whereas the estimated posterior probabilities (expressed in %) were 80.72 and 80.43%, depending on the choice of the prior probability. In the case of the mildest disease state (state 1), the pattern of the posterior estimates clearly followed the pattern of the clinical trial data estimates when the informative prior was used. The posterior probability of staying in the mildest state, however, was much lower when the uninformative (flat) prior was used. This result was expected, as the experts’ opinions were in line with the observations in the clinical trial, and only 2 and 3 cases occurred in the risperidone arm and in the cariprazine arm, respectively, in the initial 4 weeks. This behaviour also occurred for the patient movements from state 3 in the risperidone arm. As the prior probability and the relative frequency from the trial had the same level of uncertainty (both were based on one case) in the cariprazine arm for state 3, the posterior probability distribution reflects equal influence of the prior probabilities and clinical trial data. As the same prior was used for both treatment arms, and there were no observations in states 7 and 8 in the initial 4 weeks of the trial, the posterior distributions of the transition probabilities from these health states were essentially the same in the two arms.
When the informative priors were used, these distributions reflected the opinions of the experts and were not at all uniform, avoiding the bias that would have resulted from the use of vague priors that assumed that the probability of each transition was the same from these health states (i.e., 1/8).
Table 4 shows the results regarding the health benefits with the use of the three different transition probability matrices. The estimated difference in QALYs was small but not negligible over a 54-week-long time period. Compared to the model that incorporated expert opinions, the model including only the clinical trial data estimated 2.8% higher, while the model using vague priors estimated 4% smaller QALY differences between the treatment arms. Nevertheless, these differences were very small compared to the precision of the QALY difference estimate, as the standard deviation of it was 0.014 after 1000 model runs in the probability sensitivity analysis in the cost-effectiveness analysis.