The Effect of Baseline AK Counts on Efficacy Endpoints over Time
Actinic keratoses were counted at day 1 (baseline) and at weeks 4 and 8 in pooled Studies 1 and 2. The histograms of the AK counts at these visits are shown in Fig. 1. From day 1 to week 4, the AK count in the vehicle group can be seen to shift to the left and the mean number of AKs to fall by 16.2%. This “vehicle effect” is probably a regression to the mean effect: patients with fewer than five AKs were excluded from inclusion in the study. If the disease varies, the variation in itself will lead to a flattening of the histogram. This interpretation is supported by the observation that the histogram does not change from week 4 to week 8; the mean is unchanged and the distribution has the same shape. Thus, in the course of 1 month there is no measurable overall disease progression in the vehicle group.
Although in the ingenol mebutate gel population efficacy as assessed by % reduction in the AK count is around 74% at weeks 4 and 8, the proportion of patients who have no AKs, i.e., complete clearance, is around 24%. The histograms clearly show that completely cleared patients (the 0 column) are not the only ones who benefit from the treatment: the whole distribution is shifted to the left, indicating a reduction in the AK counts.
Furthermore, the efficacy endpoint of around 74% reduction in AK count in the ingenol mebutate treatment group at weeks 4 and 8 is obviously not simply due to “flattening” of the distribution, which only resulted in an approximately 16% reduction in the vehicle group.
In Fig. 1 the net changes in % reduction and complete clearance in the AK counts between weeks 4 and 8 are small. However, even though there is no net change, patients can move between the categories. The shaded areas in Fig. 1 illustrate this shifting: the percentage of patients who were completely cleared at week 4 are shown in black in all figure parts. Around 60% of the ingenol mebutate-treated patients who were completely clear at week 4 (black 0 column at week 4) are to be found in the 0 column at week 8. The remaining 40% are spread in the non-completely cleared columns at week 8. The black part of the 0-column at week 8 represents patients who had a zero count at both week 4 and week 8.
The complete clearance data for the ingenol mebutate group for weeks 4 and 8 are shown in Table 1. The overall agreement is 0.80 (95% CI 0.78–0.84), the positive agreement is 0.59 (95% CI 0.55–0.67), and the negative agreement is 0.87 (95% CI 0.86–0.90).
A scatterplot of the counts at the two visits (weeks 4 and 8) is shown in Fig. 2. The size of the dots reflects the number of patients. It is clear from the figures that the paired counts are associated, but it is also clear that patients do not always stay in their category. If they did, all dots would be on the identity line. This applies to both vehicle and ingenol mebutate gel patients.
Dependency of the Complete Clearance Rate on the Baseline AK Number
As has been reported by Szeimies et al.  and Dirschka et al. . complete clearance depends among other things on the number of baseline AKs and the size of the treatment area. The aggregated data for ingenol mebutate treatment on the face and scalp on small and large areas is given in Table 2; the baseline AK counts range from 4 to 20. The % reduction in AK counts is virtually unaffected by the number of baseline AKs, the complete clearance rate falls with increasing baseline AK count, and the absolute difference in AK count increases. With four baseline AKs the complete clearance rate is around 45%, and the number of AKs removed by the treatment is approximately three. At the other end of the spectrum, 14 AKs are removed by the treatment and the complete clearance rate is around 5%. Thus, paradoxically, the clinical benefit for the patient increases as the complete clearance decreases. The same data broken down by each baseline count from 4 to 20 are shown in Fig. 3, and the findings are remarkably consistent with those given in Table 2.
The Recurrence Endpoint Is Inaccurate and Should Be Replaced with % Reduction from Baseline
The standard for estimating the long-term effect after a treatment is completed is based on patients who obtained complete clearance at the end of treatment. This means that only those who fail (get a recurrence) are taken into account, whereas those who were not clear at the end of the treatment but would have subsequently cleared are ignored. In the example investigated in this article (Fig. 1, Table 1), if the end of treatment were to have been at week 4, the patients who were cleared at week 8 but not at the assessment at the end of treatment at week 4 would not have been included, and a recurrence rate of 77/(116 + 77) = 40% would have been computed at week 8 when in reality the proportion of patients who are completely clear at the two visits is almost identical: (116 + 77)/(116 + 77 + 70 + 520) = 25% at week 4 and (116 + 70)/(116 + 77 + 70 + 520) = 24% at week 8.
In addition, with subsequent repeat assessments over time of only those who were completely cleared at the previous visit, the recurrence rate will be further inflated.
A more appropriate assessment of long-term efficacy would include AK counts for all patients at all visits, whether cleared or not at the end of treatment efficacy assessment, preferably presented as the percentage reduction relative to the baseline AK count.