Performance of the Population Bioequivalence (PBE) Statistical Test Using an IPAC-RS Database of Delivered Dose from Metered Dose Inhalers

Abstract

This article reports performance characteristics of the population bioequivalence (PBE) statistical test recommended by the US Food and Drug Administration (FDA) for orally inhaled products. A PBE Working Group of the International Pharmaceutical Aerosol Consortium on Regulation and Science (IPAC-RS) assembled and considered a database comprising delivered dose measurements from 856 individual batches across 20 metered dose inhaler products submitted by industry. A review of the industry dataset identified variability between batches and a systematic lifestage effect that was not included in the FDA-prescribed model for PBE. A simulation study was designed to understand PBE performance when factors identified in the industry database were present. Neglecting between-batch variability in the PBE model inflated errors in the equivalence conclusion: (i) The probability of incorrectly concluding equivalence (type I error) often exceeded 15% for non-zero between-batch variability, and (ii) the probability of incorrectly rejecting equivalence (type II error) for identical products approached 20% when product and between-batch variabilities were high. Neglecting a systematic through-life increase in the PBE model did not substantially impact PBE performance for the magnitude of lifestage effect considered. Extreme values were present in 80% of the industry products considered, with low-dose extremes having a larger impact on equivalence conclusions. The dataset did not support the need for log-transformation prior to analysis, as requested by FDA. Log-transformation resulted in equivalence conclusions that depended on the direction of product mean differences. These results highlight a need for further refinement of in vitro equivalence methodology.

Appendix

Equation 2 Full simulation model

$$ {\displaystyle \begin{array}{l}{y}_{ijkl}=\alpha +{\tau}_k+{B}_i+{C}_{j(i)}+B{\tau}_{ik}+C{\tau}_{j(i)k}+{\varepsilon}_{ijkl}\\ {}i:\mathrm{Batch}\ \mathrm{index},\kern0.5em j:\mathrm{Can}\ \mathrm{index},\kern0.5em k:\mathrm{Lifestage},\kern0.62em l:\operatorname{Re}\mathrm{plicate}\\ {}\alpha =\mathrm{Product}\ \mathrm{mean}\\ {}{\tau}_k=\mathrm{Lifestage}\ \mathrm{effect}\ \mathrm{for}\ \mathrm{the}\ \mathrm{kth}\ \mathrm{lifestage}\\ {}{B}_i=N\left(0,{\sigma^2}_{\mathrm{B}}\right)\ \mathrm{Random}\ \mathrm{batch}\ \mathrm{effect}\\ {}{C}_{j(i)}=N\left(0,{\sigma^2}_{\mathrm{C}}\right)\ \mathrm{Random}\ \mathrm{unit}\ \left(\mathrm{can}\right)\ \mathrm{effect}\\ {}B{\tau}_{ik}=N\left(0,{\sigma^2}_{\mathrm{B}\mathrm{L}}\right)\ \mathrm{Random}\ \mathrm{batch}\hbox{-} \mathrm{by}\hbox{-} \mathrm{lifestage}\ \mathrm{effect}\\ {}C{\tau}_{j(i)k}=N\left(0,{\sigma^2}_{\mathrm{C}\mathrm{L}}\right)\ \mathrm{Random}\ \mathrm{unit}\hbox{-} \mathrm{by}\hbox{-} \mathrm{lifestage}\ \mathrm{effect}\\ {}{\varepsilon}_{ijkl}=\left\{\begin{array}{l}N\left(0,{\sigma^2}_{\varepsilon}\right)\ \mathrm{with}\ \mathrm{probability}\ 1-{p}_{\mathrm{H}}-{p}_{\mathrm{L}}\left(\mathrm{random}\ \mathrm{error}\right)\\ {}\mathrm{Uniform}\ \left(M,N\right)\ \mathrm{with}\ \mathrm{probability}\ {p}_{\mathrm{H}}\left(\mathrm{high}\ \mathrm{extreme}\right)\\ {}\mathrm{Uniform}\ \left(R,S\right)\ \mathrm{with}\ \mathrm{probability}\ {p}_{\mathrm{L}}\left(\mathrm{low}\ \mathrm{extreme}\right)\end{array}\right.\\ {}\mathrm{where}\ R<S<0<M<N\\ {}{\sigma^2}_{\mathrm{Total}}={\sigma^2}_{\mathrm{B}}+{\sigma^2}_{\mathrm{C}}+{\sigma^2}_{\mathrm{B}\mathrm{L}}+{\sigma^2}_{\mathrm{C}\mathrm{L}}+{\sigma^2}_{\varepsilon}\end{array}} $$

(2)

Fig. A1

Histograms of simulated data from Normal (left) and Log-normal (right) distributions with (a) 10% between-batch variance, and (b) 50% between-batch variance. Data were simulated from Eq. 2 in the Appendix, assuming mean (α) = 100 and σ²_Total = 100, corresponding to relative standard deviation (RSD) = 10%, with three lifestages (n = 90) and no extreme values or lifestage effect

Fig. A2

Operating characteristic (OC) curves for data simulated from a normal (black) or log-normal (red) distribution with 10% (solid) or 50% (dashed) between-batch variance, assuming three lifestages, no extreme values or lifestage effect, and equal RSD between test and reference products. The long-dashed gray line shows the “ideal” OC curve that marks the PBE equivalence region for the given settings

Table III ᅟ

Table AII PBE Performance for Test and Reference Products with Equal Relative Standard Deviation (RSD), Percent Between-Batch Variability, Number of Lifestages and Lifestage Effect, and Presence of Extreme Values. Power: Proportion of Simulations That Conclude Bioequivalence, For [(Test Mean − Ref Mean)/Ref Mean. False Equivalence (FE): the Highest Proportion of (Incorrect) Bioequivalence Conclusions Outside the PBE Region for the Given Settings. The Max Standard Error for the Reported Power and FE Errors Were 0.007 and 0.007, Respectively.

Table AIII PBE Performance with Fixed Relative Standard Deviation (RSD, italics) or Fixed Standard Deviation (SD). In This Study, Test and Reference (Ref) Products Had Three Lifestages, No Lifestage Effect or Extremes, and Equal Percent Between-Batch (BB) Variability. False Equivalence (FE): the Highest Proportion of (Incorrect) Bioequivalence Conclusions Outside the PBE Equivalence Region for the Given Settings. FE Errors Marked with Asterisk Were Based on the Lower Bound (− 20%) of the PBE Region. The Max Standard Error for the Reported FE Error Was 0.007