Notes on the methods
Estimation of intra-subject variability
The EMA proposed a linear model of log-transformed PK responses of the reference treatment
sequence, subject(sequence), period
where all effects are fixed. Estimated via function lm() of library stats:
For informational purposes in full replicate designs (required by the WHO for reference-scaling of AUC) the same model is run with data = data[data$treatment = "T", ].
Special conditions for the sample size in three period full replicate designs:
The question raised asks if it is possible to use a design where subjects are randomised to receive treatments in the order of TRT or RTR.
[…] if a 3-period replicate design, where treatments are given in the order TRT or RTR, is to be used to justify widening of a confidence interval for Cmax then it is considered that at least 12 patients would need to provide data from the RTR arm. This implies a study with at least 24 patients in total would be required if equal number of subjects are allocated to the 2 treatment sequences.
— Q&A document (June 2015 and later revisions)
If less than twelve subjects remain in sequence RTR of a TRT | RTR design (and in analogy in sequence TRR of a TRR | RTT design), the user is notified about the uncertain estimate of CVwR. However, in a sufficiently powered study such a case is extremely unlikely.
Model structure
The EMA’s models assumes equal [sic] intra-subject variances of Test and Reference (like in 2×2×2 trials) – even if proven false in one of the full replicate designs (were both CVwT and CVwR can be estimated). Hence, amongst biostatisticians it is called the ‘crippled model’ because the replicative nature of the study is ignored.
The nested structure subject(sequence) of the methods leads to an over-specificed model.8 The simple model
sequence, subject, period, treatment
gives identical estimates of the residual variance and the treatment effect and hence, its confidence interval.
The same holds true for the EMA’s model to estimate CVwR. The simple model
subject, period
gives an identical estimate of the residual variance.
Reference-scaling is acceptable for Cmax (immediate release products9) and Cmax, Cmax,ss, Cτ,ss, partialAUC (modified release products10) if high variability of the reference product (CVwR >30%) was demonstrated.
The intention to widen the limits has to be stated in the protocol and – contrary to the FDA’s RSABE – a clinical justification provided.
Those HVDP for which a wider difference in Cmax is considered clinically irrelevant based on a sound clinical justification can be assessed with a widened acceptance range. The request for widened interval must be prospectively specified in the protocol.
— BE Guideline (January 2010)
BE limits, PE restriction, rounding issues
The limits can be expanded based on CVwR.
- CVwR ≤ 30%
Lower cap, i.e., no scaling, conventional limits:
\(\left[ {L,U} \right] = {80.00 - 125.00\%}\) - 30% < CVwR ≤ 50%
Expanded limits based on \(s_{wR}\):
\(\left[ {L,U} \right] = 100{e^{ \mp 0.760 \cdot {s_{wR}}}}\) - CVwR > 50%
Upper cap, i.e., applying \(s^*_{wR}=\sqrt{log(0.50^2+1)}\) in the expansion formula or
\(\left[ {L,U} \right] = {69.84 - 143.19\%}\).
In reference-scaling a so-called mixed (a.k.a. aggregate) criterion is applied. In order to pass BE,
- the 90% confidence interval has to lie entirely within the acceptance range \(\left[ {L,U} \right]\) and
- the point estimate has to lie within 80.00 – 125.00%.
In order to avoid discontinuities due to double rounding, expanded limits are calculated in full numeric precision and only the confidence interval is rounded according to the GL.
# Calculate limits with library PowerTOST
CV <- c(29, 30, 40, 50, 51)
df <- data.frame(CV = CV, L = NA, U = NA, cap = "",
stringsAsFactors = FALSE)
for (i in seq_along(CV)) {
df[i, 2:3] <- sprintf("%.8f", PowerTOST:::scABEL(CV[i]/100)*100)
}
df$cap[df$CV <= 30] <- "lower"
df$cap[df$CV >= 50] <- "upper"
names(df)[1:3] <- c("CV(%)", "L(%)", "U(%)")
print(df, row.names = FALSE)
#> CV(%) L(%) U(%) cap
#> 29 80.00000000 125.00000000 lower
#> 30 80.00000000 125.00000000 lower
#> 40 74.61770240 134.01645559
#> 50 69.83678198 143.19101936 upper
#> 51 69.83678198 143.19101936 upper
Discrete limits resulting from rounding
Degrees of freedom, comparison of methods
The SAS code provided by the EMA in the Q&A document does not specify how the degrees of freedom should be calculated in ‘Method B’. Hence, the default in PROC MIXED, namely DDFM=CONTAIN is applied, i.e., method.B(..., option = 2). For incomplete data (missing periods) Satterthwaite’s approximation of the degrees of freedom, i.e., method.B(..., option = 1) or Kenward-Roger method.B(..., option = 3) might be a better choice – if stated as such in the SAP.
The EMA seemingly prefers ‘Method A’:
A simple linear mixed model, which assumes identical within-subject variability (Method B), may be acceptable as long as results obtained with the two methods do not lead to different regulatory decisions. However, in borderline cases […] additional analysis using Method A might be required.
— Q&A document (January 2011 and later revisions)
The half-width of the confidence interval in log-scale allows a comparison of methods (B v.s. A) where a higher value might point towards a more conservative decision.11 In the provided example data sets – with one exception – the conclusion of BE (based on the mixed criterion) agrees between ‘Method A’ and ‘Method B’.
However, for the highly incomplete data set 14 ‘Method B’ was liberal (passing by ANOVA but failing by the mixed effects model):
# Compare Method B acc. to the GL with Method A for all reference data sets.
ds <- substr(grep("rds", unname(unlist(data(package = "replicateBE"))),
value = TRUE), start = 1, stop = 5)
for (i in seq_along(ds)) {
A <- method.A(print=FALSE, details = TRUE, data = eval(parse(text = ds[i])))$BE
B <- method.B(print=FALSE, details = TRUE, data = eval(parse(text = ds[i])))$BE
r <- paste0("A ", A, ", B ", B, " \u2013 ")
cat(paste0(ds[i], ":"), r)
if (A == B) {
cat("Methods agree.\n")
} else {
if (A == "fail" & B == "pass") {
cat("Method A is conservative.\n")
} else {
cat("Method B is conservative.\n")
}
}
}
#> rds01: A pass, B pass – Methods agree.
#> rds02: A pass, B pass – Methods agree.
#> rds03: A pass, B pass – Methods agree.
#> rds04: A fail, B fail – Methods agree.
#> rds05: A pass, B pass – Methods agree.
#> rds06: A pass, B pass – Methods agree.
#> rds07: A pass, B pass – Methods agree.
#> rds08: A pass, B pass – Methods agree.
#> rds09: A pass, B pass – Methods agree.
#> rds10: A pass, B pass – Methods agree.
#> rds11: A pass, B pass – Methods agree.
#> rds12: A fail, B fail – Methods agree.
#> rds13: A fail, B fail – Methods agree.
#> rds14: A pass, B fail – Method B is conservative.
#> rds15: A fail, B fail – Methods agree.
#> rds16: A fail, B fail – Methods agree.
#> rds17: A fail, B fail – Methods agree.
#> rds18: A fail, B fail – Methods agree.
#> rds19: A fail, B fail – Methods agree.
#> rds20: A fail, B fail – Methods agree.
#> rds21: A fail, B fail – Methods agree.
#> rds22: A pass, B pass – Methods agree.
#> rds23: A pass, B pass – Methods agree.
#> rds24: A pass, B pass – Methods agree.
#> rds25: A pass, B pass – Methods agree.
#> rds26: A fail, B fail – Methods agree.
#> rds27: A pass, B pass – Methods agree.
#> rds28: A pass, B pass – Methods agree.Exploring data set 14:
A <- method.A(print = FALSE, details = TRUE, data = rds14)
B2 <- method.B(print = FALSE, details = TRUE, data = rds14, option = 2)
B1 <- method.B(print = FALSE, details = TRUE, data = rds14, option = 1)
B3 <- method.B(print = FALSE, details = TRUE, data = rds14, option = 3)
# Rounding of CI according to the GL
A[15:19] <- round(A[15:19], 2) # all effects fixed
B1[15:19] <- round(B1[15:19], 2) # Satterthwaite's df
B2[15:19] <- round(B2[15:19], 2) # df acc. to Q&A
B3[15:19] <- round(B3[15:19], 2) # Kenward-Roger df
cs <- c(2, 10, 15:23)
df <- rbind(A[cs], B1[cs], B2[cs], B3[cs])
names(df)[c(1, 11)] <- c("Meth.", "hw")
df[, c(2, 11)] <- signif(df[, c(2, 11)], 5)
print(df[order(df$BE, df$hw, decreasing = c(FALSE, TRUE)), ],
row.names = FALSE)
#> Meth. DF EL.lo(%) EL.hi(%) CI.lo(%) CI.hi(%) PE(%) CI GMR BE
#> B-1 197.44 69.84 143.19 69.21 121.27 91.62 fail pass fail
#> B-2 192.00 69.84 143.19 69.21 121.28 91.62 fail pass fail
#> B-3 195.99 69.84 143.19 69.21 121.28 91.62 fail pass fail
#> A 192.00 69.84 143.19 69.99 123.17 92.85 pass pass pass
#> hw
#> 0.28043
#> 0.28046
#> 0.28052
#> 0.28261All variants of ‘Method B’ are more conservative than ‘Method A’. Before rounding the confidence interval, option = 2 with 192 degrees of freedom would be more conservative (lower CL 69.21029) than option = 1 with 197.44 degrees of freedom (lower CL 69.21286). However, given the incompleteness of this data set (four missings in period 2, twelve in period 3, and 19 in period 4), Satterthwaite’s or Kenward-Roger degrees of freedom probably are the better choice.
Outlier analysis
It is an open issue how outliers should be handled.
The applicant should justify that the calculated intra-subject variability is a reliable estimate and that it is not the result of outliers.
Box plots were ‘suggested’ by the author as a joke [sic] at the EGA/EMA workshop, being aware of their nonparametric nature and the EMA’s reluctance towards robust methods. Unfortunately, this joke was included in the Q&A document.
[…] a study could be acceptable if the bioequivalence requirements are met both including the outlier subject (using the scaled average bioequivalence approach and the within-subject CV with this subject) and after exclusion of the outlier (using the within-subject CV without this subject).
An outlier test is not an expectation of the medicines agencies but outliers could be shown by a box plot. This would allow the medicines agencies to compare the data between them.
— Q&A
With the additional argument ola = TRUE in method.A() and method.B() an outlier analysis is performed, where the default fence = 212 is used.
Results slightly differ depending on software’s algorithms to calculate the median and quartiles. Example with the methods implemented in R:
### Compare different types with some random data
x <- rnorm(48)
p <- c(25, 50, 75)/100
q <- matrix(data = "", nrow = 9, ncol = 4,
dimnames=list(paste("type", 1:9),
c("1st quart.", "median", "3rd quart.",
"software / default")))
for (i in 1:9) {
q[i, 1:3] <- sprintf("%.5f", quantile(x, prob = p, type = i))
}
q[c(2, 4:8), 4] <- c("SAS, STaTa", "SciPy", "",
"Phoenix, Minitab, SPSS",
"R, S, MATLAB, Octave, Excel", "Maple")
print(as.data.frame(q))
#> 1st quart. median 3rd quart. software / default
#> type 1 -0.70377 -0.17472 0.63750
#> type 2 -0.65978 -0.14859 0.65125 SAS, STaTa
#> type 3 -0.70377 -0.17472 0.63750
#> type 4 -0.70377 -0.17472 0.63750 SciPy
#> type 5 -0.65978 -0.14859 0.65125
#> type 6 -0.68178 -0.14859 0.65812 Phoenix, Minitab, SPSS
#> type 7 -0.63779 -0.14859 0.64437 R, S, MATLAB, Octave, Excel
#> type 8 -0.66711 -0.14859 0.65354 Maple
#> type 9 -0.66528 -0.14859 0.65296- Box plots of studentized13 and standarized14 model residuals are constructed.15
- Potential outliers are flagged based on the argument
fenceprovided by the user. - With the additional argument
verbose = TRUEdetailed information is shown in the console.
Example for the reference data set 01:
Outlier analysis
(externally) studentized residuals
Limits (2×IQR whiskers): -1.717435, 1.877877
Outliers:
subject sequence stud.res
45 RTRT -6.656940
52 RTRT 3.453122
standarized (internally studentized) residuals
Limits (2×IQR whiskers): -1.69433, 1.845333
Outliers:
subject sequence stand.res
45 RTRT -5.246293
52 RTRT 3.214663If based on studentized residuals outliers are detected, additionally to the scaled limits based on the complete reference data, tighter limits are calculated based on CVwR after exclusion of outliers and BE assessed with the new limits. Standardized residuals are shown for informational purposes only and are not used for exclusion of outliers.
Output for the reference data set 01 (re-ordered for clarity):
CVwR : 46.96% (reference-scaling applicable)
swR : 0.44645
Expanded limits : 71.23% ... 140.40% [100exp(±0.760·swR)]
Assessment based on original CVwR 46.96%
────────────────────────────────────────
Confidence interval: 107.11% ... 124.89% pass
Point estimate : 115.66% pass
Mixed (CI & PE) : pass
╟────────┼─────────────────────┼───────■────────◊─────────■───────────────╢
Outlier fence : 2×IQR of studentized residuals.
Recalculation due to presence of 2 outliers (subj. 45|52)
─────────────────────────────────────────────────────────
CVwR (outl. excl.) : 32.16% (reference-scaling applicable)
swR (recalculated) : 0.31374
Expanded limits : 78.79% ... 126.93% [100exp(±0.760·swR)]
Assessment based on recalculated CVwR 32.16%
────────────────────────────────────────────
Confidence interval: pass
Point estimate : pass
Mixed (CI & PE) : pass
╟┼─────────────────────┼───────■────────◊─────────■─╢Note that the PE and its CI are not affected since the entire data are used and therefore, these values not reported in the second analysis (only the conclusion of the assessement).
The line plot is given for informational puposes (its resolution is only ~0.5%). The filled squares ■ are the lower and upper 90% confidence limits, the rhombus ◊ the point estimate, the vertical lines │ at 100% and the PE restriction (80.00 – 125.00%), and the double vertical lines ║ the expanded limits. The PE and CI take presedence over other symbols. In this case the upper limit of the PE restriction is not visible.
Since both analyses arrive at the same conclusion, the study should be acceptable according to the Q&A document.