--- title: "Re-estimating the Sample Size from Interim Data" author: "Gosuke Homma" date: "`r Sys.Date()`" output: rmarkdown::html_vignette: toc: true toc_depth: 2 vignette: > %\VignetteIndexEntry{Re-estimating the Sample Size from Interim Data} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 4.5, fig.align = "center" ) ``` ```{r load} library(bbssr) ``` ## Two questions, two functions `BinaryPowerBSSR()` answers a design question. Given a re-estimation rule, what is the power and the expected sample size across the scenarios that might occur? It averages over every possible interim outcome, and it is used before the trial starts. `BinaryBSSR()` answers an operational question. The interim analysis has produced one specific number, so how many more patients must be enrolled? It is used once, while the trial is running. This vignette follows a single trial through both. ## Planning A trial compares an experimental treatment against control with a binary response endpoint. The clinical team assumes a response probability of 0.45 under treatment and 0.20 under control, so a risk difference of 0.25. Randomization is one to one, the one-sided level is 0.025 and the target power is 0.80. The Z-pooled exact unconditional test is chosen. ```{r plan} Delta.A <- 0.25 plan <- BinarySampleSize(p1 = 0.45, p2 = 0.20, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Z-pool') plan ``` The assumed pooled response probability is `r 0.5 * (0.45 + 0.20)`. Its value also drives the required sample size, and unlike the risk difference it can be estimated from blinded data, which is what makes it worth re-estimating. ```{r sensitivity} pooled <- seq(0.20, 0.45, by = 0.05) sens <- data.frame( pooled = pooled, p1 = pooled + Delta.A / 2, p2 = pooled - Delta.A / 2 ) sens$N <- vapply(seq_len(nrow(sens)), function(i) { BinarySampleSize(sens$p1[i], sens$p2[i], 1, 0.025, 0.8, 'Chisq')$N }, numeric(1)) sens ``` The required total sample size goes from `r sens$N[1]` at a pooled probability of `r sens$pooled[1]` to `r sens$N[nrow(sens)]` at `r sens$pooled[nrow(sens)]`, while the risk difference stays fixed at `r Delta.A`. The reason is that the variance of a binary endpoint is largest at one half, so a pooled probability closer to that value costs patients. An error in the assumed pooled probability is therefore expensive, and it is exactly the quantity that blinded interim data can estimate. ## Evaluating the re-estimation rule Before committing to the design, check what the re-estimation buys. The interim analysis is placed at half of the planned sample size. The evaluations in this vignette use the chi-squared test, whose rejection regions are quicker to compute than those of the Z-pooled test, as an approximation to the final analysis. ```{r evaluate} design <- BinaryPowerBSSR( p = seq(0.20, 0.45, by = 0.05), Delta.A = Delta.A, Delta.T = Delta.A, N1 = plan$N1, N2 = plan$N2, omega = 0.5, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Chisq', restricted = TRUE ) design ``` ```{r evaluate-plot} plot(design) ``` The fixed-sample design was sized at a pooled probability of `r 0.5 * (0.45 + 0.20)`. It keeps its power at and below that value and loses it above, falling further behind as the pooled probability approaches one half. Under the restricted rule the re-estimation never goes below the planned sample size, so it leaves the power alone in the over-powered scenarios and adds patients in the under-powered ones. The `E.N` column shows what that costs. ## The interim analysis Enrolment reaches the interim milestone. The unblinded statistician is not involved. The only number extracted from the database is the total count of responders across both groups. ```{r interim-data} n1 <- ceiling(plan$N1 / 2) n2 <- ceiling(plan$N2 / 2) S <- round(0.40 * (n1 + n2)) data.frame(n1 = n1, n2 = n2, S = S, pooled.rate = round(S / (n1 + n2), 3)) ``` The observed pooled rate of `r round(S / (n1 + n2), 3)` is above the assumed `r 0.5 * (0.45 + 0.20)`. By the argument of the previous section a higher pooled rate at the same risk difference calls for more patients, so the trial has to grow. ```{r reestimate} re <- BinaryBSSR( n1 = n1, n2 = n2, S = S, Delta.A = Delta.A, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Z-pool', restricted = TRUE, N1 = plan$N1, N2 = plan$N2 ) re ``` The output separates three quantities that are easy to confuse. `N1.re` and `N2.re` are the total sample sizes the re-estimation calls for. `n1.stage2` and `n2.stage2` are the additional patients still to enrol, which is what the enrolment plan needs. `N1.final` and `N2.final` are what the trial will end up with, and they can exceed the re-estimated totals when the restricted rule applies or when the interim sample size already exceeds the requirement. ## What the decision looks like across interim outcomes Running the same call over the possible values of $S$ shows the shape of the rule. ```{r decision-curve} S.grid <- round(seq(0.15, 0.55, by = 0.05) * (n1 + n2)) decision <- data.frame(S = S.grid, pooled = round(S.grid / (n1 + n2), 3)) decision$N.restricted <- vapply(S.grid, function(s) { BinaryBSSR(n1, n2, s, Delta.A, 1, 0.025, 0.8, 'Chisq', restricted = TRUE, N1 = plan$N1, N2 = plan$N2)$N.final }, numeric(1)) decision$N.unrestricted <- vapply(S.grid, function(s) { BinaryBSSR(n1, n2, s, Delta.A, 1, 0.025, 0.8, 'Chisq')$N.final }, numeric(1)) decision ``` The restricted curve is flat at the planned total until the interim data are unfavourable enough to trigger an increase. The unrestricted curve dips below the plan when the interim data are favourable. The two curves meet once the required sample size exceeds the plan. ## Truncation at the boundary When the blinded pooled rate is extreme, the recovered group probabilities can fall outside the unit interval and are truncated. ```{r truncation} edge <- BinaryBSSR(n1 = 20, n2 = 20, S = 2, Delta.A = 0.4, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Chisq') data.frame(hat.p = edge$hat.p, hat.p1 = edge$hat.p1, hat.p2 = edge$hat.p2, N.re = edge$N.re) ``` With a pooled rate of `r edge$hat.p` and an assumed effect of 0.4, the control probability would be negative, so it is set to zero while the treatment probability stays at $\hat{p} + \Delta_{A} / 2$. The recovered risk difference is then only `r edge$hat.p1 - edge$hat.p2` rather than 0.4, and the re-estimated sample size is correspondingly larger. An assumed effect that the observed pooled rate cannot accommodate therefore produces conservative re-estimation, which is worth checking when the response rate is low. This truncation belongs to the default `ss.method = 'exact'`, whose search needs probabilities inside the unit interval. The normal approximation of `ss.method = 'standard'` keeps the recovered probabilities before truncation, with each Bernoulli variance truncated at zero, so the assumed effect is kept. ```{r truncation-standard} edge.standard <- BinaryBSSR(n1 = 20, n2 = 20, S = 2, Delta.A = 0.4, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Chisq', ss.method = 'standard') c(exact = edge$N.re, standard = edge.standard$N.re) ``` The `bbssr-reestimation-rules` vignette compares the two methods over all interim outcomes. ## Unequal allocation With an allocation ratio of $r$ to 1, the recovered probabilities are weighted accordingly and the second-stage enrolment follows the same ratio. ```{r allocation} BinaryBSSR(n1 = 40, n2 = 20, S = 18, Delta.A = 0.25, r = 2, alpha = 0.025, tar.power = 0.8, Test = 'Chisq') ``` ## Practical notes The interim total of responders is the only quantity that leaves the database, so the analysis stays blinded and no interim test is performed. The rule can still change the type I error rate, as the end of this section shows. The re-estimated sample size still depends on $\Delta_{A}$, which is fixed at the design stage and must not be revised in the light of the interim data. Fix the timing of the interim analysis, the value of $\Delta_{A}$, the test and the choice between the restricted and unrestricted rules in the protocol before the trial starts. Report the operating characteristics from `BinaryPowerBSSR()` alongside them, since the type I error rate of the procedure is a property of the whole rule rather than of the final test alone. Setting `Delta.T` to zero in `BinaryPowerBSSR()` makes the two groups identical, so `power.BSSR` and `power.TRAD` become rejection probabilities under the null hypothesis at the values passed through `p`. `BinaryTypeIErrorBSSR()` evaluates the same rejection probabilities over a grid of the common response probability and finds their largest value over the whole range of the grid, together with an upper bound, so the largest type I error rate does not have to be read off a grid. ```{r type1-check} null <- BinaryTypeIErrorBSSR( Delta.A = Delta.A, N1 = plan$N1, N2 = plan$N2, omega = 0.5, r = 1, alpha = 0.025, tar.power = 0.8, Test = 'Chisq', restricted = TRUE ) attr(null, 'max') ``` The largest type I error rate is `r signif(attr(null, 'max')$TIE[1], 4)` for the re-estimation design and `r signif(attr(null, 'max')$TIE[2], 4)` for the fixed-sample design, against the nominal 0.025. The chi-squared test is not exact, so the fixed-sample design alone can exceed the level. The `bbssr-type1-error` vignette shows how `BinaryAlphaAdjBSSR()` lowers the nominal level until the rate is controlled.