Example

​

“The primary analyses for symptom resolution will be investigated to determine whether treatment effectiveness differs according to the following subgroups:

1. Presence of concomitant STI [sexually transmitted infection] (yes/no)

2. BV [bacterial vaginosis]confirmed by positive microscopy (yes/no)

3. Type of centre at which participant presented (sexual health clinic versus GP/other clinics)

Between-group treatment effects will be provided for each subgroup, but interpretation of any subgroup effects will be based on the treatment subgroup interaction and 95% CI, estimated by fitting an appropriate interaction term in the regression models. Since the trial is powered to detect overall differences between the groups rather than interactions of this kind, these subgroup analyses will be regarded as exploratory ” [437].

 

“Sensitivity analyses will be performed, based on the assumption that the missing outcomes are the worst or best possible in the different randomization groups. If these show that the conclusions may differ based on the missing values, further multiple imputation will be performed for the missing values. These analyses will consider the results of any losses at follow-up to the extent that they relate to differences in the measured variables (i.e., under the assumption of random missingness) ” [438].

​

Explanation

​

Subgroup analyses are planned to look for evidence of whether the intervention effect varies between different subgroups (e.g., younger and older participants, or men and women). Sensitivity analyses are planned to examine the robustness of the primary analysis to different assumptions or analytical approaches.

 

Examining differences in subgroups involves assessing the statistical interaction between a variable and the intervention effect [439]. Interactions can be presented by showing results on a figure (along with relevant information, such as estimates and confidence interval for each subgroup, as well as the interaction p-value.

 

Though commonly performed [440, 441], subgroup analyses are highly susceptible to bias and misinterpretation, particularly if they are inappropriately conducted or selectively reported. Post hoc subgroup analyses (i.e., performed after looking at the data) should be avoided due to the high risk of spurious (false positive) findings [442]. For the same reason, it is problematic to conduct a large number of subgroup analyses, even if prespecified. Defining subgroups based on variables measured after randomisation is susceptible to bias and should be avoided [443]. Another common mistake is to conclude subgroup differences based on a statistically non-significant p-value in one subgroup versus a statistically significant p-value in another subgroup, or based on assessing whether confidence intervals overlap. Well-documented subgroup differences tend to be far less common than claims made for such differences [444].

 

Another consideration is the practice of categorising continuous variables to define subgroups. Although categorisation is frequently used for perceived simplicity and ease of communication, this approach has notable drawbacks. Cut-points to define subgroups are often arbitrarily chosen without clinical or biological rationale. Categorisation also leads to the loss of information and a reduction of statistical power.

 

Among protocols published from 2006–2017 or approved in 2012 and 2016, 20% to 36% described subgroup analyses but often failed to report key details including the rationale (unreported for 83% to 96% of protocols that described subgroup analyses), anticipated direction of effect (85% to 100%), specific variables (17% to 27%), tests for interaction (67% to 73%), and statistical power considerations (88% to 97%) [440, 441]. Reviews have found that for over half of trials, the subgroup analyses reported in publications do not match those described in protocols [64, 441], raising the concerns noted above about post hoc analyses and selective reporting.

 

The rationale for examining any subgroups should be outlined in the study protocol, including which baseline variables will be explored and the hypothesized direction of the subgroup effect based on plausibility [445]. If continuous baseline variables are to be categorised, then the rationale and the cut-points should be stated. All statistical methods used to analyse any subgroups should be clearly described.

 

Sensitivity analyses have a different aim than subgroup analysis. Sensitivity analyses are planned to examine whether the (primary) trial results vary substantially under a range of different assumptions about the data, methods, and models [446, 447]. Sensitivity analyses are often planned to explore the impact of missing data and any assumptions made in the primary analysis on the handling of missing data (Item 27c). When the findings from a sensitivity analysis are consistent with the primary results, there can be increased confidence that the primary results are valid [446, 448].

 

The protocol should clearly describe all planned sensitivity analyses, their rationale, and the methods to be used.

 

Summary of key elements to address

For any planned subgroup analyses:

• Baseline variables to be explored

• Rationale

• Statistical methods (e.g., test of interaction)

• Cut-points and rationale for categorisation of continuous baseline variables (if applicable)

For any planned sensitivity analyses:

• Rationale

• Statistical methods