Changes to trial protocol

Item 10: Important changes to the trial after it commenced including any outcomes or analyses that were not prespecified, with reason

Examples

“The original primary endpoint was all-cause mortality, but, during a masked analysis, the data and safety monitoring board noted that overall mortality was lower than had been predicted and that the study could not be completed with the sample size and power originally planned. The steering committee therefore decided to adopt co-primary endpoints of all-cause mortality (the original primary endpoint), together with all-cause mortality or cardiovascular hospital admissions (the first prespecified secondary endpoint)” [170].

“As described in the published protocol paper, a protocol amendment was made to revise the sample size in response to new information on the minimal clinically important difference of the primary outcome measure, the Patient-Oriented Eczema Measure (POEM). Our original sample size used a POEM score for minimal clinically important difference of 3, which was based on research carried out in secondary care among people with moderate or severe eczema. Fresh evidence, however, suggested that a change in POEM score of 2.1 to 2.9 represents a change likely to be beyond measurement error. A protocol amendment was therefore made to change the target sample size for the trials based on seeking to detect a difference in POEM score of 2.5 points between groups, increasing the target sample size from 200 to 303 for each trial” [171].

Explanation

A protocol for a randomised trial serves as the foundation for planning, conduct, reporting, and appraisal, specifying in detail how the trial will be conducted. There may be deviations from the original protocol, as it is impossible to predict every possible change in circumstances during the course of a trial. Some trials will therefore have important changes to the methods after trial commencement. There are many reasons for deviations from the initial study protocol; for example, changes could be due to external information becoming available from other studies, or internal financial difficulties, or lower than anticipated recruitment rates. In some trials, an independent data monitoring committee will have as part of its remit the possibility of recommending protocol changes based on seeing unblinded data. Authors should report the nature and timing of protocol changes because changes made at different times (eg, before or after breaking the blinding) might be associated with different risks of bias.

Authors should report all major changes to the trial after it commenced indicating the reason for the changes and when the changes occurred [172]. Such changes might affect the trial methods, such as the randomisation ratio, eligibility criteria, interventions, outcomes, method of analysis or duration of follow-up; or might affect the trial conduct, such as dropping a trial site with poor data quality [173].

Some trials are set up with a formal adaptive design, which allows pre-planned changes to an ongoing trial without compromising the validity of conclusions [174]. It is therefore essential to distinguish pre-planned changes from unplanned changes that may also occur. Such adaptive trial design modifications are usually to the sample size and number of treatment groups, and can lead to decisions being made more quickly and with more efficient use of resources than would be possible with traditional, non-adaptive parallel group trials. Specific guidance has been developed for reporting trials with a formal adaptive design; authors could consult this for more detailed information [174].

Most trials record multiple outcomes, with the risk that results will be reported for only a selected subset. Prespecification and reporting of completely defined primary and secondary outcomes for both benefits and harms (item 14) should remove such a risk. In some trials, however, circumstances require a change in the way an outcome is assessed, the designation of outcomes as primary or secondary or even, as in the example above, a switch to a different outcome. For example, there may be external evidence from other trials or systematic reviews suggesting the time point for the primary outcome might not be appropriate; or recruitment or the overall event rate in the trial may be lower than expected [170]. Changing an endpoint based on unblinded data are much more problematic, although may be specified in the context of an adaptive trial design [174].

Whether the modifications are explicitly part of the trial design or in response to changing circumstances, it is essential that they are fully reported and the reason for the change explained to help the reader interpret the results. Such information is not always reported. A comparison of protocols and publications of 102 randomised trials found that 62% of trial reports had at least one primary outcome that was changed, introduced, or omitted compared with the protocol [51]. Primary outcomes also differed between protocols and publications for 40% of a cohort of 48 trials funded by the Canadian Institutes of Health Research [175]. None of these subsequent 150 trial reports mentioned, let alone explained, changes from the protocol. Similar results from other studies have been reported in a systematic review of empirical studies, comparing trial registers or protocols to published trial reports [176].