What can we learn from publishing endoscopy trial protocols?

Abstract

Clinical trials in medicine have come under public scrutiny as never before in the last decade with scandal after scandal about misrepresentation or misinterpretation of results leading to erroneous clinical conclusions. High-profile campaigns have been started by the British Medical Journal and others to allow access to original trial data, even down to the original case record forms, so that trials can be reanalyzed where foul play is suspected. In some cases, this has altered the primary outcome of the trial1; however, more subtle changes to trial analysis can occur during the study process. A common issue is that it is harder to publish negative trials than positive ones, which shouldn't be the case if the original trial was correctly designed to answer the clinical question with sufficient statistical precision. Nevertheless, it is not uncommon to see a secondary outcome or a per protocol analysis (rather than the more rigorous intention-to-treat analysis) that is statistically significant rise to prominence in the study report as the key finding, despite the trial being powered for a different primary outcome measure. The lack of correction for multiple testing of the data is another common omission, sometimes justified by describing such analysis as exploratory or hypothesis generating. While acceptable, clinical inference cannot be drawn for such uncorrected statistical presentation, nor should these results be presented as main trial outcomes. Endoscopic trials are certainly not immune from these phenomena, although the level of commercial input and drive has been less than seen with the millions of dollars that pharmaceutical companies have riding on a successful new drug. Endoscopic trials have often been single-center and often single-operator, with or without minimal funding, save for the time and enthusiasm of a committed endoscopic innovator. Such trials are often of very limited generalizability and essentially simply represent phase one safety studies. More recently much larger endoscopic studies have been developed that are multicenter, multi-operator and often international. These are phase two safety and efficacy studies, with some phase three regulatory studies now occurring and subsequent phase four post-marketing registries now also seen. This represents a maturing of endoscopic research approaches with grant funding for some of these larger trials, which entails formal peer review. Detailed peer review by grant funding bodies is to be commended, as it usually strengthens studies by making them more conservative in their assumptions, especially around recruitment, which makes completion of recruitment (and hence delivery of planned statistical power) more likely, requires patient involvement, and ensures the statistical analysis plan is rigorous and predefined, with a clear primary outcome measure. In many areas of the world there is an expectation that to secure grant funding, a clinical trials unit will be involved as part of the study team and help with trial design and ensure trial logistics can be delivered. Industry-delivered studies increasingly involve clinical trials units for endoscopic studies as well a clinical research organizations to ensure delivery on the substantial investment that is required. As medical devices come under increasing scrutiny, and companies wish to make regulatory claims about devices, this is likely to become a bigger part of endoscopic research 2. Transparency has already become part of publishing endoscopic studies with all major gastrointestinal and endoscopic journals requiring pre-recruitment trial registration to publish studies 3. The purpose of this registration is mainly to publicly disclose study rationale and ethical considerations before recruiting subjects into the trial. Moreover, these registries minimize publication bias, since even high-quality studies may end up unpublished due to negative or unwanted results. This raises the question whether we actually need publications of endoscopy trials protocols and what we can learn from them. Publishing endoscopy study protocols can help improve the quality of endoscopy research in multiple ways 4. First, it allows researchers to obtain feedback on study protocols through peer review. Although in studies that have received ethics approval or a grant from a major funding body, feedback on draft protocols is preexisting, there is no doubt that even with the most elegantly designed study, it may be difficult to defend controversial elements in the protocol. Moreover, protocol publication enables readers to compare what was originally intended with what was actually executed. That is of great importance, preventing the manipulation or dredging of data. It also minimizes post-hoc revision of study aims and endpoints (Table 1). Table 1 Key reasons to publish trial protocols. Ensure transparency of and detail in study methodology Provide feedback to research team Minimize manipulation of data or endpoints Provide assistance to systemic reviewers Help patients and investigators identify trials under way Furthermore, publishing studies protocols allows for researchers and funders to understand what studies are under way, thus minimizing unnecessary duplication of research effort. In addition, it enables systemic reviewers to find trials, which may in turn reduce distortion from publication bias. Finally, publishing endoscopy study protocols enables patients to see what studies are under way for which they may consider volunteering. The publication of the ADENOMA trial study protocol for colonoscopy with or without endocuff with adenoma detection rate as the primary outcome measure is the first study protocol to be published in Endoscopy International Open 5. It is a large multicenter study with public and industry funding, and is supported by a clinical trials unit. As such, it represents the leading edge of a new wave of high-quality endoscopic studies which are potentially widely practice changing, hence there is a need for special rigor and oversight of them. Is publication of the trial protocol worth the paper (electronic or otherwise) it is written on? To some extent, that judgment can only be made when the final study report is published, as the protocol allows us to establish what the study team originally proposed, and compare it to what was delivered and reported. Importantly, the level of methodological detail in a protocol is far greater than the few hundred words usually devoted to methodology in a standard publication and allows us to drill down into the detail of the claims made. Examples here include details of masking and allocation concealment: The study is not blinded, since by nature of the intervention the allocation is known to the endoscopist and allocation concealment cannot be offered to the personnel who will register the outcomes. A degree of bias favoring the intervention arm cannot be excluded when identifying or purposely “ignoring” polyps (especially since colonoscopy is not video recorded or evaluated by a second endoscopist), or when recording time to insertion to cecum. The same applies to endoscopy nurses when recording patient satisfaction and discomfort score during colonoscopy. How withdraws and dropouts are handled is also important. During a colonoscopy procedure, the endoscopist can remove the endocuff if there is an acute angulation in a fixed sigmoid, preventing further scope advancement. It is important that these patients are not excluded from the final analysis so as to ensure that the Intention-to-Treat (ITT) analysis is preserved. Some of these more subtle aspects of the study design can easily be lost in a brief methodological summary. Ultimately we believe that publication of trial protocols for endoscopic studies improves the transparency and accountability of our research, making it the best possible standard upon which to strive to improve care for our patients. We have seen the movement to study protocol publication in other major clinical specialties, notably cardiology and oncology. It is time for endoscopy to step up to the same level of research excellence to drive rigorous widespread practice changes based on the highest quality evidence 2.