Underestimation of treatment effects in sequentially monitored clinical trials that did not stop early for benefit

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

In recent years, there has been a prominent discussion in the literature about the potential for overestimation of the treatment effect when a clinical trial stops at an interim analysis due to the experimental treatment showing a benefit over the control. However, there has been much less attention paid to the converse issue, namely, that sequentially monitored clinical trials which did not stop early for benefit tend to underestimate the treatment effect. In meta-analyses of many studies, these two sources of bias will tend to balance each other to produce an unbiased estimate of the treatment effect. However, for the interpretation of a single study in isolation, underestimation due to interim analysis may be an important consideration. In this paper, we discuss the nature of this underestimation, including theoretical and simulation results demonstrating that it can be substantial in some contexts. Furthermore, we show how a conditional approach to estimation, in which we condition on the study reaching its final analysis, may be used to validly inflate the observed treatment difference from a sequentially monitored clinical trial. Expressions for the conditional bias and information are derived, and these are used in supplied R code that computes the bias-adjusted estimate and an associated confidence interval. As well as simulation results demonstrating the validity of the methods, we present a data analysis example from a pivotal clinical trial in cardiovascular disease. The methods will be most useful when an unbiased treatment effect estimate is critical, such as in cost-effectiveness analysis or risk prediction.

Related collections

Most cited references 20

Record: found
Abstract: not found
Article: not found

The Design and Analysis of Sequential Clinical Trials.

J. Whitehead (1997)

0 comments Cited 48 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Cost effectiveness of thrombolytic therapy with tissue plasminogen activator as compared with streptokinase for acute myocardial infarction.

Doug Armstrong, Christopher P. Nelson, R Califf … (1995)

Patients with acute myocardial infarction who were treated with accelerated tissue plasminogen activator (t-PA) (given over a period of 1 1/2 hours rather than the conventional 3 hours, and with two thirds of the dose given in the first 30 minutes) had a 30-day mortality that was 15 percent lower than that of patients treated with streptokinase in the Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO) study. This was equivalent to an absolute decrease of 1 percent in 30-day mortality. We sought to assess whether the use of t-PA, as compared with streptokinase, is cost effective. Our primary, or base-case, analysis of cost effectiveness used data from the GUSTO study and life expectancy projected on the basis of the records of survivors of myocardial infarction in the Duke Cardiovascular Disease Database. In the primary analysis, we assumed that there were no additional treatment costs due to the use of t-PA after the first year and that the comparative survival benefit of t-PA was still evident one year after enrollment. One year after enrollment, patients who received t-PA had both higher costs ($2,845) and a higher survival rate (an increase of 1.1 percent, or 11 per 1000 patients treated) than streptokinase-treated patients. On the basis of the projected life expectancy of each treatment group, the incremental cost-effectiveness ratio--with both future costs and benefits discounted at 5 percent per year--was $32,678 per year of life saved. The use of t-PA was least cost effective in younger patients and most cost effective in older patients. At all ages, the use of t-PA in patients with anterior infarctions yielded more favorable cost-effectiveness values. In our secondary analyses, the cost-effectiveness values were most sensitive to a lowering of the projected long-term survival benefits of t-PA and to moderate or greater increases in the projected medical costs for patients in the t-PA group after the first year. In contrast, our results were not sensitive to even very unfavorable assumptions about the additional costs associated with the higher rate of disabling stroke that was noted in patients treated with t-PA in the GUSTO study. The cost effectiveness of treatment with accelerated t-PA rather than streptokinase compares favorably with that of other therapies whose added medical benefit for dollars spent is judged by society to be worthwhile.