Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis.

Methods

We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms.

Results

All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms.

Conclusions

In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0559-x) contains supplementary material, which is available to authorized users.

Related collections

Most cited references 17

Record: found
Abstract: found
Article: not found

Design and analysis of clinical trials with clustering effects due to treatment.

Chris Roberts, Craig S. Roberts (2004)

Where patients receive therapy as a group, there are good theoretical reasons to believe that variation in the outcome will be smaller for patients treated in the same group than for patients treated in different groups. Similarly, where different therapists treat different groups of patients, outcome for patients treated by the same therapist may differ less than outcome for patients treated by different therapists. Clinical trials evaluating such therapies need to consider this potential lack of independence. As with cluster-randomized trials, this has implications for the precision of treatment effects estimates and statistical power. There are nevertheless differences between clustering due to the organization of treatment and that due to randomization. In cluster-randomized trials the distribution of cluster sizes in each treatment arm should be similar as a consequence of randomization unless there is differential loss to follow-up. With clustering due to therapy group or therapist, cluster size may differ systematically between treatment arms, due to size of therapy groups or differing health professional caseload. Intra-cluster correlation may also differ between treatment arms. The implications of differential cluster size and intracluster correlation for design and analysis will be illustrated by data from two trials, the first comparing nurse practitioner care with general practitioner care, and the second comparing a group therapy with individual treatment as usual. The special case where a group therapy or therapist is compared with an unclustered treatment is examined in detail using a simulation study. The implications of differential clustering effects for sample size and power are addressed. It is argued that the design and analysis of this type of trial should take account of possible heterogeneity in cluster size and intracluster correlation.

0 comments Cited 43 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported

Marion K Campbell, Jeremy M. Grimshaw, Diana Elbourne (2004)

Background Increasingly, researchers are recognizing that there are many situations where the use of a cluster randomized trial may be more appropriate than an individually randomized trial. Similarly, the need for appropriate standards of reporting of cluster trials is more widely acknowledged. Methods In this paper, we describe the results of a survey to inform the appropriate reporting of the intracluster correlation coefficient (ICC) – the statistical measure of the clustering effect associated with a cluster randomized trial. Results We identified three dimensions that should be considered when reporting an ICC – a description of the dataset (including characteristics of the outcome and the intervention), information on how the ICC was calculated, and information on the precision of the ICC. Conclusions This paper demonstrates the development of a framework for the reporting of ICCs. If adopted into routine practice, it has the potential to facilitate the interpretation of the cluster trial being reported and should help the development of new trials in the area.

0 comments Cited 31 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Cost effectiveness of community leg ulcer clinics: randomised controlled trial.

S Walters, M Peters, Doris Brooker … (1998)

To establish the relative cost effectiveness of community leg ulcer clinics that use four layer compression bandaging versus usual care provided by district nurses. Randomised controlled trial with 1 year of follow up. Eight community based research clinics in four trusts in Trent. 233 patients with venous leg ulcers allocated at random to intervention (120) or control (113) group. Weekly treatment with four layer bandaging in a leg ulcer clinic (clinic group) or usual care at home by the district nursing service (control group). Time to complete ulcer healing, patient health status, and recurrence of ulcers. Satisfaction with care, use of services, and personal costs were also monitored. The ulcers of patients in the clinic group tended to heal sooner than those in the control group over the whole 12 month follow up (log rank P=0.03). At 12 weeks, 34% of patients in the clinic group were healed compared with 24% in the control. The crude initial healing rate of ulcers in intervention compared with control patients was 1.45 (95% confidence interval 1.04 to 2. 03). No significant differences were found between the groups in health status. Mean total NHS costs were 878.06 pounds per year for the clinic group and 859.34 pounds for the control (P=0.89). Community based leg ulcer clinics with trained nurses using four layer bandaging is more effective than traditional home based treatment. This benefit is achieved at a small additional cost and could be delivered at reduced cost if certain service configurations were used.

0 comments Cited 25 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Jane Candlish:

ORCID: http://orcid.org/0000-0001-5280-7891

jane.candlish@sheffield.ac.uk

M. Dawn Teare: m.d.teare@sheffield.ac.uk

Munyaradzi Dimairo: m.dimairo@sheffield.ac.uk

Laura Flight: l.flight@sheffield.ac.uk

Laura Mandefield: l.mandefield@sheffield.ac.uk

Stephen J. Walters: s.j.walters@sheffield.ac.uk

Journal

Journal ID (nlm-ta): BMC Med Res Methodol

Journal ID (iso-abbrev): BMC Med Res Methodol

Title: BMC Medical Research Methodology

Publisher: BioMed Central (London )

ISSN (Electronic): 1471-2288

Publication date (Electronic): 11 October 2018

Publication date PMC-release: 11 October 2018

Publication date Collection: 2018

Volume: 18

Affiliations

ISNI 0000 0004 1936 9262, GRID grid.11835.3e, School of Health and Related Research (ScHARR), , University of Sheffield, ; 30 Regent Street, S1 4DA, Sheffield, UK

Article

Publisher ID: 559

DOI: 10.1186/s12874-018-0559-x

PMC ID: 6186141

PubMed ID: 30314463

License:

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Funding

Funded by: Harry Worthington Scholarship, University of Sheffield

Custom metadata

Data availability:

ScienceOpen disciplines: Medicine

Keywords: clustering, randomised controlled trial, partially nested, partially clustered, therapist effects, individually randomised group treatment, individually randomised cluster trial, intervention studies

Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: a simulation study

Read this article at

Abstract

Background

Methods

Results

Conclusions

Electronic supplementary material

Related collections

Chinese Journal of Schistosomiasis Control

Most cited references 17

Design and analysis of clinical trials with clustering effects due to treatment.

Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported

Cost effectiveness of community leg ulcer clinics: randomised controlled trial.

Author and article information

Contributors

Journal

Affiliations

Article

Funding

Categories

Custom metadata

Comments

Comment on this article

Similar content 79

Cited by 3

Most referenced authors 248