Stability of response characteristics of a Delphi panel: application of bootstrap data expansion

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Delphi surveys with panels of experts in a particular area of interest have been widely utilized in the fields of clinical medicine, nursing practice, medical education and healthcare services. Despite this wide applicability of the Delphi methodology, there is no clear identification of what constitutes a sufficient number of Delphi survey participants to ensure stability of results.

Methods

The study analyzed the response characteristics from the first round of a Delphi survey conducted with 23 experts in healthcare quality and patient safety. The panel members had similar training and subject matter understanding of the Malcolm Baldrige Criteria for Performance Excellence in Healthcare. The raw data from the first round sampling, which usually contains the largest diversity of responses, were augmented via bootstrap sampling to obtain computer-generated results for two larger samples obtained by sampling with replacement. Response characteristics (mean, trimmed mean, standard deviation and 95% confidence intervals) for 54 survey items were compared for the responses of the 23 actual study participants and two computer-generated samples of 1000 and 2000 resampling iterations.

Results

The results from this study indicate that the response characteristics of a small expert panel in a well-defined knowledge area are stable in light of augmented sampling.

Conclusion

Panels of similarly trained experts (who possess a general understanding in the field of interest) provide effective and reliable utilization of a small sample from a limited number of experts in a field of study to develop reliable criteria that inform judgment and support effective decision-making.

Related collections

Most cited references 52

Record: found
Abstract: found
Book: not found

An Introduction to the Bootstrap

Bradley Efron, R.J. Tibshirani (1994)

Statistics is a subject of many uses and surprisingly few effective practitioners. The traditional road to statistical knowledge is blocked, for most, by a formidable wall of mathematics. The approach in An Introduction to the Bootstrap avoids that wall. It arms scientists and engineers, as well as statisticians, with the computational techniques they need to analyze and understand complicated data sets.

0 comments Cited 3145 times – based on 0 reviews

Bookmark

Record: found
Abstract: found
Article: not found

What does it mean to involve consumers successfully in NHS research? A consensus study.

Rosemary Telford, Jonathan Boote, Cindy L Cooper (2004)

To obtain consensus on the principles and indicators of successful consumer involvement in NHS research. Consensus methods were used. An expert workshop, employing the nominal group technique was used to generate potential principles and indicators. A two-round postal Delphi process was used to obtain consensus on the principles and indicators. Participants were drawn from health, social care, universities and consumer organizations. A purposive sampling strategy was used to identify people who had experience and/or knowledge of consumer involvement in NHS research. Six researchers and seven consumers participated in an expert workshop. Ninety-six people completed both rounds of the Delphi process. Consensus on principles and indicators of successful consumer involvement in NHS research. Eight principles were developed through an expert workshop and Delphi process, and rated as both clear and valid. Consensus was reached on at least one clear and valid indicator by which to measure each principle. Consensus has been obtained on eight principles of successful consumer involvement in NHS research. They may help commissioners, researchers and consumers to deepen their understanding of this issue, and can be used to guide good practice.

0 comments Cited 67 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

The bootstrap: a technique for data-driven statistics. Using computer-intensive analyses to explore experimental data.

Theodore A. Henderson (2005)

The concept of resampling data--more commonly referred to as bootstrapping--has been in use for more than three decades. Bootstrapping has considerable theoretical advantages when it is applied to non-Gaussian data. Most of the published literature is concerned with the mathematical aspects of the bootstrap but increasingly this technique is being utilized in medical and other fields. I reviewed the published literature following a 1994 publication assessing the transfer of technology, including the bootstrap, to the biomedical literature. In the ten-year period following that 1994 paper there were 1679 published references to the technique in Medline. In that same time period the following citations were found in the four major medical journals-British Medical Journal (48), JAMA (51), Lancet (52) and the New England Journal of Medicine (45). I introduce the basic theory of the bootstrap, the jackknife, and permutation tests. The bootstrap is used to estimate the accuracy of an estimator such as the standard error, a confidence interval, or the bias of an estimator. The technique may be useful for analysing smallish expensive-to-collect data sets where prior information is sparse, distributional assumptions are unclear, and where further data may be difficult to acquire. Some of the elementary uses of bootstrapping are illustrated by considering the calculation of confidence intervals such as for reference ranges or for experimental data findings, hypothesis testing such as comparing experimental findings, linear regression, and correlation when studying association and prediction of variables, non-linear regression such as used in immunoassay techniques, and ROC curve processing. These techniques can supplement current nonparametric statistical methods and should be included, where appropriate, in the armamentarium of data processing methodologies.

0 comments Cited 65 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Journal

Journal ID (nlm-ta): BMC Med Res Methodol

Title: BMC Medical Research Methodology

Publisher: BioMed Central (London )

ISSN (Electronic): 1471-2288

Publication date Collection: 2005

Publication date (Electronic): 1 December 2005

Volume: 5

Page: 37

Affiliations

[1 ]Quality and Patient Safety Initiatives, Rural and Community Health Institute, The Texas A&M University System Health Science Center, College Station, Texas, USA

[2 ]Department of Educational Administration and Human Resource Development, The Texas A&M University, College Station, Texas, USA

Article

Publisher ID: 1471-2288-5-37

DOI: 10.1186/1471-2288-5-37

PMC ID: 1318466

PubMed ID: 16321161

SO-VID: aa3cf6e5-1a0a-47e6-8bdb-2e9dc681c59f

License:

This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Stability of response characteristics of a Delphi panel: application of bootstrap data expansion

Read this article at

Abstract

Background

Methods

Results

Conclusion

Related collections

Karger: Coronavirus / COVID-19

Most cited references 52

An Introduction to the Bootstrap

What does it mean to involve consumers successfully in NHS research? A consensus study.

The bootstrap: a technique for data-driven statistics. Using computer-intensive analyses to explore experimental data.

Author and article information

Journal

Affiliations

Article

History

Categories

Comments

Comment on this article

Similar content 225

Cited by 244

Most referenced authors 437