Developing outcome, process and balancing measures for an emergency department longitudinal patient monitoring system using a modified Delphi

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Early warning score systems have been widely recommended for use to detect clinical deterioration in patients. The Irish National Emergency Medicine Programme has developed and piloted an emergency department specific early warning score system. The objective of this study was to develop a consensus among frontline healthcare staff, quality and safety staff and health systems researchers regarding evaluation measures for an early warning score system in the Emergency Department.

Methods

Participatory action research including a modified Delphi consensus building technique with frontline hospital staff, quality and safety staff, health systems researchers, local and national emergency medicine stakeholders was the method employed in this study. In Stage One, a workshop was held with the participatory action research team including frontline hospital staff, quality and safety staff and health systems researchers to gather suggestions regarding the evaluation measures. In Stage Two, an electronic modified-Delphi study was undertaken with a panel consisting of the workshop participants, key local and national emergency medicine stakeholders. Descriptive statistics were used to summarise the characteristics of the panellists who completed the questionnaires in each round. The mean Likert rating, standard deviation and 95% bias-corrected bootstrapped confidence interval for each variable was calculated. Bonferroni corrections were applied to take account of multiple testing. Data were analysed using Stata 14.0 SE.

Results

Using the Institute for Healthcare Improvement framework, 12 process, outcome and balancing metrics for measuring the effectiveness of an ED-specific early warning score system were developed.

Conclusion

There are currently no published measures for evaluating the effectiveness of an ED early warning score system. It was possible in this study to develop a suite of evaluation measures using a modified Delphi consensus approach. Using the collective expertise of frontline hospital staff, quality and safety staff and health systems researchers to develop and categorise the initial set of potential measures was an innovative and unique element of this study.

Electronic supplementary material

The online version of this article (10.1186/s12873-018-0220-3) contains supplementary material, which is available to authorized users.

Related collections

Most cited references 11

Record: found
Abstract: found
Article: not found

Economic evaluation and decision making in the UK.

J. Buxton (2005)

This article reviews the development of economic evaluation of health technologies in the UK and its impact on decision making. After a long period of limited impact from studies mainly carried out as academic exercises, the advent of the National Institute for Health and Clinical Excellence (NICE) in 1999 provided a transparent decision-making context where economic evaluation plays a central role. This article reviews some of the key characteristics about the way NICE works, for example, the way NICE has defined the form of analysis that it requires, reflecting its objective of maximising health gain (QALYs) from the predetermined and limited UK NHS budget. Two broad areas of widespread concern are noted. The first relates to the cost-effectiveness thresholds that NICE uses and the basis for them. The second is the patchy implementation of NICE guidance and the possible reasons for this. But even within the UK, NICE is the exception in making extensive and explicit use of economic evaluation and this article goes on to suggest that if there is to be a more widespread and consistent use of economic evaluation at both central and local levels, then health economists and others need to address three issues. The first is to be clear about what is the correct conceptual basis for determining the cost-effectiveness threshold and then to ensure that NICE has the empirical evidence to set it appropriately. The second is to recognise that even using the limited view of costs adopted by NICE, economic evaluations imply temporal and cross-service budgetary flexibility that the NHS locally does not in practice enjoy. The third issue is that with academic pressures for ever-increasing sophistication of 'state of the art' economic evaluation analysis, the NHS has more and more precise understanding of the cost effectiveness of just a few new technologies and little or no analysis of most. This limits the value of the former by reducing further the scope for appropriately disinvesting from cost-ineffective technologies to meet the additional costs of investing in cost-effective new ones. Whilst NICE stands out as an example of a context where high-quality economic evaluation plays a major role in decision making, the process is far from perfect and certainly is not representative of the use made of economic evaluation by the NHS as a whole. Health economists need to engage with the public and the health service to better understand their perspectives, rather than focusing on academic concerns relating to details of theory and analytical method.

0 comments Cited 43 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

An accessible analytical approach for investigating what happens between the rounds of a Delphi study.

David Dexter, J Greatorex (2000)

The Delphi method is used to investigate consensus amongst a panel of experts using repeated rounds of a questionnaire, often in healthcare settings. However, many Delphi studies do not report any investigation into what happens to the stability of consensus or the convergence of agreement between the rounds in the study, which may be of importance. In this paper an accessible analytical approach is outlined using graphical presentations of means and standard deviations to identify what happens between rounds. For Delphi studies where the scale upon which experts are expressing their opinions can be considered to be interval, the mean will represent the group opinion whilst the standard deviation will represent the level of agreement. An example Delphi study from a healthcare setting is used to illustrate the methodology.

0 comments Cited 29 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Development of key performance indicators for emergency departments in Ireland using an electronic modified-Delphi consensus approach.

Cathal Walsh, Fergal Hickey, Patrick Plunkett … (2013)

The objective of this study was to develop a consensus among emergency medicine (EM) specialists working in Ireland for emergency department (ED) key performance indicators (KPIs).

0 comments Cited 24 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Marie E. Ward: m.ward@ucd.ie

Abel Wakai: AWakai@rcsi.ie

Ronald McDowell: ronaldmcdowell@rcsi.ie , R.McDowell@qub.ac.uk

Fiona Boland: fionaboland@rcsi.ie

Eoin Coughlan: eoincoughlan@yahoo.com

Moayed Hamza: Moayed.hamza@ucdconnect.ie

John Browne: j.browne@ucc.ie

Ronan O’Sullivan: rgosullivan@bonsecours.ie

Una Geary: ugeary@stjames.ie

Fiona McDaid: fiona.mcdaid@hse.ie

Éidín Ní Shé: eidin.nishe@ucd.ie

Frances J. Drummond: francesjdrummond22@gmail.com

Conor Deasy: Conor.Deasy@hse.ie

Eilish McAuliffe: 00 353-(0)1-716 6456 , Eilish.mcauliffe@ucd.ie

Journal

Journal ID (nlm-ta): BMC Emerg Med

Journal ID (iso-abbrev): BMC Emerg Med

Title: BMC Emergency Medicine

Publisher: BioMed Central (London )

ISSN (Electronic): 1471-227X

Publication date (Electronic): 14 January 2019

Publication date PMC-release: 14 January 2019

Publication date Collection: 2019

Volume: 19

Electronic Location Identifier: 7

Affiliations

[1 ]ISNI 0000 0001 0768 2743, GRID grid.7886.1, School of Nursing, Midwifery and Health Systems, C129, UCD Health Sciences Centre, , University College Dublin, ; Belfield, Dublin, 4 Ireland

[2 ]ISNI 0000 0004 0617 6058, GRID grid.414315.6, Emergency Care Research Unit (ECRU), Division of Population Health Sciences (PHS), Royal College of Surgeons in Ireland (RCSI), Dublin 2 and Department of Emergency Medicine, , Beaumont Hospital, ; Dublin, 9 Ireland

[3 ]ISNI 0000 0004 0374 7521, GRID grid.4777.3, General Practice and HRB Centre for Primary Care Research, Royal College of Surgeons in Ireland, Cancer Epidemiology and Health Services Research Group, , Centre for Public Health, Queen’s University Belfast, ; Belfast, BT126BA UK

[4 ]ISNI 0000 0004 0488 7120, GRID grid.4912.e, Division of Population Health Sciences, , Royal College of Surgeons in Ireland, ; Dublin, Ireland

[5 ]ISNI 0000000123318773, GRID grid.7872.a, Department of Epidemiology and Public Health, , University College Cork, ; Western Rd, Cork, Ireland

[6 ]ISNI 0000 0004 0389 5639, GRID grid.460892.1, Bon Secours Hospital, ; Cork, Ireland

[7 ]ISNI 0000 0004 0617 8280, GRID grid.416409.e, Department of Emergency Medicine, , St James’s Hospital, ; Dublin, 8 Ireland

[8 ]Department of Emergency Medicine, Naas Hospital, Naas, Co, Kildare, Ireland

[9 ]ISNI 0000000123318773, GRID grid.7872.a, Cancer Research@UCC, , University College Cork, ; Cork, Ireland

[10 ]ISNI 0000 0004 0617 6269, GRID grid.411916.a, Department of Emergency Medicine, , Cork University Hospital, ; Cork, Ireland

Article

Publisher ID: 220

DOI: 10.1186/s12873-018-0220-3

PMC ID: 6332627

PubMed ID: 30642263

SO-VID: 9543233a-f30c-40f0-a426-e136587179c5

License:

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.