A retrospective quantitative implementation evaluation of Safer Opioid Prescribing, a Canadian continuing education program

Abstract

Background

Continuing health professions education (CHPE) is an important policy intervention for the opioid epidemic. Besides effectiveness or impact, health policy implementation should be studied to understand how an intervention was delivered within complex environments. Implementation outcomes can be used to help interpret CHPE effects and impacts, help answer questions of “how” and “why” programs work, and inform transferability. We evaluated Safer Opioid Prescribing (SOP), a national CHPE program, using implementation outcomes of reach, dose, fidelity, and participant responsiveness.

Methods

We conducted a retrospective quantitative implementation evaluation of the 2014–2017 cohorts of SOP. To measure reach and dose, we examined participation and completion data. We used Ontario physician demographic data, including regulatory status with respect to controlled substances, to examine relevant trends. To measure fidelity and participant responsiveness, we analyzed participant-provided evaluations of bias, active learning, and relevance to practice. We used descriptive statistics and measures of association for both continuous and categorical variables. We used logistic regression to determine predictors of workshop participation and analysis of covariance to examine variation in satisfaction across different-sized sessions.

Results

Reach: In total, there were 472 unique participants, 84.0% of whom were family physicians. Among Ontario physician participants, 90.0% were family physicians with characteristics representative of province-wide demographics. Dose: Webinar completion rate was 86.2% with no differences in completion based on rurality, gender, or controlled substance prescribing status with medical regulatory authorities. Fidelity and participant responsiveness: Nearly all participants rated the three webinars and workshop as balanced, and each element of SOP was also rated as highly relevant to clinical practice.

Conclusions

This evaluation demonstrates that Safer Opioid Prescribing was implemented as intended. Over a short period and without any external funding, the program reached more than 1% of the Ontario physician workforce. This suggests that the program may be a good model for using virtual CHPE to reach a critical mass of prescribers. This study represents a methodological advance of adapting evaluation methods from health policy and complex interventions for continuing health professions education. Future studies will assess effectiveness and impact on opioid prescribing and utilization within evaluation models of complex interventions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12909-021-02529-7.