Reliability and validity of the Veterans Administration Mobility Screening and Solutions Tool

Abstract

Objectives

The Veterans Administration (VA) Mobility Screening and Solutions Tool (VA MSST) was developed to screen a patient’s safe mobility level ‘in the moment’ and provide clinical decision support related to the use of safe patient handling and mobility (SPHM) equipment. This evidence-based flowchart tool is a common language tool that enables any healthcare worker at any time to accurately measure and communicate patient mobility and transfer equipment needs across disciplines and settings.

Methods

The VA MSST has four levels and differentiates between the need for powered and non-powered equipment depending on the patient’s independence. Subject matter experts wrote scenarios for interrater reliability and validity testing. The initial VA MSST draft iteration was reviewed by 163 VA staff (mostly physical therapists and occupational therapists) amongst simulation scenarios and provided content validity, and additional insight and suggestions. Revisions were made to create the final VA MSST which was evaluated by over 200 healthcare workers from varied disciplines (including medical doctors, advanced practice registered nurses, registered nurses, licensed practical nurses, certified nursing assistants, occupational therapists, physical therapists, speech therapists, radiology and ultrasound technicians, etc.). An instruction video and eighteen scenario videos were embedded in an online survey. The survey intended to demonstrate the interrater reliability and validity (concurrent and construct) of the VA MSST. Over 500 VA staff (raters) received a survey invitation via email.

Results

Raters ( N = 230) from multiple disciplines and healthcare settings independently screened patient mobility status for each of 18 scenarios using the VA MSST. The raters were diverse in their age and years of experience. The estimated interrater reliability (IRR) for VA MSST was excellent and statistically significant with an estimated Krippendorff’s alpha (ICC (C, k)) of 0.998 [ 95% CI: 0.996–0.999]. Eighty-two percent of raters reported that overall VA MSST instructions were clear or very clear and understandable. VA MSST ratings made by technicians and nursing assistants group correlated strongly (r = 0.99, p < 0.001) with the ‘gold standard’ (experienced physical therapists), suggesting a high concurrent validity of the tool. The VA MSST significantly discriminated between the different levels of patient mobility required for safe mobilization as intended (each difference, p < 0.0001); this suggests a good construct validity.

Conclusions

The VA MSST is an evidence-based flowchart screening and decision support tool that demonstrates excellent interrater reliability across disciplines and settings. VA MSST has strong face and content validity, as well as good concurrent and construct validity.