Effectiveness of Immersive Virtual Reality on Orthopedic Surgical Skills and Knowledge Acquisition Among Senior Surgical Residents : A Randomized Clinical Trial

Key Points

Question

What is the quantifiable skill and knowledge transfer for surgical trainees using immersive virtual reality to learn both pathology recognition and complex procedural skills?

Findings

In this randomized clinical trial of 18 senior orthopedic surgery residents, those trained using immersive virtual reality demonstrated significant improvements in knowledge and procedural metrics compared with a control group receiving technical video instruction. A transfer effectiveness ratio of 0.79 was demonstrated, indicating that immersive virtual reality substituted for 47.4 minutes of equivalent real operating room training.

Meaning

These findings suggest that immersive virtual reality may play a significant role in the future of procedural training, supplementing and perhaps augmenting traditional teaching and effectively reducing early surgical learning curves.

Abstract

This randomized clinical trial evaluates whether immersive virtual reality improves learning effectiveness among orthopedic surgery residents performing reverse shoulder arthroplasty.

Abstract

Importance

Video learning prior to surgery is common practice for trainees and surgeons, and immersive virtual reality (IVR) simulators are of increasing interest for surgical training. The training effectiveness of IVR compared with video training in complex skill acquisition should be studied.

Objectives

To evaluate whether IVR improves learning effectiveness for surgical trainees and to validate a VR rating scale through correlation to real-world performance.

Design, Setting, and Participants

This block randomized, intervention-controlled clinical trial included senior (ie, postgraduate year 4 and 5) orthopedic surgery residents from multiple institutions in Canada during a single training course. An intention-to-treat analysis was performed. Data were collected from January 30 to February 1, 2020.

Intervention

An IVR training platform providing a case-based module for reverse shoulder arthroplasty (RSA) for advanced rotator cuff tear arthropathy. Participants were permitted to repeat the module indefinitely.

Main Outcomes and Measures

The primary outcome measure was a validated performance metric for both the intervention and control groups (Objective Structured Assessment of Technical Skills [OSATS]). Secondary measures included transfer of training (ToT), transfer effectiveness ratio (TER), and cost-effectiveness (CER) ratios of IVR training compared with control. Additional secondary measures included IVR performance metrics measured on a novel rating scale compared with real-world performance.

Results

A total of 18 senior surgical residents participated; 9 (50%) were randomized to the IVR group and 9 (50%) to the control group. Participant demographic characteristics were not different for age (mean [SD] age: IVR group, 31.1 [2.8] years; control group, 31.0 [2.7] years), gender (IVR group, 8 [89%] men; control group, 6 [67%] men), surgical experience (mean [SD] experience with RSA: IVR group, 3.3 [0.9]; control group, 3.2 [0.4]), or prior simulator use (had experience: IVR group 6 [67%]; control group, 4 [44%]). The IVR group completed training 387% faster considering a single repetition (mean [SD] time for IVR group: 4.1 [2.5] minutes; mean [SD] time for control group: 16.1 [2.6] minutes; difference, 12.0 minutes; 95% CI, 8.8-14.0 minutes; P < .001). The IVR group had significantly better mean (SD) OSATS scores than the control group (15.9 [2.5] vs 9.4 [3.2]; difference, 6.9; 95% CI, 3.3-9.7; P < .001). The IVR group also demonstrated higher mean (SD) verbal questioning scores (4.1 [1.0] vs 2.2 [1.7]; difference, 1.9; 95% CI, 0.1-3.3; P = .03). The IVR score (ie, Precision Score) had a strong correlation to real-world OSATS scores ( r = 0.74) and final implant position ( r = 0.73). The ToT was 59.4%, based on the OSATS score. The TER was 0.79, and the system was 34 times more cost-effective than control, based on CER.

Conclusions and Relevance

In this study, surgical training with IVR demonstrated superior learning efficiency, knowledge, and skill transfer. The TER of 0.79 substituted for 47.4 minutes of operating room time when IVR was used for 60 minutes.

Trial Registration

ClinicalTrials.gov Identifier: NCT04404010