Deep Neural Networks Predict Remaining Surgery Duration from Cholecystectomy Videos

Surgical workflow recognition has numerous potential medical applications, such as the automatic indexing of surgical video databases and the optimization of real-time operating room scheduling, among others. As a result, surgical phase recognition has been studied in the context of several kinds of surgeries, such as cataract, neurological, and laparoscopic surgeries. In the literature, two types of features are typically used to perform this task: visual features and tool usage signals. However, the used visual features are mostly handcrafted. Furthermore, the tool usage signals are usually collected via a manual annotation process or by using additional equipment. In this paper, we propose a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information. In previous studies, it has been shown that the tool usage signals can provide valuable information in performing the phase recognition task. Thus, we present a novel CNN architecture, called EndoNet, that is designed to carry out the phase recognition and tool presence detection tasks in a multi-task manner. To the best of our knowledge, this is the first work proposing to use a CNN for multiple recognition tasks on laparoscopic videos. Experimental comparisons to other methods show that EndoNet yields state-of-the-art results for both tasks.

Disruptions in surgical flow have the potential to increase the occurrence of surgical errors; however, little is known about the frequency and nature of surgical flow disruptions and their effect on the etiology of errors, which makes the development of evidence-based interventions extremely difficult. The goal of this project was to study surgical errors and their relationship to surgical flow disruptions in cardiovascular surgery prospectively to understand better the effect of these disruptions on surgical errors and ultimately patient safety. A trained observer recorded surgical errors and flow disruptions during 31 cardiac surgery operations over a 3-week period and categorized them by a classification system of human factors. Flow disruptions were then reviewed and analyzed by an interdisciplinary team of experts in operative and human factors. Flow disruptions consisted of teamwork/communication failures, equipment and technology problems, extraneous interruptions, training-related distractions, and issues in resource accessibility. Surgical errors increased significantly with increases in flow disruptions. Teamwork/communication failures were the strongest predictor of surgical errors. These findings provide preliminary data to develop evidence-based error management and patient safety programs within cardiac surgery with implications to other related surgical programs.

Effective time and resource management in the operating room requires process information concerning the surgical procedure being performed. A major parameter relevant to the intraoperative process is the remaining intervention time. The work presented here describes an approach for the prediction of the remaining intervention time based on surgical low-level tasks.