R. Harari1,2, L. Kennedy-Metz3, G. Varni4, V. Unhelkar5, E. Salas5, R. D. Dias1,2, M. A. Zenati2,6 1Mass General Brigham, Boston, MA, USA 2Harvard Medical School, Boston, MA, USA 3Roanoke College, Salem, VA, USA 4University of Trento, Trento, TRENTO, Italy 5Rice University, Houston, TX, USA 6Division of Cardiac Surgery, VA Boston Healthcare System, West Roxbury, MA, USA
Introduction:
Objective assessment of surgical team’s non-technical skills (NTS) in operating room (OR) remains challenging, as they encompass a broad range of cognitive, interpersonal, and situational factors that are not easily quantifiable. To address this gap, our study introduces a novel approach through a multimodal assessment method built on four key components: team physiological data, team motion data, OR environment data, and patient/procedure-related data. By integrating these diverse data streams, we aim to establish a comprehensive understanding of the intricate interactions underlying NTS in the cardiac OR.
Method:
In this prospective study, we recorded complete audio and RGB-video data of 30 on-pump cardiac surgery procedures to evaluate teamwork quality during the separation from bypass phase.
Metrics: Using this dataset, we extracted metrics from four distinct sets of data: (1) team physiological data, including HRV metrics using the MindWare system for data collection and Kubios for HRV analysis; (2) team motion data, such as team displacement and entropy using the OpenPose deep learning library on recorded videos; (3) OR environment data, involving factors such as alarm frequencies and external distractions using video annotation and audio analysis; and (4) patient/procedure data, encompassing factors like procedure and bypass lengths sourced from patient records and videos.
Modeling: We employed LASSO technique to select important features contributing in NTS score, preventing model overfitting. Subsequently, we utilized a multilinear regression model with the selected features to analyze the relationship between predictors and the NTS scores.
Results:
The dataset comprised 30 surgeries, with 86.6% being on-pump coronary artery bypass graft (CABG) procedures and 13.4% involving aortic valve replacement (AVR) surgeries. All patients were male, with an average age of 72 years ± 6.3. The NOTSS-derived team performance score had a median value of 3.31, indicating consistently high-quality team behaviors.
Using LASSO, 12 features were selected from the initial 37 variables: 5 from team physiological data, 2 from team motion data, 3 from OR environment data, and 2 from patient/procedural data. The model produced an adjusted R-squared of 0.439 (an R-squared=0.671), explaining about 44% of NTS score variability through the selected variables using the multimodal approach (F= 2.895, p= 0.02).
Conclusion:
Our findings support the utility of the proposed multimodal approach to evaluate NTS during complex surgery. Further validation of these findings across a larger and more diverse sample is necessary. Investigating the applicability of the proposed framework to various surgical procedures may contribute to the advancement of NTS assessment and training programs, thereby leading to improved surgical outcomes and improved patient safety.
