A. Thangirala1, E. Loftspring1, Y. Aphinyanaphongs2, P. Shah3, J. Chen2, J. Oeding1, A. Kelleher1, E. Hu1, J. Martin4, N. Ostberg1, G. Katz4, S. Brejt4, N. Singh1, K. Kan4 1New York University School Of Medicine, Department Of Medicine, New York, NY, USA 2New York University School Of Medicine, Department Of Population Health, New York, NY, USA 3New York University School Of Medicine, Department Of Surgery, New York, NY, USA 4New York University School Of Medicine, Division Of Cardiology, New York, NY, USA
Introduction: Risk calculators to predict postoperative mortality often rely on logistic regression (LR) analysis. Machine learning models are able to incorporate a greater number of input variables by identifying non-linear relationships. Automated machine learning (AutoML) processes regularly outperform regular machine learning (ML) and LR methods for predictive accuracy. Autogluon, an AutoML system that has demonstrated superior benchmark results to other AutoML frameworks, has not yet been applied to predict 30-day mortality.
Methods: We used an AutoML system developed and released by Amazon in 2020, AutoGluon v0.3.1, to predict 30-day post-operative mortality in the 2019 ACS NSQIP database. A total of 3,049,617 patients and 79 pre-operative variables were included. Post-operative mortality was defined as death that occurred within 30 days of the surgery. Models were trained for four hours to optimize performance on the Brier score, with lower being better. Validation of all performance metrics was done using the 2019 ACS NSQIP database.
Results: 0.95% of the patients (n = 28,961) died post-operatively. Brier scores were calculated for each model with the top performing model being an ensembled Random Forest model having a Brier score of 0.00849 on the validation set. The corresponding AUROC and AUC-PR was 0.938 and 0.300 respectively (Figure).
Conclusion: Automated machine learning models offer similar or improved discriminatory characteristics to existing post-operative mortality calculators. Future post-op mortality models may benefit from AutoML analysis.
