S.C. Fairburn1, R. Godwin1 1University Of Alabama at Birmingham, Marnix E. Heersink Institute For Biomedical Innovation, Birmingham, Alabama, USA
Introduction: Accurate detection and classification of surgical instruments are critical for enhancing surgical education and improving training outcomes. The ability to reliably identify different orientations of needles and needle drivers during suturing can significantly aid in assessing and teaching surgical skills. This study represents a preliminary step towards developing a tool that provides quantitative feedback to students, allowing them to improve their suturing skills and techniques. By leveraging YOLOv5, an advanced object detection model, this project aims to enhance surgical education through precise and automated instrument state recognition.
Methods: A dataset of 777 images from suturing videos on synthetic pads was collected and annotated, capturing six distinct classes: "needle_driver_closed," "needle_driver_open," "needle_pre_wound," "needle_within_wound," "needle_post_wound," and "wound." The dataset was divided into five folds for cross-validation. A YOLOv5 model was trained on each fold, with performance metrics such as precision, recall, mean Average Precision (mAP) at Intersection over Union (IoU) threshold 0.5 (mAP@50), and mAP across IoU thresholds from 0.5 to 0.95 (mAP@50-95) calculated to assess the model's performance.
Results: The YOLOv5 model demonstrated high performance across all folds, indicating its robustness and reliability in detecting and classifying needle and needle driver states in suturing training. The average precision across all folds was 0.9536, while the average recall was 0.9549. The mean Average Precision at an IoU threshold of 0.5 (mAP@50) was particularly high, averaging 0.9699 across all folds, suggesting that the model has a high level of accuracy in detecting the presence of the objects. The mAP across IoU thresholds from 0.5 to 0.95 (mAP@50-95) averaged 0.6572, demonstrating the model's effectiveness in accurately localizing and classifying objects with varying levels of overlap.
Conclusion: The YOLOv5-based model provides robust and accurate detection and classification of needle and needle driver states in suturing training. The high precision and recall across multiple classes highlight the model's effectiveness and potential for integration into surgical education programs. This model can be used to enhance technical skill acquisition by providing detailed feedback on instrument handling, ultimately improving students' suturing techniques and overall surgical outcomes. Future steps for the project include testing newer YOLO models with hyperparameter optimization, isolating video segments in time and space from larger videos , and implementing a user-friendly tool for surgical trainees.