A Step Towards Automated Tool Tracking to Objectively Track Trainee’s Intra-operative Autonomy: Leveraging Black Box Technology as a Proof of Concept Study

Introduction: Surgical competency remains a challenging metric to quantify and therefore ensure that trainees are achieving. Currently, case minimums outlined by the Residency Review Committee (RRC) are the standard for ensuring neurosurgery residents are competent at the time of graduation, however there is evidence that these are not a valid marker of competency. Several efforts have attempted to subjectively quantify a resident trainee’s progress including the Surgical Autonomy Program or objective structured assessment of technical skills (O-SATS). To date, no reliable tool exists to objectively measure what portion of the case a resident participated in.

Methods: A symptomatic 5 cm sympathetic chain schwannoma tumor resection was performed by a resident, fellow and faculty member and was recorded using Black Box Technology. Several video feeds of the operating room and from the microscope were synced. At each time point, both the primary and assisting surgeon were identified using video footage of the room as well as which surgical instruments were being used through microscope footage. Audio transcript of the intraoperative communication was also obtained and included in the training model.

Results: The procedure was successfully subdivided into over 25 steps. Using the audio and video feeds, each operator was identified as being in the role of primary surgeon, assisting surgeon or observing for each step. Further, during the entire operation all utilized surgical instrument were attributed to one of the operator’s (resident, fellow, attending) hands (dominant, non-dominant). This was superimposed with the audio transcript to assess the degree of verbal assistance the trainee was requiring to perform the step. Through the combination of these data streams, trainee autonomy was assessed over the course of the operation.

Conclusion : Objectively tracking trainee autonomy remains an ongoing challenge in surgical education. This proof-of-concept study will serve as a template for future machine learning solutions.