S. Holla2, N. Lau2, S. E. Parker3, S. D. Safford1 1Virginia Tech Carilion School of Medicine,Surgery,Roanoke, VA, USA 2Virginia Polytechnic Institute and State University,Blacksburg, VA, USA 3Carilion Clinic,Human Factors,Roanoke, VA, USA
Introduction:
Deliberately training novice surgeons on where to attend during an operation has the potential to rapidly improve surgical performance. Eye-tracking enables the use of machine learning to simulate or predict where surgeons would look during surgical procedures. The capability to predict where surgeons look could have important applications. First, this capability can inform instructors where learners tend to look by simulating their eye movements in real time. Second, this capability can help instruct learners on where experts look for any given operation, thereby providing immediate guidance on visual attention. Finally, this capability can provide visual assistance (i.e., “a second pair of eyes”) to alert the physicians performing the operations on where additional attention appears warranted.
Methods:
To advance the application of machine learning for surgical training, we conducted an exploratory study developing and evaluating a computer vision deep neural network (DNN) that adapted the ADNet algorithm and employed data from our previous experiment which involved eye-tracking of an expert, attendings and residents viewing laparoscopic surgery videos. Using a subset of 13 videos incorporated with eye-gaze data from several attendings and residents, we trained the DNN to predict the eye gaze of the attending and resident surgeons separately. We evaluated the DNN by testing whether the DNN predicted eye gaze could produce significant findings similar to our previous experiment.
Results:
The DNN predicted eye gaze yielded two significant findings similar to our empirical experiment. In our prior experiment, one significant finding is that eye-gaze of the expert agreed less with those of attendings than residents and thus implied a greater field of gaze and possibly awareness. In this study, the expert eye-gaze was marginally further away from the DNN predicted eye-gaze of the attendings than the predicted one of the residents (F(1,14)=3.68, p=.076). In addition, we also observed that the DNN predicted eye-gaze was more closely matched to those of residents than the attendings (F(1,14)=6.48, p=.023), indicating that resident eye-gaze was more predictable or learnable.
Figure: A video frame indicating location of the DNN predicted eye-gaze in the (bigger) blue rectangle in comparison to the actual eye gaze of a participant in the (smaller) green rectangle.
Conclusion:
By confirming a prior significant finding, these two preliminary results highlight the potential of machine learning in mimicking eye gaze behaviors of different medical professionals viewing laparoscopic surgery without excessive data collection. Future work entails improving the DNN algorithm and utilizing the remaining portion
