S. S. Kulkarni1, M. R. Rosengart1, B. Fischhoff4, A. E. Barnato5, D. C. Angus2, D. M. Yealy3, D. J. Wallace2, D. Mohan2 1University Of Pittsburgh,Department Of Surgery,Pittsburgh, PA, USA 2University of Pittsburgh,Department Of Critical Care Medicine,Pittsburgh, PA, USA 3University of Pittsburgh,Department Of Emergency Medicine,Pittsburgh, PA, USA 4Carnegie Mellon University,College Of Engineering And Public Policy,Pittsburgh, PA, USA 5Dartmouth Medical School,The Dartmouth Institute For Health Policy And Clinical Practice,Lebanon, NH, USA
Introduction: Up to 70% of severely-injured patients fail to receive care at a trauma center according to best practice guidelines (under-triage),in part, because of physicians’ reliance on poorly-calibrated heuristics (intuitive judgments). In prior work, we developed two video game interventions to recalibrate these heuristics, and, in clinical trials, found that game-based training improved performance on a series of detailed clinical simulations that evaluated triage decision-making. In this post-hoc secondary analysis, we evaluated how baseline triage practices might modify the effect of the interventions using the novel approach of assessing trial participants’ practice patterns in Medicare claims data.
Methods: We obtained Medicare claims records from 2010 to 2015 to measure baseline physician triage practices prior to their participation in one of the two clinical trials conducted in 2016 and 2017. We categorized physicians as having received game-based training or being assigned to the control arm and estimated the proportion of severely-injured patients under-triaged by the two groups. We compared group-level baseline triage performance using the Mann-Whitney test. We then used multivariable models to assess for heterogeneity of the effect of game-based training between high and low baseline performers on triage decision-making in clinical simulations.
Results: Of the 374 eligible physicians from our trials, we identified 319 (85.3%) who had filed a claim for a Medicare fee-for-service beneficiary between 2010-2015. Their median age was 41 years (SD 9.0), 211 (66.1%) were male, and 212 (66.5%) were white. Physicians in both groups managed a similar number of severely-injured patients (median 5 vs. 4, p=0.75) prior to enrollment and both groups under-triaged a similar proportion of patients (mean 0.78 vs. 0.75, p = 0.52) indicating successful randomization. After adjusting for baseline triage performance, physicians who completed game-based training under-triaged 14.2% fewer patients (p = 0.005) in the simulations compared to the control group. The interaction between baseline performance and intervention effect was not significant (p = 0.86).
Conclusions: We used claims data to test the heterogeneity of the treatment effect of behavioral interventions designed to recalibrate physician heuristics. Video game-based training significantly improved under-triage independent of physicians’ baseline triage patterns. We did not find a large heterogeneity of treatment effect between high and low baseline performers.