L. V. Selby1, W. R. Narain2, A. Russo1, H. McGowan1, V. E. Strong1, P. D. Stetson2 1Memorial Sloan-Kettering Cancer Center,Surgery,New York, NY, USA 2Memorial Sloan-Kettering Cancer Center,Health Informatics,New York, NY, USA
Introduction: Natural language processing (NLP) is a computer science technique that allows interpretation of narrative text, but is infrequently used to identify surgical complications. Our institution tracks post-operative complications using both the American College of Surgeons – National Surgical Quality Improvement Program (NSQIP) and our in-house surgical secondary events (SSE) database, which captures and grades complications for all surgical patients, but sub-optimally records lower-grade complications. We attempted to use NLP to improve the entry of lower extremity deep venous thrombosis (DVT) and pulmonary embolisms (PE) (collectively: venous thromboembolism [VTE]) in the SSE database.
Methods: In our 2011 – 2014 cohort of NSQIP patients all lower extremity duplex ultrasounds and computerized tomography angiographies (CTA) of the chest performed within 30 days of surgery were divided into training and validation datasets. These studies were chosen as they represent the most frequent methods of detecting DVT and PE at our institution, and a bag-of-words-approach with a support vector machine (SVM) model was used for training. Electronic health record data was used to classify the severity of the VTE according to our modification of the Clavien-Dindo classification. Due to definition differences between NSQIP and the SSE database, we excluded cephalic and portal vein thromboses identified in NSQIP and compared NLP identified VTEs to VTEs identified by both NSQIP and our SSE database, and undertook a blinded review of all instances of discordance.
Results: Of the 10,295 NSQIP patients, 251 were used in our DVT validation cohort (273 total ultrasounds) and 506 in our PE cohort (552 total CTAs). The SVM DVT model had a sensitivity of 85.1% and a specificity of 94.6%, while the PE model had a sensitivity of 90.0% and a specificity of 98.7% (Table 1). The majority of discordances were due to identification of a VTE in studies other than duplex ultrasound or CTA of the chest (9/13; 69.2%), studies not in our original NLP dataset. The majority of DVTs (23 patients, 57.5%) and PEs (20 patients, 69.0%) in the validation set were grade 2 on our modified Clavien-Dindo classification, meaning they required administration of therapeutic intravenous or subcutaneous anticoagulation.
Conclusion: NLP can reliably detect the presence and severity of post-operative lower extremity DVTs and PEs without requiring manual chart review from trained NSQIP surgical case reviewers. We are extending our NLP pilot to real-time identification and grading of all VTEs and to the detection of other post-operative complications, including wound infections.
