Feasibility of Natural Language Processing–Assisted Auditing of Critical Findings in Chest Radiology

Objective: Time-sensitive communication of critical imaging findings like pneumothorax or pulmonary embolism to referring physicians is essential for patient safety. The definitive communication is the radiology free-text report. Quality assurance initiatives require that institutions audit these communications, a time-intensive manual task. We propose using a rule-based natural language processing system to improve the process for auditing critical findings communications. Methods: We present a pilot assessment of the feasibility of using an automated critical finding identification system to assist quality assurance teams’ evaluation of critical findings communication compliance. Our assessment is based on chest imaging reports. Critical findings are identified in radiology reports using pyConTextNLP, an open source Python implementation of the ConText algorithm. Results: In our test set, there were 75 reports with critical findings and 591 reports without critical findings. pyConTextNLP correctly identified 69 of the positive cases with 8 false-positives for a sensitivity of 0.92 and a specificity of 0.99. Discussion: Natural language processing can provide valuable assistance to auditing critical findings communications.