Assessing the difficulty and time cost of de-identification in clinical narratives

David A. Dorr, W. F. Phillips, S. Phansalkar, S. A. Sims, J. F. Hurdle

Research output: Contribution to journalReview articlepeer-review

51 Scopus citations


Objective: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. Methods: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the "easy" PHI that are number- or date-based. Results: From 262 notes, 2074 PHI, or 7.9 ± 6.1 per note, were found. The average recall (or sensitivity) was 95.9% while precision was 99.6% for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5%) but worse precision (88.4%) for its subset of identifiers. Manually de-identifying a note took 87.3 ± 61 seconds on average. Conclusions: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.

Original languageEnglish (US)
Pages (from-to)246-252
Number of pages7
JournalMethods of Information in Medicine
Issue number3
StatePublished - 2006


  • Computerized medical records systems
  • De-identification
  • Health Insurance Portability and Accountability Act
  • Medical informatics computing
  • natural language processing

ASJC Scopus subject areas

  • Health Informatics
  • Advanced and Specialized Nursing
  • Health Information Management


Dive into the research topics of 'Assessing the difficulty and time cost of de-identification in clinical narratives'. Together they form a unique fingerprint.

Cite this