Quantitative imaging biomarkers: A review of statistical methods for computer algorithm comparisons

Nancy A. Obuchowski, Anthony P. Reeves, Erich P. Huang, Xiao Feng Wang, Andrew J. Buckler, Hyun J. Kim, Huiman X. Barnhart, Edward F. Jackson, Maryellen L. Giger, Gene Pennello, Alicia Y. Toledano, Jayashree Kalpathy-Cramer, Tatiyana V. Apanasovich, Paul E. Kinahan, Kyle J. Myers, Dmitry B. Goldgof, Daniel P. Barboriak, Robert J. Gillies, Lawrence H. Schwartz, Daniel C. Sullivan

Research output: Contribution to journalReview articlepeer-review

125 Scopus citations


Quantitative biomarkers from medical images are becoming important tools for clinical diagnosis, staging, monitoring, treatment planning, and development of new therapies. While there is a rich history of the development of quantitative imaging biomarker (QIB) techniques, little attention has been paid to the validation and comparison of the computer algorithms that implement the QIB measurements. In this paper we provide a framework for QIB algorithm comparisons. We first review and compare various study designs, including designs with the true value (e.g. phantoms, digital reference images, and zero-change studies), designs with a reference standard (e.g. studies testing equivalence with a reference standard), and designs without a reference standard (e.g. agreement studies and studies of algorithm precision). The statistical methods for comparing QIB algorithms are then presented for various study types using both aggregate and disaggregate approaches. We propose a series of steps for establishing the performance of a QIB algorithm, identify limitations in the current statistical literature, and suggest future directions for research.

Original languageEnglish (US)
Pages (from-to)68-106
Number of pages39
JournalStatistical methods in medical research
Issue number1
StatePublished - Feb 27 2015
Externally publishedYes


  • agreement
  • bias
  • image metrics
  • imaging biomarkers
  • precision
  • quantitative imaging
  • repeatability
  • reproducibility

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability
  • Health Information Management


Dive into the research topics of 'Quantitative imaging biomarkers: A review of statistical methods for computer algorithm comparisons'. Together they form a unique fingerprint.

Cite this