TY - JOUR
T1 - Agreement and reliability statistics for shapes
AU - Smith, Travis B.
AU - Smith, Ning
N1 - Funding Information:
Supported by grant P30 EY010572 from the National Institutes of Health (Bethesda, MD) and by unrestricted departmental funding from Research to Prevent Blindness (New York, NY). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Publisher Copyright:
© 2018 Smith, Smith. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
PY - 2018/8
Y1 - 2018/8
N2 - We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.
AB - We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.
UR - http://www.scopus.com/inward/record.url?scp=85052075673&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85052075673&partnerID=8YFLogxK
U2 - 10.1371/journal.pone.0202087
DO - 10.1371/journal.pone.0202087
M3 - Article
C2 - 30138326
AN - SCOPUS:85052075673
SN - 1932-6203
VL - 13
JO - PLoS One
JF - PLoS One
IS - 8
M1 - e0202087
ER -