Variation in outcomes of quality measurement by data source

OBJECTIVE: To evaluate selected Children's Health Insurance Program Reauthorization Act claims-based quality measures using claims data alone, electronic health record (EHR) data alone, and both data sources combined. METHODS: Our population included pediatric patients from 46 clinics in the OCHIN network of community health centers, who were continuously enrolled in Oregon's public health insurance program during 2010. Within this population, we calculated selected pediatric care quality measures according to the Children's Health Insurance Program Reauthorization Act technical specifications within administrative claims. We then calculated these measures in the same cohort, by using EHR data, by using the technical specifications plus clinical data previously shown to enhance capture of a given measure. We used the κ statistic to determine agreement in measurement when using claims versus EHR data. Finally, we measured quality of care delivered to the study population, when using a combined dataset of linked, patient-level administrative claims and EHR data. RESULTS: When using administrative claims data, 1.0% of children (aged 3-17) had a BMI percentile recorded, compared with 71.9% based on the EHR data (κ agreement [k] ≤ 0.01), and 72.0% in the combined dataset. Among children turning 2 in 2010, 20.2% received all recommended immunizations according to the administrative claims data, 17.2% according to the EHR data (k = 0.82), and 21.4% according to the combined dataset. CONCLUSIONS: Children's care quality measures may not be accurate when assessed using only administrative claims. Adding EHR data to administrative claims data may yield more complete measurement.