Development of a quality assessment scale for retrospective clinical studies in pediatric surgery

Shawn J. Rangel, Jennifer Kelsey, Christopher E. Colby, Jo Dee Anderson, R. Lawrence Moss, D. Dudgeon, S. Rangel

Research output: Contribution to journalArticlepeer-review

71 Scopus citations


Background/Purpose: More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery. Methods: The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers. Results: Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P < .01). Conclusions: The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systemati(review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.

Original languageEnglish (US)
Pages (from-to)390-396
Number of pages7
JournalJournal of pediatric surgery
Issue number3
StatePublished - Mar 1 2003
Externally publishedYes


  • Quality assessment
  • Retrospective clinical studies

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Surgery


Dive into the research topics of 'Development of a quality assessment scale for retrospective clinical studies in pediatric surgery'. Together they form a unique fingerprint.

Cite this