Methodological quality of randomized controlled trials of postoperative epidural analgesia: Validation of the epidural analgesia trial checklist as a specific instrument to evaluate methodology

Background: The overall benefits of epidural analgesia are controversial, in part because of the varying quality of methodology in published randomized controlled trials (RCTs). We performed a systematic review of available RCTs to examine the methodological quality of epidural analgesia trials. Current instruments for evaluating the quality of methodology are generic; thus, we also developed a specific assessment tool named Epidural Analgesia Trial Checklist (EATC). Methods: The National Library of Medicine's PubMed database was searched (1966 to January 2006) for RCTs of epidural analgesia. All RCTs that had epidural infusion analgesia in at least 1 study arm and as primary intervention for randomization were included. Two independent reviewers were given blinded full-text paper versions of each article and reviewed all articles for inclusion in this study. Study characteristics were extracted from accepted RCTs, and reviewers completed the standardized 7-item Jadad score, 22-item Consolidated Standards of Reporting Trials (CONSORT) checklist, and 8-item EATC for evaluation of methodological quality. Results: A total of 321 articles met all inclusion criteria. The overall median (first, third quartiles) Jadad, CONSORT, and EATC scores were 2 (1, 3), 10 (8, 11), and 4 (3, 6) (of maximum scores of 5, 22, and 8), respectively. For all assessments, we found significantly higher methodological study quality for articles with a larger study population size, those written by a first author affiliated with an anesthesiology department, and studies published after release of the CONSORT statement with a significant overall increase in methodological quality over time. There was no effect on methodological quality with regard to region of publication or number of centers. There was relatively high interrater agreement when using the EATC (κ = 0.92). The items most frequently lacking from the studies captured using the EATC were appropriate description/definition of adverse effects (11.8% of all studies properly reported this), proper presentation of visual analog scale (VAS) pain scores (31.2%), and assessment of VAS pain both at rest and with activity (39.9%). Conclusions: Methodology scores for epidural analgesia RCTs have improved over time. The EATC seems to correlate well with other commonly used generic assessments for methodological RCT quality and be useful for assessing methodological quality of epidural RCTs. Future epidural analgesia RCTs should focus on improving appropriate description/definition of adverse effects, proper presentation of VAS pain scores, and assessment of VAS pain both at rest and with activity.