Assessing fidelity of treatment delivery in group and individual 12-step facilitation

Barbara K. Campbell; Jennifer K. Manuel; Sarah Turcotte Manser; K. Michelle Peavy; Julija Stelmokas; Dennis McCarty; Joseph R. Guydish

doi:10.1016/j.jsat.2012.07.003

Assessing fidelity of treatment delivery in group and individual 12-step facilitation

Barbara K. Campbell
, Jennifer K. Manuel
, Sarah Turcotte Manser
, K. Michelle Peavy
, Julija Stelmokas
, Dennis McCarty
, Joseph R. Guydish

School Of Public Health

Research output: Contribution to journal › Article › peer-review

9 Scopus citations

Abstract

Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, Stimulant Abuser Groups to Engage in 12-Step. Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69-.91). Internal consistency for the instrument was variable (.47-.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance provided support for concurrent and convergent validity. Implications and future directions for the use of TSF ACES in clinical trials and community treatment implementation are discussed.

Original language	English (US)
Pages (from-to)	169-176
Number of pages	8
Journal	Journal of Substance Abuse Treatment
Volume	44
Issue number	2
DOIs	https://doi.org/10.1016/j.jsat.2012.07.003
State	Published - Feb 1 2013

Funding

This work was supported by NIDA ( R01 DA025600 ), by the Western States research node of NIDA CTN ( U10 DA015815 ) and by the NIDA San Francisco Treatment Research Center ( P50 DA009253 ). Jennifer Manuel is currently supported by a NIDA training grant T32 DA007250 . We are grateful to Dennis Donovan, Dennis Daley and the STAGE-12 study team for their help in providing access to STAGE-12 adherence scales, digital recordings of treatment sessions and other study data, as well as to staff at all the participating treatment sites. In addition, we appreciate the dedication and professionalism of our fidelity raters, who worked tirelessly to complete all ratings, and the formatting assistance of Lindsay Docto.

Funders	Funder number
NIDA San Francisco Treatment Research Center	P50 DA009253
NIDA training grant	T32 DA007250
National Institute on Drug Abuse	R01 DA025600, P50DA009253, U10 DA015815

Keywords

12-Step facilitation
Coding system
Treatment fidelity

ASJC Scopus subject areas

Medicine (miscellaneous)
Phychiatric Mental Health
Clinical Psychology
Psychiatry and Mental health

Access to Document

10.1016/j.jsat.2012.07.003

Cite this

@article{1bcd72ab101048f68a4412423777af52,

title = "Assessing fidelity of treatment delivery in group and individual 12-step facilitation",

abstract = "Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, Stimulant Abuser Groups to Engage in 12-Step. Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97\% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69-.91). Internal consistency for the instrument was variable (.47-.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance provided support for concurrent and convergent validity. Implications and future directions for the use of TSF ACES in clinical trials and community treatment implementation are discussed.",

keywords = "12-Step facilitation, Coding system, Treatment fidelity",

author = "Campbell, \{Barbara K.\} and Manuel, \{Jennifer K.\} and Manser, \{Sarah Turcotte\} and Peavy, \{K. Michelle\} and Julija Stelmokas and Dennis McCarty and Guydish, \{Joseph R.\}",

note = "Funding Information: This work was supported by NIDA ( R01 DA025600 ), by the Western States research node of NIDA CTN ( U10 DA015815 ) and by the NIDA San Francisco Treatment Research Center ( P50 DA009253 ). Jennifer Manuel is currently supported by a NIDA training grant T32 DA007250 . We are grateful to Dennis Donovan, Dennis Daley and the STAGE-12 study team for their help in providing access to STAGE-12 adherence scales, digital recordings of treatment sessions and other study data, as well as to staff at all the participating treatment sites. In addition, we appreciate the dedication and professionalism of our fidelity raters, who worked tirelessly to complete all ratings, and the formatting assistance of Lindsay Docto. ",

year = "2013",

month = feb,

day = "1",

doi = "10.1016/j.jsat.2012.07.003",

language = "English (US)",

volume = "44",

pages = "169--176",

journal = "Journal of Substance Abuse Treatment",

issn = "0740-5472",

publisher = "Elsevier Inc.",

number = "2",

}

TY - JOUR

T1 - Assessing fidelity of treatment delivery in group and individual 12-step facilitation

AU - Campbell, Barbara K.

AU - Manuel, Jennifer K.

AU - Manser, Sarah Turcotte

AU - Peavy, K. Michelle

AU - Stelmokas, Julija

AU - McCarty, Dennis

AU - Guydish, Joseph R.

N1 - Funding Information: This work was supported by NIDA ( R01 DA025600 ), by the Western States research node of NIDA CTN ( U10 DA015815 ) and by the NIDA San Francisco Treatment Research Center ( P50 DA009253 ). Jennifer Manuel is currently supported by a NIDA training grant T32 DA007250 . We are grateful to Dennis Donovan, Dennis Daley and the STAGE-12 study team for their help in providing access to STAGE-12 adherence scales, digital recordings of treatment sessions and other study data, as well as to staff at all the participating treatment sites. In addition, we appreciate the dedication and professionalism of our fidelity raters, who worked tirelessly to complete all ratings, and the formatting assistance of Lindsay Docto.

PY - 2013/2/1

Y1 - 2013/2/1

N2 - Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, Stimulant Abuser Groups to Engage in 12-Step. Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69-.91). Internal consistency for the instrument was variable (.47-.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance provided support for concurrent and convergent validity. Implications and future directions for the use of TSF ACES in clinical trials and community treatment implementation are discussed.

AB - Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, Stimulant Abuser Groups to Engage in 12-Step. Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69-.91). Internal consistency for the instrument was variable (.47-.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance provided support for concurrent and convergent validity. Implications and future directions for the use of TSF ACES in clinical trials and community treatment implementation are discussed.

KW - 12-Step facilitation

KW - Coding system

KW - Treatment fidelity

UR - https://www.scopus.com/pages/publications/84870706469

UR - https://www.scopus.com/pages/publications/84870706469#tab=citedBy

U2 - 10.1016/j.jsat.2012.07.003

DO - 10.1016/j.jsat.2012.07.003

M3 - Article

C2 - 22944595

AN - SCOPUS:84870706469

SN - 0740-5472

VL - 44

SP - 169

EP - 176

JO - Journal of Substance Abuse Treatment

JF - Journal of Substance Abuse Treatment

IS - 2

ER -

Assessing fidelity of treatment delivery in group and individual 12-step facilitation

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this