TY - JOUR
T1 - Measuring antenatal counseling skill with a milestone-based assessment tool
T2 - a validation study
AU - Bartlett, Michelle J.
AU - Umoren, Rachel
AU - Amory, Josephine H.
AU - Huynh, Trang
AU - Kim, Amanda J.H.
AU - Stiffler, Amanda K.
AU - Mastroianni, Rossella
AU - Ficco, Ellie
AU - French, Heather
AU - Gray, Megan
N1 - Publisher Copyright:
© 2023, The Author(s).
PY - 2023/12
Y1 - 2023/12
N2 - Background: Antenatal counseling for parents in the setting of expected preterm delivery is an important component of pediatric training. However, healthcare professionals receive a variable amount and quality of formal training. This study evaluated and discussed validity of a practical tool to assess antenatal counseling skills and provide evaluative feedback: the Antenatal Counseling Milestones Scale (ACoMS). Methods: Experts in antenatal counseling developed an anchored milestone-based tool to evaluate observable skills. Study participants with a range of antenatal counseling skills were recruited to participate in simulation of counseling sessions in person or via video with standardized patient actors presenting with preterm labor at 23 weeks’ gestation. Two faculty observers scored each session independently using the ACoMS. Participants completed an ACoMS self-assessment, demographic, and feedback survey. Validity was measured with weighted kappas for inter-rater agreement, Kruskal–Wallis and Dunn’s tests for milestone levels between degrees of expertise in counseling, and cronbach’s alpha for item consistency. Results: Forty-two participants completed observed counseling sessions. Of the 17 items included in the tool, 15 items were statistically significant with scores scaling with level of training. A majority of elements had fair-moderate agreement between raters, and there was high internal consistency amongst all items. Conclusion: This study demonstrates that the internal structure of the ACoMS rubric has greater than fair inter-rater reliability and high internal consistency amongst items. Content validity is supported by the scale’s ability to discern level of training. Application of the ACoMS to clinical encounters is needed to determine utility in clinical practice.
AB - Background: Antenatal counseling for parents in the setting of expected preterm delivery is an important component of pediatric training. However, healthcare professionals receive a variable amount and quality of formal training. This study evaluated and discussed validity of a practical tool to assess antenatal counseling skills and provide evaluative feedback: the Antenatal Counseling Milestones Scale (ACoMS). Methods: Experts in antenatal counseling developed an anchored milestone-based tool to evaluate observable skills. Study participants with a range of antenatal counseling skills were recruited to participate in simulation of counseling sessions in person or via video with standardized patient actors presenting with preterm labor at 23 weeks’ gestation. Two faculty observers scored each session independently using the ACoMS. Participants completed an ACoMS self-assessment, demographic, and feedback survey. Validity was measured with weighted kappas for inter-rater agreement, Kruskal–Wallis and Dunn’s tests for milestone levels between degrees of expertise in counseling, and cronbach’s alpha for item consistency. Results: Forty-two participants completed observed counseling sessions. Of the 17 items included in the tool, 15 items were statistically significant with scores scaling with level of training. A majority of elements had fair-moderate agreement between raters, and there was high internal consistency amongst all items. Conclusion: This study demonstrates that the internal structure of the ACoMS rubric has greater than fair inter-rater reliability and high internal consistency amongst items. Content validity is supported by the scale’s ability to discern level of training. Application of the ACoMS to clinical encounters is needed to determine utility in clinical practice.
KW - Antenatal counseling
KW - Simulation
KW - Validity
UR - http://www.scopus.com/inward/record.url?scp=85158912187&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85158912187&partnerID=8YFLogxK
U2 - 10.1186/s12909-023-04282-5
DO - 10.1186/s12909-023-04282-5
M3 - Article
C2 - 37165398
AN - SCOPUS:85158912187
SN - 1472-6920
VL - 23
JO - BMC medical education
JF - BMC medical education
IS - 1
M1 - 325
ER -