TY - JOUR
T1 - ParAlg
T2 - A Paraphasia Algorithm for Multinomial Classification of Picture Naming Errors
AU - Casilio, Marianne
AU - Fergadiotis, Gerasimos
AU - Salem, Alexandra C.
AU - Gale, Robert C.
AU - McKinney-Bock, Katy
AU - Bedrick, Steven
N1 - Publisher Copyright:
© 2023 American Speech-Language-Hearing Association.
PY - 2023/3
Y1 - 2023/3
N2 - Purpose: A preliminary version of a paraphasia classification algorithm (hence-forth called ParAlg) has previously been shown to be a viable method for coding picture naming errors. The purpose of this study is to present an updated version of ParAlg, which uses multinomial classification, and comprehensively eval-uate its performance when using two different forms of transcribed input. Method: A subset of 11,999 archival responses produced on the Philadelphia Naming Test were classified into six cardinal paraphasia types using ParAlg under two transcription configurations: (a) using phonemic transcriptions for responses exclusively (phonemic-only) and (b) using phonemic transcriptions for nonlexical responses and orthographic transcriptions for lexical responses (ortho-graphic-lexical). Agreement was quantified by comparing ParAlg-generated para-phasia codes between configurations and relative to human-annotated codes using four metrics (positive predictive value, sensitivity, specificity, and F1 score). An item-level qualitative analysis of misclassifications under the best performing configuration was also completed to identify the source and nature of coding discrepancies. Results: Agreement between ParAlg-generated and human-annotated codes was high, although the orthographic-lexical configuration outperformed phone-mic-only (weighted-average F1 scores of.78 and.87, respectively). A qualitative analysis of the orthographic-lexical configuration revealed a mix of human-and ParAlg-related misclassifications, the former of which were related primarily to phonological similarity judgments whereas the latter were due to semantic similarity assignment. Conclusions: ParAlg is an accurate and efficient alternative to manual scoring of paraphasias, particularly when lexical responses are orthographically tran-scribed. With further development, it has the potential to be a useful software application for anomia assessment. Supplemental Material: https://doi.org/10.23641/asha.22087763.
AB - Purpose: A preliminary version of a paraphasia classification algorithm (hence-forth called ParAlg) has previously been shown to be a viable method for coding picture naming errors. The purpose of this study is to present an updated version of ParAlg, which uses multinomial classification, and comprehensively eval-uate its performance when using two different forms of transcribed input. Method: A subset of 11,999 archival responses produced on the Philadelphia Naming Test were classified into six cardinal paraphasia types using ParAlg under two transcription configurations: (a) using phonemic transcriptions for responses exclusively (phonemic-only) and (b) using phonemic transcriptions for nonlexical responses and orthographic transcriptions for lexical responses (ortho-graphic-lexical). Agreement was quantified by comparing ParAlg-generated para-phasia codes between configurations and relative to human-annotated codes using four metrics (positive predictive value, sensitivity, specificity, and F1 score). An item-level qualitative analysis of misclassifications under the best performing configuration was also completed to identify the source and nature of coding discrepancies. Results: Agreement between ParAlg-generated and human-annotated codes was high, although the orthographic-lexical configuration outperformed phone-mic-only (weighted-average F1 scores of.78 and.87, respectively). A qualitative analysis of the orthographic-lexical configuration revealed a mix of human-and ParAlg-related misclassifications, the former of which were related primarily to phonological similarity judgments whereas the latter were due to semantic similarity assignment. Conclusions: ParAlg is an accurate and efficient alternative to manual scoring of paraphasias, particularly when lexical responses are orthographically tran-scribed. With further development, it has the potential to be a useful software application for anomia assessment. Supplemental Material: https://doi.org/10.23641/asha.22087763.
UR - http://www.scopus.com/inward/record.url?scp=85150000876&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85150000876&partnerID=8YFLogxK
U2 - 10.1044/2022_JSLHR-22-00255
DO - 10.1044/2022_JSLHR-22-00255
M3 - Article
C2 - 36791263
AN - SCOPUS:85150000876
SN - 1092-4388
VL - 66
SP - 966
EP - 986
JO - Journal of Speech, Language, and Hearing Research
JF - Journal of Speech, Language, and Hearing Research
IS - 3
ER -