Detection of Mild Cognitive Impairment from Language Markers with Crossmodal Augmentation

Guangliang Liu, Zhiyu Xue, Liang Zhan, Hiroko H. Dodge, Jiayu Zhou

Research output: Contribution to journalConference articlepeer-review

1 Scopus citations


Mild cognitive impairment is the prodromal stage of Alzheimers disease. Its detection has been a critical task for establishing cohort studies and developing therapeutic interventions for Alzheimers. Various types of markers have been developed for detection. For example, imaging markers from neuroimaging have shown great sensitivity, while its cost is still prohibitive for large-scale screening of early dementia. Recent advances from digital biomarkers, such as language markers, have provided an accessible and affordable alternative. While imaging markers give anatomical descriptions of the brain, language markers capture the behavior characteristics of early dementia subjects. Such differences suggest the benefits of auxiliary information from the imaging modality to improve the predictive power of unimodal predictive models based on language markers alone. However, one significant barrier to the joint analysis is that in typical cohorts, there are only very limited subjects that have both imaging and language modalities. To tackle this challenge, in this paper, we develop a novel crossmodal augmentation tool, which leverages auxiliary imaging information to improve the feature space of language markers so that a subject with only language markers can benefit from imaging information through the augmentation. Our experimental results show that the multi-modal predictive model trained with language markers and auxiliary imaging information significantly outperforms unimodal predictive models.

Original languageEnglish (US)
Pages (from-to)7-18
Number of pages12
JournalPacific Symposium on Biocomputing
Issue number2023
StatePublished - 2023
Event28th Pacific Symposium on Biocomputing, PSB 2023 - Kohala Coast, United States
Duration: Jan 3 2023Jan 7 2023


  • Crossmodal Augmentation
  • Mild Cognitive Impairment
  • Multi-modality Analysis

ASJC Scopus subject areas

  • Biomedical Engineering
  • Computational Theory and Mathematics


Dive into the research topics of 'Detection of Mild Cognitive Impairment from Language Markers with Crossmodal Augmentation'. Together they form a unique fingerprint.

Cite this