Neurocognitive impairment (NCI) is one of the most relevant clinical manifestations of multiple sclerosis (MS). The profile of NCI and the structural and functional changes in the brain structures relevant for cognition in MS share some similarities to those in Alzheimer's disease (AD), the most common cause of neurocognitive disorders. Additionally, despite clear etiopathological differences between MS and AD, an accumulation of effector/memory CD8+ T cells and CD8+ tissue-resident memory T (Trm) cells in cognitively relevant brain structures of MS/AD patients, and higher frequency of effector/memory CD8+ T cells re-expressing CD45RA (TEMRA) with high capacity to secrete cytotoxic molecules and proinflammatory cytokines in their blood, were found. Thus, an active pathogenetic role of CD8+ T cells in the progression of MS and AD may be assumed. In this mini-review, findings supporting the putative role of CD8+ T cells in the pathogenesis of MS and AD are displayed, and putative mechanisms underlying their pathogenetic action are discussed. A special effort was made to identify the gaps in the current knowledge about the role of CD8+ T cells in the development of NCI to “catalyze” translational research leading to new feasible therapeutic interventions.