Diabetes mellitus is associated with an increase in the risk of dementia and the proportion of patients who convert from mild cognitive impairment (MCI) to dementia. In addition to MCI and dementia, the stages of diabetes-associated cognitive dysfunction include subtle cognitive changes that are unlikely to affect activities of daily life or diabetes self-management. These diabetes-associated cognitive decrements have structural brain correlates detectable with brain MRI, but usually show little progression over time. Although cognitive decrements do not generally represent a pre-dementia stage in patients below the age of 60–65 years, in older individuals these subtle cognitive changes might represent the earliest stages of a dementia process. Acknowledgment of diabetes-associated cognitive decrements can help to improve understanding of patients’ symptoms and guide management. Future challenges are to establish the importance of screening for cognitive impairment in people with diabetes, to identify those at increased risk of accelerated cognitive decline at an early stage, and to develop effective treatments.