Background and Objective: Pre-hospital stroke diagnosis remains a major clinical challenge due to the variability of symptoms, which can often be subtle or atypical. This variability leads to Emergency Medical Services (EMS) personnel frequently misdiagnosing stroke and therefore missing the treatment time window. This study aimed to develop an automated approach to detect strokes using video assessment of neurological tasks.

Methods: We created a large database of 300 patients and 86 healthy control subjects, totaling over 50 hours of video recordings. Each subject performed all the tasks of the National Institutes of Health Stroke Scale (NIHSS), which is a standardized scale used to assess the presence and severity of stroke, creating, to our knowledge, the world’s largest video dataset of stroke patients. Using this dataset, we developed preprocessing algorithms and machine learning (ML) models to detect clinically observable stroke-related neurological symptoms in patients. A direct comparison was made between the performance of our approach and the performance of 2,000 EMS personnel trained in stroke recognition. Statistical evaluation included macro F1-score, sensitivity, and specificity metrics.

Results: The proposed approach achieved superior performance compared to EMS personnel, with a 7.2% increase in macro F1-score and a 12.0% increase in sensitivity. Specificity values also demonstrated robust classification capability, confirming the reliability of the automated approach.

Conclusions: Machine learning models can outperform trained EMS personnel in early stroke recognition by detecting clinically observable stroke-related neurological symptoms in pre-hospital settings. These results suggest promise for pre-hospital stroke screening, but clinical impact requires prospective real-world evaluation.