Unstable plaques can cause acute cardiovascular and cerebrovascular diseases. The stability and instability of plaque are related to the plaque echo status in ultrasound. Carotid videos provide detailed plaque information compared to static images. Ultrasound-based plaque echo classification is challenging due to noise, interference frames, small targets (plaques), and complex shape changes.

This study proposes a Multi-dimensional Aggregation Network (MA-Net) guided by key features for plaque diagnosis based on carotid ultrasound video, which uses only video-level labels. MA-Net consists of Key-Feature (KF) and Temporal-Channel-Spatial (TCS) modules. The KF module learns the contribution of each frame to the classification at the feature level, adaptively infers the importance score of each frame, thereby reducing the influence of interference frames. The TCS module includes the Temporal-Channel (TC) and Temporal-Spatial (TS) sub-modules. In addition to studying the temporal dimension, it delves into the relationship between the channel and spatial dimensions. TC analyses the temporal dependencies among the channels and filters noise. Moreover, TS extracts features more accurately through the spatio-temporal information contained in the surrounding environment of the plaque.

The performance of MA-Net on the SHU-Ultrasound-Video-2020 dataset is better than that of the state-of-the-art models of video classification, showing at least a 5% increase in accuracy, with an accuracy rate of 87.36%.

The outstanding diagnostic capability of the proposed model will help provide a more robust and reproducible diagnostic process with a lower labour cost for clinical carotid plaque diagnosis.