Accurate and timely classification of cerebellar diseases is crucial for effective diagnostic, yet it remains challenging due to the inherent heterogeneity of these disorders and the subtlety of their neuroimaging manifestations. This study investigated a novel multi-stage ensemble framework integrating SE blocks and segmentation-assisted augmentation tailored for limited cerebellum disease MRI data. Dataset included 3,296 MRI scans from four classes and we divided dataset into three parts: training, testing, and validation, and their ratio was 64:20:16. However, we performed image forensic analysis on it, such as Error Level Analysis (ELA) and Noise Residual Analysis (NRA). This study used deep learning architectures that can automatically classify cerebellum diseases and compared these models, which included six D-CNNs models, six transfer learning models, and three ensemble models. Another important contribution of our study is the significant improvement in the classification efficiency by strategically integrating squeeze and excitation and label smoothing techniques. We show that fine-tuning significantly improves the diagnostic accuracy of both D-CNNs and transfer learning models on cerebellum MRI data. Notably, our combined models consistently achieve higher performance, with FusionNet-6 reaching an exceptional accuracy of 99.83%. K-fold cross-validation was performed, yielding consistently high performance with per-class sensitivity and specificity above 99%. The study also greatly enhances the impact of dataset augmentation techniques, including the use of segmented data to reveal complex interactions that can enhance the performance of some models or, in some cases, dramatically reduce the performance of specific models. These results underscore the immense potential of deep learning ensembles to provide highly accurate and robust diagnostic support for cerebellum diseases, paving the way for more objective and efficient clinical workflows.