TOPOLOGY-AWARE HYBRID STATE-SPACE AND TRANSFORMER-CNN ARCHITECTURES WITH METAHEURISTIC OPTIMIZATION FOR EXPLAINABLE RADIOGENOMIC BRAIN TUMOR SEGMENTATION

Abstract

The accurate segmentation of brain tumour sub-regions: necrotic core (NCR), peritumoral edema (ED), and gadolinium-enhancing tumour (ET), using multi-parametric Magnetic Resonance Imaging (mpMRI) is a clinically essential problem for pre-operative surgical planning, radiotherapy contouring, and treatment response assessment. While there have been considerable advances in this field, two major families of Deep Learning architectures face significant intrinsic limitations. CNNs models local spatial dependencies but fail to represent long-range volumetric dependencies; ViTs tackle this problem but make them computationally infeasible due to their quadratic complexity on high-resolution 3D volumes. This thesis proposes a clinically-oriented deep learning approach that integrates three different technologies. Firstly, the selective state-space models (SSMs) more specifically, the Mamba architecture allow to efficiently learn the global spatial context on volumetric MRI with linear complexity O(N). Secondly, the integration of the Topological Data Analysis (TDA) technique, provides a means of imposing an anatomically plausible regularisation on the tumour segmentation process. In addition to the segmentation component, the proposed system architecture employs radiogenomic multi-task learning, allowing for non-invasive virtual biopsy by predicting clinically relevant molecular biomarkers, including IDH mutations and MGMT promoter methylation, using image-based features.Domain adaptation techniques with GRL and federated learning approaches are included to tackle the problem of domain shift, especially the scarcity of paediatric glioma cases. Interpretability in clinical settings is ensured through the application of CBMs and Grad-CAM saliency maps, making it possible to audit and trust the model's decision-making process. Comparing with previous state-of-the-art frameworks on the BraTS dataset, our benchmarking study shows that an IChOA optimised CNN baseline reaches 0.97 mean DSC, and the proposed hybrid CNN Mamba TARMN architecture is built to extend this accuracy with linear-complexity global context modelling, while the topology-aware loss minimises anatomically inconsistent outputs.