A SEQUENTIAL APPROACH FOR CONDITIONAL RANDOM FIELDS BASED ENTITY RECOGNITION AND ROUTING IN MIXTURE OF EXPERTS STATE SPACE MODELS

Abstract

Sequential learning systems have become increasingly important for modeling tem poral and structured data in domains such as language processing, dynamical sys tems, and biomedical text mining. Recent architectures including State Space Mod els (SSMs) and Mixture-of-Experts (MoE) have improved scalability and long-range sequence modeling, while structured prediction methods such as Conditional Random Fields (CRFs) continue to provide effective sequence-level inference. The behavior of modular sequential architectures, by contrast, under heterogeneous conditions remains insufficiently understood. This thesis presents a study of modular sequential learning systems through empiri cal analysis and application-oriented evaluation. The first part investigates Mixture of-Experts based spectral State Space Models in controlled regime-switching envi ronments containing chaotic, oscillatory, and noisy temporal patterns. Experimental results demonstrate routing collapse, instability, and inverse scaling behavior, where increasing the number of experts does not consistently improve performance. The sec ond part presents a sequential Biomedical Named Entity Recognition framework com bining BioBERT, Bidirectional LSTM, multi-head self-attention, and a Conditional Random Field decoder. Evaluated on the BioCreative V Chemical–Disease Relation dataset, the proposed model achieves an entity-level micro-averaged F1-score above 86% with improved sequence-level consistency. The findings of this thesis show that modularity alone does not guarantee improved sequential learning performance. Instead, effective architectures depend on the align ment between model structure and the underlying characteristics of the data. The study highlights the importance of structured inference, stable routing mechanisms, and alignment-aware design for future sequential learning systems.