RELIABILITY AND RESILIENCE IN AI DRIVEN SYSTEMS: BENCHMARKING CAUSAL ESTIMATORS, PROFILING INFOSTEALER MALWARE, AND ADVANCING MULTILINGUAL SENTIMENT ANALYSIS

Abstract

Artificial intelligence and data-driven systems are increasingly deployed in high stakes environments where the conditions encountered in practice diverge substantially from the conditions assumed during design and evaluation. This divergence — between the idealised assumptions under which a method achieves its theoretical guarantees and the realistic constraints of deployment — is a structural feature of the relationship between scientific method and real-world application. It manifests across domains, disciplines, and problem types, yet it is rarely examined as a unified phenomenon. This thesis investigates the idealisation–deployment gap across three independent research contributions spanning causal inference, cybersecurity, and natural language processing, and argues that the convergent findings across all three domains support a general principle for reliable AI-driven system design: context must be diagnosed before methods are selected, rather than allowing theoretical sophistication or benchmark performance to make that selection by default. The first contribution is a systematic benchmark of seven causal estimators — ranging from the naive difference-in-means to the Causal Forest with Double Machine Learning — evaluated across three datasets from the RealCause benchmarking framework over 100 independently generated samples each, supplemented by a controlled knob-sweep experiment varying covariate overlap and treatment effect heterogeneity independently across a five-by-three parameter grid, yielding a total of 3,600 evaluation records. The central finding is a sophistication paradox: on datasets with poor covariate overlap, the naive estimator achieves root mean squared error of $1,132 on LaLonde PSID while the Causal Forest achieves $15,850 — a factor of fourteen difference in the direction opposite to what the theoretical sophistication ordering of the methods would predict. The T-Learner is identified as the most robust machine learning estimator under distributional mismatch, achieving the lowest root mean squared error of all seven estimators on LaLonde CPS at $2,816, with Wilcoxon signed-rank tests confirming statistically significant advantages over all six competitors. The controlled knob-sweep experiment isolates the mechanism: it is treatment effect heterogeneity, not covariate overlap degradation, that drives estimator ranking reversal. OLS degrades by a factor of seventeen as heterogeneity increases while remaining statistically flat across all five overlap levels, confirmed by a Kruskal Wallis H statistic of 264.99 for the heterogeneity effect and a non-significant H of 4.65 for the overlap effect. On the Twins dataset, which has excellent overlap, all adjusted estimators converge to equivalent performance, confirming that the LaLonde differences are caused by distributional properties rather than implementation artefacts. These findings are operationalised in a two-stage diagnostic decision framework that places heterogeneity assessment before overlap assessment — a deliberate inversion of the conventional practitioner workflow — with specific estimator recommendations for each diagnostic condition grounded in the benchmark evidence. vi The second contribution is a PRISMA-compliant systematic literature review of infostealer malware mechanisms and credential theft prevention techniques, drawing on a final corpus of 166 sources selected through a five-stage screening process from 3,495 initial records. The review is motivated by a June 2025 breach that exposed approximately sixteen billion login credentials across thirty unsecured databases, representing the largest single password breach in documented history. The structural formatting of the exposed data — URL-username-password entries including active session cookies and authentication tokens — is characteristic of contemporary infostealer malware output, reflecting the maturation of a professionalised criminal ecosystem in which sophisticated credential theft tools are available through subscription-based Malware-as-a-Service platforms at monthly fees of $75–$200. The review documents that 60–70% of contemporary infostealer variants evade sandbox detection through environmental awareness techniques, that the 2–4 week new variant release cycle is shorter than the detection effectiveness recovery period, and that this persistent vulnerability window makes detection-centric strategies structurally inadequate as primary defences. Against this threat landscape, the review conducts a comparative multi-dimensional analysis of authentication methods and establishes a security-implementation gap: FIDO2 platform authenticators provide 99% phishing resistance, 2.1-second authentication times, 1% user error rates, and a three-year total cost of ownership 22% lower than password-only systems, yet SMS-based one-time passcodes with zero phishing resistance remain the most widely deployed second factor. The gap is sustained by legacy system incompatibility affecting 72% of enterprises, usability misperceptions held by 74% of security professionals, and human behavioural factors including a 52% password reuse rate and 43% social engineering susceptibility for MFA codes. A multi-dimensional evaluation framework assessing visual cryptography, blockchain-based identity, and behavioural biometrics across seven dimensions concludes that behavioural biometrics are production-ready at global scale and represent the highest research investment priority, while blockchain identity is appropriate only for enterprise business-to-business contexts, and visual cryptography has a fundamental accessibility barrier affecting 2.2 billion people with visual impairments globally. The third contribution is a systematic literature review of advancements and challenges in multilingual and multimodal sentiment analysis, drawing on 18 primary studies selected through a structured quality assessment from 29 initial studies, all published between 2020 and 2023, guided by seven research questions. The review documents a generalisation gap between strong benchmark performance under controlled, English dominant, high-resource conditions and substantially weaker performance under the linguistic diversity, data scarcity, cultural specificity, and multimodal complexity that characterise real-world deployment. The current performance landscape shows audio models combining Mel-Frequency Cepstral Coefficient features with convolutional neural networks achieving up to 92.7% accuracy on audio sentiment benchmarks, multimodal approaches achieving up to 92.28% on CMU-MOSI, Self-MM achieving an F1 score of 84.30% on binary sentiment classification, and XLM-T demonstrating cross-lingual transfer improvements from 40.29% to 56.39% F1 with domain-specific fine-tuning. These results represent the ceiling of performance under optimal benchmark conditions. The structural causes of the generalisation gap — data scarcity vii for low-resource languages, cultural embeddedness of evaluative language, the cost of multimodal annotation, and evaluation fragmentation across research communities — require responses at the level of dataset construction, model architecture, and evaluation methodology rather than incremental benchmark optimisation. Partially effective mitigation strategies including cross-lingual transfer learning, synthetic data generation through back-translation, community-driven dataset initiatives, and weak supervision through emoji-based labelling are documented and assessed. The synthesis across all three studies reveals that the sophistication paradox, the security-implementation gap, and the generalisation gap are structurally identical manifestations of the same underlying problem: a divergence between the idealised conditions of design and evaluation and the realistic constraints of deployment, sustained in each domain by a combination of genuine technical barriers and cognitive or organisational factors that prevent context-appropriate method selection. The T Learner's robustness in causal inference arises from its design avoiding the cross-group extrapolation that the deployment violation induces. FIDO2's security advantage arises from its cryptographic architecture making stolen credentials non-exploitable at the point of deployment rather than relying on preventing theft. Cross-lingual transfer learning's advantage arises from leveraging representations that match the actual linguistic diversity of the deployment context rather than the benchmark context. In each case, the robust solution is the one designed for the actual conditions of use. The practical applications of these findings span several domains. In applied econometrics and policy evaluation, the decision framework provides practitioners with a reproducible diagnostic workflow for estimator selection that corrects a widespread misallocation of diagnostic effort toward overlap assessment at the expense of heterogeneity assessment. In information security, the review provides organisations with evidence-based guidance for authentication migration that directly addresses the misperception that security and usability trade off against each other, and the finding that the most secure authentication methods also have the lowest long term cost challenges the primary organisational barrier to their adoption. In natural language processing, the review provides system designers with a structured characterisation of the conditions under which current multilingual and multimodal models will fail in deployment, enabling more honest capability assessment and more targeted research investment. Across all three domains, the general principle — diagnose context before selecting method — is actionable without requiring domain specific translation, and its derivation from independent empirical evidence in three different fields strengthens its claim to generality. Future research directions identified by this thesis span both domain-specific and cross-domain investigations. Within causal inference, priority directions include confidence interval coverage evaluation across the benchmark grid, extension of the benchmark to CATE estimation, and replication across additional datasets from domains beyond labour economics and infant health. Within cybersecurity, priority directions include longitudinal deployment studies of authentication migration outcomes, federated learning architectures for privacy-preserving behavioural biometrics, and authentication security research specifically calibrated to the viii constraints of small and medium businesses. Within multilingual sentiment analysis, priority directions include lightweight model development through knowledge distillation, culturally aware model architectures incorporating sociolinguistic knowledge, unified multilingual multimodal evaluation benchmarks, and community driven dataset construction for underrepresented language groups. At the cross-domain level, the formalisation of the idealisation–deployment gap as a theoretical construct — characterising the conditions under which the gap is large versus small, and the properties of methods that make them robust versus brittle — represents the most ambitious and potentially most impactful direction for future work building on the contributions of this thesis.