ENHANCED SECURITY AT HYPERVISORS IN CLOUD COMPUTING

Abstract

Cloud computing has revolutionized service delivery by enabling users to access applications and data through remote servers, offering scalability, flexibility, and cost-effectiveness. A critical element of this architecture is the hypervisor, which virtualizes physical resources across multiple virtual machines (VMs) while ensuring isolation and efficient utilization. However, the same hypervisor-based design introduces severe security vulnerabilities. Threats such as hyperjacking, VM escapes, and unauthorized access to shared resources jeopardize confidentiality, integrity, and availability in cloud systems. As cloud infrastructures scale and evolve, intrusion detection becomes vital. Conventional intrusion detection systems (IDSs), which rely on rules or signatures, perform well against known attacks but fail to detect complex, dynamic, and unknown threats. Key challenges include achieving scalability across large infrastructures, adapting to evolving attack vectors, and improving detection accuracy with minimal false positives and negatives. To address these challenges, this research defines four primary objectives. The first is to enhance hypervisor network traffic analysis using fuzzy logic. We introduce the Fuzzy Logic-driven Optimized Long Short-Term Memory (FLOLSTM) model, optimized by the Waterwheel Plant Optimization (WPO) algorithm. FLOLSTM effectively classifies hypervisor traffic, outperforming existing methods with a recall of 98.19%, precision of 98.24%, specificity of 95.19%, and F-measure of 98.22%. The second objective is to improve hypervisor attack detection using deep learning. We propose the Xception Stacked Autoencoder Network with Addax Optimization Algorithm (XSA-Net-AOA). By combining feature extraction, log scaling, and weight optimization, XSA-Net-AOA enhances detection accuracy, achieving a detection rate of 95.87%, a false negative rate of 3.79%, and a false positive rate of 0.98%, significantly reducing misclassification. The third objective focuses on identifying malicious user profiles in hypervisor environments. We develop the Addax Flamingo Optimization Algorithm- vi Deep Residual Network (AFOA-DRN), which integrates hash-based profile matching with secure hash algorithms and feature selection. By fusing outcomes through a weighted average model, AFOA-DRN detects malicious profiles with 91.77% accuracy, 94.88% true positive rate, and 89.77% true negative rate, ensuring robust detection in unknown attack scenarios. The fourth objective is to secure data transfer between hypervisors and VMs through cryptographic innovation. We propose the Round Decreased AES Dynamic S-box (RDAES-DSbox) algorithm, which reduces AES rounds while strengthening substitution processes. This modification enhances both efficiency and security, decreasing encryption and decryption times by 7.30 ms and 7.51 ms, respectively, compared to traditional AES. Collectively, these contributions provide a comprehensive security framework for hypervisor-based cloud systems. By combining intelligent intrusion detection, malicious profile identification, and efficient cryptographic mechanisms, the proposed approaches address the limitations of traditional IDSs and strengthen cloud infrastructure against both known and unknown threats. Experimental results confirm that the models deliver superior performance, scalability, and adaptability, making them promising solutions for securing modern cloud environments.