ADVANCED STATISTICAL FEATURE REDUCTION FOR MOBILE SECURITY: ENHANCING MALWARE DETECTION WITH ML

Abstract

With mobile devices becoming ubiquitous in today's society due to the processing and storage of confidential data such as credentials for bank access, messages and emails, and user's identity, it has been possible for the increasing number of opportunities of cyber attacks through use of malicious software. Machine learning based malware detection is a very efficient method but its performance greatly depends on feature selection in a dataset. High dimensional mobile data, which includes permissions, intents, hardware, system and network information can be filled up with redundant and non-informative attributes. Such feature set leads to high computational complexity and lower classifier performance. The current work focuses on Statistical Test-Driven Mobile Security Feature Selection. As opposed to other machine learning approaches, the suggested method involves systematic evaluation of feature importance via several statistically proven techniques such as Chi-Square test, Kruskal-Wallis test, and Mann-Whitney U test. All statistical techniques consider distribution difference, class separability, and feature-malware dependency for identifying discriminating features. An important contribution of the paper is the introduction of hybrid statistical feature selection technique, where several statistical scores are combined to represent complementary aspects of feature importance. Experiments performed on a practical dataset of mobile malwares available from UCI Machine Learning Repository showed that statistical feature selection techniques, in particular hybrid combinations, lead to significant feature reduction of above 60%-90% while maintaining competitive and even better detection accuracy compared with non-statistical techniques. Actually, some hybrid approaches, such as Mann-Whitney + Kruskal-Wallis with Logistic Regression provided near perfect results with accuracy 99.73% and 100% precision. Consequently, the paper provides an efficient solution to the problem of feature selection for mobile security, that could be potentially used as part of on-device scanner system.