Sensitive system calls based packed malware variants detection using principal component initialized MultiLayers neural networks

Malware detection has become mission sensitive as its threats spread from computer systems to Internet of things systems.Modern malware variants are generally equipped with sophisticated packers,which allow them bypass modern machine learning based detection systems.To detect packed malware variants,unpacking techniques and dynamic malware analysis are the two choices.However,unpacking techniques cannot always be useful since there exist some packers such as private packers which are hard to unpack.Although dynamic malware analysis can obtain the running behaviours of executables,the unpacking behaviours of packers add noisy information to the real behaviours of executables,which has a bad affect on accuracy.To overcome these challenges,in this paper,we propose a new method which first extracts a series of system calls which is sensitive to malicious behaviours,then use principal component analysis to extract features of these sensitive system calls,and finally adopt multi-layers neural networks to classify the features of malware variants and legitimate ones.Theoretical analysis and real-life experimental results show that our packed malware variants detection technique is comparable with the the state-of-art methods in terms of accuracy.Our approach can achieve more than 95.6\%of detection accuracy and 0.048 s of classification time cost.

Malware variants detection based on ensemble learning

Application programming interface(API)is a procedure call interface to operation system resource.API-based behavior features can capture the malicious behaviors of malware variants.However,existing malware detection approaches have a deal of complex operations on constructing and matching.Furthermore,graph matching is adopted in many approaches,which is a nondeterministic polynominal(NP)-complete problem because of computational complexity.To address these problems,a novel approach is proposed to detect malware variants.Firstly,the API of the malware are divided by their functions and parameters.Then,the classified behavior graph(CBG)is constructed from the API call sequences.Finally,the signature based on CBGs for each malware family is generated.Besides,the malware variants are classified by ensemble learning algorithm.Experiments on 1220 malware samples show that the true positive rate(TPR)is up to 89.0%with the low false positive rate(FPR)3.7%by ensemble learning.