VMP虚拟指令逆向分析算法

Reverse analysis algorithm of VMP virtual instruction

针对现有虚拟机保护逆向分析集中式解释执行结构不适用于线索式解释执行,且虚拟机字节码序列隐含的控制流表现在解释例程相邻跳转关系上的问题,提出一种基于动态追踪记录和解释例程语义特征的虚拟机保护软件逆向分析算法。以动态二进制逆向调试作为支撑,跟踪记录被虚拟机保护的代码在动态执行过程中的指令执行轨迹信息,对记录的虚拟机动态执行轨迹信息进行聚类分析,识别虚拟机指令的动态解释执行例程,根据解释执行过程的语义特征分析还原动态解释例程的虚拟指令。测试结果表明,该算法能够正确还原虚拟机保护程序的虚拟指令,有效简化虚拟机保护过的程序,降低软件逆向分析难度。

The current reverse analysis of virtual machine protection aims at the indirect threaded interpretation,but it is not suitable to the direct threaded interpretation and the control flow of byte codes of virtual machine is reflected in the jump of handlers implicitly.To address this problem,a reverse analysis algorithm of virtual machine protection software based on dynamic trace record and semantic feature of handlers was proposed.With the support of dynamic binary reverse debug,the instruction execution information in which the codes protected by virtual machine were executed dynamically was recorded by tracing.The recorded dynamic execution trace information of virtual machine was clustered,which recognized the dynamic handlers of virtual machine instructions,and analyzed the semantic feature of handlers during the execution that could recover virtual instructions of dynamic handlers.The test shows that the proposed algorithm can recover the virtual instructions of software protected by virtual machine,and effectively simplify the software protected by virtual machine that can reduce the difficulty of software reverse analysis enormously.