抗噪的未知应用层协议报文格式最佳分段方法

Noise-Tolerant and Optimal Segmentation of Message Formats for Unknown Application-Layer Protocols

为了自动解析未知应用层协议的报文格式,提出一种未知应用层协议报文格式的最佳分段方法.这种方法不需要关于未知应用层协议的先验知识.它首先建立一种用于最佳分段的隐半马尔可夫模型(HSMM),并利用未知应用层协议在网络会话过程中传输的报文序列样本集来估计该模型的参数;再通过基于HSMM的最大似然概率分段方法,对报文中的各个字段进行最佳划分,同时获取代表各个字段语义的关键词.这种方法并不要求训练集绝对纯净.它能够基于观测序列的似然概率分布,发现混杂在训练集中的其他协议数据(噪声)并进行有效过滤.实验结果表明,该方法能够解析文本和二进制协议的报文格式,依据关键词构建的协议识别特征有很高的准确识别率,并能有效地检测出噪声.

In order to automatically parse message formats of unknown application-layer protocols, this paper proposes an approach to optimally segment the message formats without a priori knowledge. A hidden semi-Markov model （HSMM） is established for the segmentation and its parameters are estimated from a set of message sequences collected from application sessions. By using the estimated HSMM in the maximum most likely segmentation, a message can be optimally divided into segments and keywords that provide semantic information about the segments can be extracted. This approach does not require the training set to be absolutely pure. The noise mixed in the training set can be filtered out based on its likelihood fitting to the HSMM. The experiments conducted in this paper show that the approach is suited to both text and binary protocols. The application-layer signatures constructed from the extracted keywords are highly accurate in identifying the protocols, The noise mixed in the training set can be efficiently detected and automatically filtered out.