基于词嵌入的源码相似度研究

Research on Source Code Similarity Based on Word Embedding

源码相似性度量是代码推荐、缺陷监测、代码搜索等很多软件工程领域任务的基础工作。传统的源码相似性度量方法主要利用统计方法从代码的结构属性、文本特性两方面进行度量,缺乏对代码的语义相似性研究。为解决此类问题,在词嵌入基础上提出结合TF-IDF和Word2vec的向量空间模型,利用向量间距离衡量代码间的相似性,融合代码的语义信息和统计信息。实验结果表明,相比于传统基于统计的方法,该模型效果提高了15%。

Source code similarity measurement is the basis of code recommendation,defect monitoring,code search and many other tasks in the field of software engineering.Traditional source code similarity measurement methods mainly use statistical methods to measure from both the structural properties of the cod e and textual properties,but lack research on code semantic similarity.In order to solve such problems,a vector space model combining TF-IDF and Word2vec is proposed on the basis of word embedding,using inter-vector distance to measure the similarity between codes,and the model integrates the semantic and statistical information of codes.The experimental results show that the model is 15% more effective than the traditional statistics-based approach.