大语言模型C/C++代码漏洞检测效能的实证研究

An Empirical Study of Vulnerability Detection Effectiveness in C/C++for LLM

代码漏洞检测是软件安全领域的研究热点,涌现出了大量的工具与算法,但受制于代码复杂抽象的逻辑实现,高效的漏洞检测仍未实现。近年来,由于大语言模型技术展现出极强的语言理解和文本生成能力,大语言模型赋能漏洞检测的研究应运而生。选取了4款大语言模型在juliet-test-suite基准数据集上对其漏洞检测效能进行实证研究,并与传统的静态分析工具进行对比。实验结果显示,当前商业大语言模型具备一定的漏洞检测能力,但无法完全替代传统的检测方法。最后,分析梳理了大语言模型在漏洞挖掘领域的能力评估、现有局限和未来发展趋势,有助于未来大语言模型在漏洞挖掘领域的普及和应用。

logic implementation of the code,efficient vulnerability detection remains a challenge.In recent years,LLM(Large Language Model)has demonstrated strong language understanding and text-generating capabilities,thus the research on large language models to empower vulnerability detection has emerged.This paper selects four large language models and empirically investigates their vulnerability detection effectiveness on the juliet-test-suite benchmark dataset and compares them with conventional static analysis tools.Experimental results indicate that the current commercial large language models possess certain vulnerability detection capabilities,but cannot be a substitute for conventional detection method yet.Finally,the paper analyzes and reviews the capability evaluation,existing limitations and future development trends of large language models in the vulnerability mining field,which will help the popularization and application of large models in the vulnerability mining field in the future.