面向多维稀疏数据仓库的欺诈销售行为挖掘

Sale Fraud Behavior Detection over Multidimensional Sparse Data Warehouse

分销渠道系统中,产品制造商会分配给销售额较大的分销商更多返点利润鼓励销售,而分销商之间可能会联合起来将多个分销商的销售业绩累计在其中一个分销商上,获取高额利润,这种商业欺诈行为被称为挂单或窜货.由于数据中大量正常极值点的存在,使得传统异常探测算法很难区分正常极值和由挂单导致的异常极值;另外,多维销售数据本身就存在的稀疏性导致多维数据异常探测算法无法有效运行.为了克服上述问题,将人工智能和数据库技术结合起来,提出了基于分割率的特征提取方法和基于张量重构的挂单行为挖掘算法.同时,由于分销商之间存在多种挂单行为,设计了基于挂单模式偏序格的特征提取方法来对销售数据集中存在的挂单行为进行分类.在合成数据的实验中,所提出的挂单点挖掘算法能达到65%的平均AUC值,而传统特征提取方法仅达到36%和30%的平均AUC值.在真实数据上的实验结果表明,挂单行为探测方法能区分正常销售极值和挂单行为产生的异常极值.

In distribution channel system,product manufacturer will often reward retail trader who makes big deal to increase the sales.On the other hand,in order to obtain high reward,retail traders may form alliance,where a cheating retail trader accumulates the deals of other retail traders.This type of commercial fraud is called deal cheating or cross region sale.Because the sales contain a lot of normal big deals,traditional outlier detection methods cannot distinguish the normal extreme value and the true outlier generated by deal cheating behavior.Meanwhile,the sparsity of the multidimensional sales data makes the outlier detection methods based on multidimensional space cannot work effectively.To handle the aforementioned problems,this study proposes deal cheating mining algorithms based on ratio characteristic and tensor reconstruction method.These algorithms combine artificial intelligence and database technique.Meanwhile,because there are multiple types of deal cheating patterns,this study proposes deal cheating pattern classification methods based on the partially ordered lattice of deal cheating patterns.In the experiments on synthetic data,the deal cheating detection algorithm based on the ratio characteristic can achieve an average AUC-value of 65%.The traditional feature extraction methods can only achieve average AUC-values of 36%and 30%.In the experiments on the real data,the results shows the deal cheating detection algorithm is capable of distinguishing normal big deal from abnormal big deal which may be generated by the deal cheating behaviors.