基于历史行为与高低阶特征的点击率预估模型

A CTR Prediction Model Based on Historical Behavior and High-Low Order Features

针对点击率预估模型对用户历史行为特征提取能力不足以及忽略低阶特征的问题,提出一种新的点击率预估模型TDFA。该模型首先基于自注意力机制构建Transformer网络获取用户历史行为之间的关联信息,再将历史行为与目标项目关联后获得用户历史行为特征,并结合用户信息、上下文信息和目标项目信息输入多层神经网络获取高阶组合特征;然后通过因子分解机模块获取低阶特征;最后利用注意力机制为高低阶特征分配权重并进行加权融合,其结果输出到输出层获取预测的点击率值。在亚马逊电子、Movielens-1m和淘宝用户行为集3个公开数据集上进行实验,将TDFA模型与DeepFM、DIN、DIEN、MIAN等模型进行比较,发现TDFA模型的AUC指标值分别平均提升了1.16%、1.51%和1.10%,Logloss分别平均降低了5.40%、3.51%和3.73%,证明了该模型的有效性。

Aiming at the problem that the traditional click-through rate prediction model is insufficient in extracting user historical behavior features and ignores low-order features,a new click-through rate prediction model TDFA(TransDeepFM-AttentionBased)is proposed.First‐ly,a Transformer network is constructed based on the self-attention mechanism to obtain the correlation information among the user's histori‐cal behaviors,and then the historical behaviors are associated with the target item to obtain the user's historical behavior features.In order to get higher-order combinatorial features,the historical behavior features,user information,context information and target item information are combined into the multi-layer neural network.Second,the low-level features are obtained through the FM(Factorization Machine)module.Finally,the attention mechanism is used to allocate weights for high-low-order features,and the result of fusion is sent to the output layer to obtain the predicted click-through rate value.Three public datasets Amazon Electronics,Movielens-1m and Taobao User Behavior Set are used to test TDFA.Compared with DeepFM,DIN,DIEN and MIAN,the AUC of TDFA increased by 1.16%,1.51%and 1.10%respectively,and the Logloss decreased by 5.40%,3.51%and 3.73%respectively,which proves the effectiveness of the TDFA model.