点击率的深度交叉注意力预估模型

Add to Favorite Get Latest Update Prediction model of deep cross attention based on click through rate

为了区分不同高阶交叉特征的重要程度并剔除冗余交叉特征,提高点击率的预估精度,提出了一种深度交叉注意力预估网络(deep cross attention prediction network,DAPN)模型。它将具有稀疏高维特征的输入信息表示为低维稠密向量后,分别送入因子分解机(factorization machine,FM)和深度交叉注意力层(deep cross attention,DCA)。FM通过一阶特征和二阶特征交叉挖掘训练数据中从未出现或者很少出现的低阶交叉特征。DCA层利用缩放点积注意力机制(dot-product attention,DP_Att)设计交叉注意力层,用于区分高阶交叉特征的重要度,并设计深度神经网络(deep neural network,DNN)实现对高阶交叉特征建模。仿真表明,DAPN在MovieLens-1m和Avazu等公开数据集上均具有良好的预估性能,它使用并行结构能同时有效地学习低阶和高阶交叉特征,提高预估精度。

In order to distinguish the importance of different high-order crossover features and eliminate redundant cross features,and improve the accuracy of click through rate,a deep cross attention prediction network(DAPN)algorithm is proposed.DAPN represents the input information with sparse high-dimensional features as low-dimensional dense vectors and sends them to the factorization machine(FM)and deep cross attention(DCA)layer respectively.FM uses first-order features and second-order features to cross mine low-order cross features that never appear or rarely appear in training data.DCA layer uses scaled dot-product attention mechanism(DP_Att)to design cross attention layer,distinguish the importance of high-order cross features,and design the deep neural network(DNN)to model the high-order cross features.Simulation results show that DAPN has good prediction performance on public data sets,such as Movielens-1m and Avazu.DAPN uses parallel structure to effectively learn both low-order and high-order crossover features and improve estimation accuracy.