基于改进FM算法和注意力机制的深度点击率预估模型

Deep click rate prediction model based on improved FM algorithm and attention mechanism

针对目前的广告点击率预估模型未能充分学习低阶特征且忽略了不同高阶特征对模型准确率的影响不同的问题,提出了一种基于注意力机制和深度学习的点击率预估模型。该模型采用改进因子分解机(Factorization machine,FM)算法,将全息简化表示(Holographic reduced representation,HRR)的压缩外积用于FM中,从而更好地学习低阶特征,帮助模型获得更好地表示。采用深度神经网络(Deep neural network,DNN)对高阶特征建模学习。引入注意力神经网络区分不同高阶特征交互的重要性来更好地学习高阶特征,从而得到一种能够同时有效学习到低阶特征和高阶特的点击率(Click-through rate,CTR)模型——基于改进FM算法和注意力机制的深度点击率预估模型(Deep click rate prediction model based on attention mechanism and improved FM algorithm,DAHFM)以提升模型的预估性能。在Criteo和MovieLens-1M数据集上大量的实验表明,DAHFM模型相比逻辑回归(Logistic regression,LR)、FM和DeepFM等模型不仅有效学习了特征信息,而且一定程度上提升了模型的性能和点击率的预估效果。

Aiming at the current advertising click-through rate prediction model that fails to fully learn low-level features and ignores the different effects of different high-level features on the accuracy of the model,a click-through rate prediction model based on attention mechanism and deep learning is proposed.First,the model adopts an improved factorization machine(FM)algorithm,and uses the compressed outer product of the holographic simplified representation(HRR)in FM,so as to better learn low-level features and help the model obtain a better representation.Secondly,deep neural network(DNN)is used to model and learn high-level features.Finally,the attention neural network is introduced to distinguish the importance of different high-level feature interactions to better learn high-level features,so as to obtain a click-through rate(CTR)model based on attention mechanism and improved FM algorithm(DAHFM)that can effectively learn low-level features and high-level features at the same time to improve the estimated performance of the model.A large number of experiments on the Criteo and MovieLens-1M data sets show that the DAHFM model not only effectively learns the feature information,but also improves the performance of the model and the prediction effect of click-through rate to a certain extent compared with such models as logistic regression(LR),FM and DeepFM.