基于融合结构的在线广告点击率预测模型

A Hybrid Network Based CTR Prediction Model for Online Advertising

点击率预测作为推荐系统和在线广告的关键环节,在学术界和工业界均受到了极大的关注.论文首先对几种典型的点击率预测模型进行研究,然后探索了基于融合结构的深度学习方法,并在此基础上提出一种基于融合结构的点击率预测模型,该模型能够灵活融合不同结构的深度神经网络来分别学习原始高维稀疏特征的高阶表示,从而使点击率预测模型能够利用更丰富的高阶特征信息.论文利用真实数据集来评价模型的预测性能,实验结果显示,基于融合结构的深度学习预测模型,能够比传统的点击率预测模型以及最新的基于深度学习的预测模型获得更好的性能。

As the key component of recommender system and online advertising, the click-through rate (CTR) prediction has received great attention in both the academia and the industry. The most common approaches to CTR prediction are regarding it as a regression prediction task in machine learning. At beginning, simple models like logistic regression (LR) and factorization machine (FM) are used to do predictions, however the prediction performances are not so good because only low-order feature interactions are explored. Therefore, models with stronger ability of feature representation learning are developed, for example, a Factorization-machine supported Neural Network (FNN) and a Product-based Neural Network (PNN), which are promising to exploit deep neural networks to learn sophisticated and selective feature interactions. The major downside of FNN and PNN is that they focus more on high-order feature interactions while capture little low-order interactions. In order to make full use of low- and high-order feature interactions, some hybrid architectures are proposed, containing both a shallow component and a deep component. In this paper, we firstly study several typical CTR prediction models, especially the deep learning models based on hybrid architectures, to describe the development process of CTR prediction;and then, inspired by existing works, a new click-through rate prediction model based on a hybrid network is proposed - DPSN (Deep & Product supported Stacking Network). The new model can integrate different deep neural networks (DNNs) to learn the high-order representation of original high-dimensional sparse features respectively, which enables the prediction model to take advantage of more abundant information of high-order feature interactions. In addition, we also design a new embedding layer for DPSN, where nodes come from not only the embedding vector but also the weight of each feature, which are both pre - trained by FM model. To our best knowledge, this is the first attempt to improve the prediction performance by adding few weight nodes in the embedding layer. Furthermore, a simplified analysis of the parameter complexity is given;meanwhile, the convergence of the DPSN model is analyzed and proved. We evaluate the prediction performance of the proposed model based on two real-world data sets, iPinYou and Criteo, by using LogLoss and AUC metrics. In the first and second experiments, we verify the convergence of the DPSN model and illustrate the performance improvements of FNN and PNN by adding the weight nodes of each feature in the embedding layer. In the third experiment, we analyze the influences of different model parameters on the prediction performance of DPSN, including the number of hidden layers, hidden layer nodes, activation function, and embedded vector dimension. The fourth experiment is used to evaluate the effects of the new embedding layer on the prediction performance of the DPSN model. The fifth and sixth experiments respectively compare the influences of different architectures and negative sampling ratios on DPSN prediction performance. The last experiment is to compare the performance of the DPSN model with other typical CTR prediction models, and the experimental results demonstrate that the new model has better performance than major stat-of-the - art models on LogLoss metrics and AUC metrics.