面向季节性时空数据的预测式循环网络及其在城市计算中的应用

Predictive Recurrent Networks for Seasonal Spatiotemporal Data with Applications to Urban Computing

实际生活中有很多带有季节特征的时空数据,在城市计算领域分布尤广,例如交通流量数据便具有较为明显的以天或周为周期的统计学特征.如何有效利用这种季节特征,如何捕捉历史观测与待预测数据之间的相关性,成为了预测此类时空数据未来变化趋势的关键.传统时序建模方法将时序数据分解为多个信号分量,并使用线性模型来进行预测.此类方法具有较强的理论基础,但对于数据的平稳性要求过于严格,难以预测趋势信息复杂的数据,更不适用于高维的时空数据.然而在真实场景下,季节性时空数据的周期长短可变,且不同周期的对应关系往往并不固定,存在时间、空间上的模式变化与偏移,很难作为理想的周期信号以传统时序方法建模.相比之下,深度神经网络建模能力更强,可拟合更为复杂的数据.近几年有许多工作研究了如何利用卷积神经网络和循环神经网络来处理时空数据,也有一些工作讨论了如何有效利用周期性信息提升预测的准确性.但深度神经网络受困于梯度消失和误差累积,难以捕捉时序数据中的长时间依赖,且少有方法讨论如何在深度神经网络中有效建模上述具有弹性周期对应关系的时空信号.本文针对真实场景下季节性时空数据的上述问题,给出具有弹性周期对应关系的时空数据预测问题的形式化定义,并提出了一种新的季节性时空数据预测模型.该模型包含季节网络、趋势网络、时空注意力模块三个部分,可捕捉短期数据中的临近变化趋势和长期数据中隐含的季节性趋势,并广泛考虑历史周期中的每个时空元素对未来预测值的影响.为了解决深度循环网络难以捕捉时序数据中的长时间依赖的问题,本文提出一种新的循环卷积记忆单元,该单元将上述模块融合于一个可端到端训练的神经网络中,一方面实现了时间和空间信息统一建模,另一方面实现了短期趋势特征与历史周期特征的统一建模.进一步地,为了解决季节性数据中的各周期时空元素对应关系不固定的问题,本文探讨了多种基于注意力模块的时空数据融合方式,创新性地提出一种级联式的时空注意力模块,并将其嵌入于上述循环卷积记忆单元内.该模块建模记忆单元的隐藏状态在不同周期内的弹性时空对应关系,自适应地选取相关度高的季节性特征辅助预测.实验部分,我们选取了两个时空数据预测在城市计算中最为典型的应用:交通流量预测和气象数据预报.本文所提出的时空周期性循环神经网络在北京、纽约的交通流量数据集、美国气象数据集上均取得了目前最高的预测准确性.

There are many real-world spatiotemporal data formats with seasonal statistical patterns,which are widely distributed in the field of urban computing.For example,the traffic flow data has significant periodic statistical characteristics with days or weeks.How to effectively use this seasonal feature,how to capture the correlation between historical observations and data to be predicted becomes the key to predicting the future trend of such spatiotemporal data.Traditional time series modeling methods decompose time series data into multiple signal components and use linear models to make predictions.These methods have strong theoretical bases,but they also have too strict assumptions on the stationarity,so that they can be hardly used for sequential data with complex variations,let alone for the high-dimensional spatiotemporal data.However,in real-world scenarios,the periods of seasonal spatiotemporal data are variable,and the correspondence between different periods is often not fixed.There are temporal and spatial changes and offsets.Thus,it is difficult to model it as an ideal seasonal time series using traditional time series methods.In contrast,deep neural networks are more powerful and can fit more complex data.In recent years,there have been many papers studying how to use convolutional neural networks and recurrent neural networks to process spatiotemporal data.Some work has also discussed how to effectively use the periodic information to improve the accuracy of prediction.However,deep neural networks can easily suffer from gradient vanishing and error accumulation,which makes it difficult to capture long-term dependencies of time series data.Moreover,there have been few methods in deep neural networks that discuss how to effectively model the above-mentioned spatiotemporal signals with elastic periodic correspondence.In this paper,based on the above problems of seasonal spatiotemporal data in real-world scenarios,we give the formal definition of spatiotemporal data prediction problem with elastic period correspondences and propose a new seasonal spatiotemporal data prediction model.The model consists of a seasonal network,a trend network,and a space-time attention module,which can capture the near-term trends in short-term data and the seasonal trends implied in long-term data,and widely consider the impact of each space-time element in the historical cycles to the predicted future data values.To solve the problem that the deep recurrent neural networks are difficult to capture long-term dependencies,this paper proposes a new recurrent convolutional memory unit,which combines the above modules into an end-to-end trainable neural network.It not only models the temporal and spatial information simultaneously but also the short-term trends and historical periodic trends.Furthermore,to solve the problem that the correspondences between elements in each cycle of the seasonal data are not strictly fixed,this paper discusses a variety of space-time data fusion methods based on different attention modules,proposes a new cascaded space-time attention module,and integrates it within the above recurrent convolution memory unit.The module models the elastic space-time correspondences of the hidden states in different cycles,and adaptively selects the seasonal features with high correlations to assist future prediction.In the experimental part,we select two typical applications of spatiotemporal prediction in urban computing:traffic flow prediction and meteorological data forecasting.Our proposed model has achieved the highest prediction accuracy in Beijing and New York traffic flow datasets,as well as a US meteorological dataset.