Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Ocean temperature is an important physical variable in marine ecosystems,and ocean temperature prediction is an important research objective in ocean-related fields.Currently,one of the commonly used methods for ocean temperature prediction is based on data-driven,but research on this method is mostly limited to the sea surface,with few studies on the prediction of internal ocean temperature.Existing graph neural network-based methods usually use predefined graphs or learned static graphs,which cannot capture the dynamic associations among data.In this study,we propose a novel dynamic spatiotemporal graph neural network(DSTGN)to predict threedimensional ocean temperature(3D-OT),which combines static graph learning and dynamic graph learning to automatically mine two unknown dependencies between sequences based on the original 3D-OT data without prior knowledge.Temporal and spatial dependencies in the time series were then captured using temporal and graph convolutions.We also integrated dynamic graph learning,static graph learning,graph convolution,and temporal convolution into an end-to-end framework for 3D-OT prediction using time-series grid data.In this study,we conducted prediction experiments using high-resolution 3D-OT from the Copernicus global ocean physical reanalysis,with data covering the vertical variation of temperature from the sea surface to 1000 m below the sea surface.We compared five mainstream models that are commonly used for ocean temperature prediction,and the results showed that the method achieved the best prediction results at all prediction scales.

Generalized unscented Kalman filtering based radial basis function neural network for the prediction of ground radioactivity time series with missing data

On the assumption that random interruptions in the observation process are modeled by a sequence of independent Bernoulli random variables, we firstly generalize two kinds of nonlinear filtering methods with random interruption failures in the observation based on the extended Kalman filtering （EKF） and the unscented Kalman filtering （UKF）, which were shortened as GEKF and CUKF in this paper, respectively. Then the nonlinear filtering model is established by using the radial basis function neural network （RBFNN） prototypes and the network weights as state equation and the output of RBFNN to present the observation equation. Finally, we take the filtering problem under missing observed data as a special case of nonlinear filtering with random intermittent failures by setting each missing data to be zero without needing to pre-estimate the missing data, and use the GEKF-based RBFNN and the GUKF-based RBFNN to predict the ground radioactivity time series with missing data. Experimental results demonstrate that the prediction results of GUKF-based RBFNN accord well with the real ground radioactivity time series while the prediction results of GEKF-based RBFNN are divergent.

Convolutional neural networks for time series classification

Time series classification is an important task in time series data mining, and has attracted great interests and tremendous efforts during last decades. However, it remains a challenging problem due to the nature of time series data: high dimensionality, large in data size and updating continuously. The deep learning techniques are explored to improve the performance of traditional feature-based approaches. Specifically, a novel convolutional neural network (CNN) framework is proposed for time series classification. Different from other feature-based classification approaches, CNN can discover and extract the suitable internal structure to generate deep features of the input time series automatically by using convolution and pooling operations. Two groups of experiments are conducted on simulated data sets and eight groups of experiments are conducted on real-world data sets from different application domains. The final experimental results show that the proposed method outperforms state-of-the-art methods for time series classification in terms of the classification accuracy and noise tolerance. © 1990-2011 Beijing Institute of Aerospace Information.

Motor Fault Diagnosis Based on Short-time Fourier Transform and Convolutional Neural Network

With the rapid development of mechanical equipment, the mechanical health monitoring field has entered the era of big data. However, the method of manual feature extraction has the disadvantages of low efficiency and poor accuracy, when handling big data. In this study, the research object was the asynchronous motor in the drivetrain diagnostics simulator system. The vibration signals of different fault motors were collected. The raw signal was pretreated using short time Fourier transform （STFT） to obtain the corresponding time-frequency map. Then, the feature of the time-frequency map was adap- tively extracted by using a convolutional neural network （CNN）. The effects of the pretreatment method, and the hyper parameters of network diagnostic accuracy, were investigated experimentally. The experimental results showed that the influence of the preprocessing method is small, and that the batch-size is the main factor affecting accuracy and training efficiency. By investigating feature visualization, it was shown that, in the case of big data, the extracted CNN features can represent complex mapping relationships between signal and health status, and can also overcome the prior knowledge and engineering experience requirement for feature extraction, which is used by tra- ditional diagnosis methods. This paper proposes a new method, based on STFT and CNN, which can complete motor fault diagnosis tasks more intelligently and accurately.

A Hybrid Neural Network and Box-Jenkins Models for Time Series Forecasting

Time series forecasting plays a significant role in numerous applications,including but not limited to,industrial planning,water consumption,medical domains,exchange rates and consumer price index.The main problem is insufficient forecasting accuracy.The present study proposes a hybrid forecastingmethods to address this need.The proposed method includes three models.The first model is based on the autoregressive integrated moving average(ARIMA)statistical model;the second model is a back propagation neural network(BPNN)with adaptive slope and momentum parameters;and the thirdmodel is a hybridization between ARIMA and BPNN(ARIMA/BPNN)and artificial neural networks and ARIMA(ARIMA/ANN)to gain the benefits of linear and nonlinearmodeling.The forecasting models proposed in this study are used to predict the indices of the consumer price index(CPI),and predict the expected number of cancer patients in the Ibb Province in Yemen.Statistical standard measures used to evaluate the proposed method include(i)mean square error,(ii)mean absolute error,(iii)root mean square error,and(iv)mean absolute percentage error.Based on the computational results,the improvement rate of forecasting the CPI dataset was 5%,71%,and 4%for ARIMA/BPNN model,ARIMA/ANN model,and BPNN model respectively;while the result for cancer patients’dataset was 7%,200%,and 19%for ARIMA/BPNNmodel,ARIMA/ANN model,and BPNNmodel respectively.Therefore,it is obvious that the proposed method reduced the randomness degree,and the alterations affected the time series with data non-linearity.The ARIMA/ANN model outperformed each of its components when it was applied separately in terms of increasing the accuracy of forecasting and decreasing the overall errors of forecasting.

Route Temporal⁃Spatial Information Based Residual Neural Networks for Bus Arrival Time Prediction

Bus arrival time prediction contributes to the quality improvement of public transport services.Passengers can arrange departure time effectively if they know the accurate bus arrival time in advance.We proposed a machine⁃learning approach,RTSI⁃ResNet,to forecast the bus arrival time at target stations.The residual neural network framework was employed to model the bus route temporal⁃spatial information.It was found that the bus travel time on a segment between two stations not only had correlation with the preceding buses,but also had common change trends with nearby downstream/upstream segments.Two features about bus travel time and headway were extracted from bus route including target section in both forward and reverse directions to constitute the route temporal⁃spatial information,which reflects the road traffic conditions comprehensively.Experiments on the bus trajectory data of route No.10 in Shenzhen public transport system demonstrated that the proposed RTSI⁃ResNet outperformed other well⁃known methods(e.g.,RNN/LSTM,SVM).Specifically,the advantage was more significant when the distance between bus and the target station was farther.

Multi-dimension and multi-modal rolling mill vibration prediction model based on multi-level network fusion

Mill vibration is a common problem in rolling production,which directly affects the thickness accuracy of the strip and may even lead to strip fracture accidents in serious cases.The existing vibration prediction models do not consider the features contained in the data,resulting in limited improvement of model accuracy.To address these challenges,this paper proposes a multi-dimensional multi-modal cold rolling vibration time series prediction model(MDMMVPM)based on the deep fusion of multi-level networks.In the model,the long-term and short-term modal features of multi-dimensional data are considered,and the appropriate prediction algorithms are selected for different data features.Based on the established prediction model,the effects of tension and rolling force on mill vibration are analyzed.Taking the 5th stand of a cold mill in a steel mill as the research object,the innovative model is applied to predict the mill vibration for the first time.The experimental results show that the correlation coefficient(R^(2))of the model proposed in this paper is 92.5%,and the root-mean-square error(RMSE)is 0.0011,which significantly improves the modeling accuracy compared with the existing models.The proposed model is also suitable for the hot rolling process,which provides a new method for the prediction of strip rolling vibration.

A Novel Operational Partition between Neural Network Classifiers on Vulnerability to Data Mining Bias

It is difficult if not impossible to appropriately and effectively select from among the vast pool of existing neural network machine learning predictive models for industrial incorporation or academic research exploration and enhancement. When all models outperform all the others under disparate circumstances, none of the models do. Selecting the ideal model becomes a matter of ill-supported opinion ungrounded on the extant real world environment. This paper proposes a novel grouping of the model pool grounded along a non-stationary real world data line into two groups: Permanent Data Learning and Reversible Data Learning. This paper further proposes a novel approach towards qualitatively and quantitatively demonstrating their significant differences based on how they alternatively approach dynamic and raw real world data vs static and prescient data mining biased laboratory data. The results across 2040 separate simulation runs using 15,600 data points in realistically operationally controlled data environments show that the two-group division is effective and significant with clear qualitative, quantitative and theoretical support. Results across the empirical and theoretical spectrum are internally and externally consistent yet demonstrative of why and how this result is non-obvious.

A Self-Organizing Memory Neural Network for Aerosol Concentration Prediction

Haze-fog,which is an atmospheric aerosol caused by natural or man-made factors,seriously affects the physical and mental health of human beings.PM2.5(a particulate matter whose diameter is smaller than or equal to 2.5 microns)is the chief culprit causing aerosol.To forecast the condition of PM2.5,this paper adopts the related the meteorological data and air pollutes data to predict the concentration of PM2.5.Since the meteorological data and air pollutes data are typical time series data,it is reasonable to adopt a machine learning method called Single Hidden-Layer Long Short-Term Memory Neural Network(SSHL-LSTMNN)containing memory capability to implement the prediction.However,the number of neurons in the hidden layer is difficult to decide unless manual testing is operated.In order to decide the best structure of the neural network and improve the accuracy of prediction,this paper employs a self-organizing algorithm,which uses Information Processing Capability(IPC)to adjust the number of the hidden neurons automatically during a learning phase.In a word,to predict PM2.5 concentration accurately,this paper proposes the SSHL-LSTMNN to predict PM2.5 concentration.In the experiment,not only the hourly precise prediction but also the daily longer-term prediction is taken into account.At last,the experimental results reflect that SSHL-LSTMNN performs the best.

Time Series Forecasting with Multiple Deep Learners: Selection from a Bayesian Network

Considering the recent developments in deep learning, it has become increasingly important to verify what methods are valid for the prediction of multivariate time-series data. In this study, we propose a novel method of time-series prediction employing multiple deep learners combined with a Bayesian network where training data is divided into clusters using K-means clustering. We decided how many clusters are the best for K-means with the Bayesian information criteria. Depending on each cluster, the multiple deep learners are trained. We used three types of deep learners: deep neural network (DNN), recurrent neural network (RNN), and long short-term memory (LSTM). A naive Bayes classifier is used to determine which deep learner is in charge of predicting a particular time-series. Our proposed method will be applied to a set of financial time-series data, the Nikkei Average Stock price, to assess the accuracy of the predictions made. Compared with the conventional method of employing a single deep learner to acquire all the data, it is demonstrated by our proposed method that F-value and accuracy are improved.

Multivariate time series imputation for energy data using neural networks

Multivariate time series with missing values are common in a wide range of applications,including energy data.Existing imputation methods often fail to focus on the temporal dynamics and the cross-dimensional correlation simultaneously.In this paper we propose a two-step method based on an attention model to impute missing values in multivariate energy time series.First,the underlying distribution of the missing values in the data is learned.This information is then further used to train an attention based imputation model.By learning the distribution prior to the imputation process,the model can respond flexibly to the specific characteristics of the underlying data.The developed model is applied to European energy data,obtained from the European Network of Transmission System Operators for Electricity.Using different evaluation metrics and benchmarks,the conducted experiments show that the proposed model is preferable to the benchmarks and is able to accurately impute missing values.

基于数据-模型混合驱动的电力系统机电暂态快速仿真方法

数据驱动建模方法改变了发电机传统的建模范式,导致传统的机电暂态时域仿真方法无法直接应用于新范式下的电力系统。为此,该文提出一种基于数据-模型混合驱动的机电暂态时域仿真(data and physics driven time domain simulation,DPD-TDS)算法。算法中发电机状态变量与节点注入电流通过数据驱动模型推理计算,并通过网络方程完成节点电压计算,两者交替求解完成仿真。算法提出一种混合驱动范式下的网络代数方程组预处理方法,用以改善仿真的收敛性;算法设计一种中央处理器单元-神经网络处理器单元(central processing unit-neural network processing unit,CPU-NPU)异构计算框架以加速仿真,CPU进行机理模型的微分代数方程求解;NPU作协处理器完成数据驱动模型的前向推理。最后在IEEE-39和Polish-2383系统中将部分或全部发电机替换为数据驱动模型进行验证,仿真结果表明,所提出的仿真算法收敛性好,计算速度快,结果准确。

结合重构和图预测的多元时序异常检测框架

高维时序异常检测一直是智能系统安全领域的重要挑战,主流解决方案通常使用基于数据降维的重构方法和基于时序建模的预测方法,但这些方法没有结合特征间相互影响和特征内时间关联进行学习,且大多使用点估计方法进行预测或重构,从而影响了异常检测的准确性。结合预测和重构的优点,考虑序列的整体分布,提出了一种新颖的端到端异常检测框架。设计改进的变分自动编码器重构模块,以学习原始时序数据中的特征内时间关联,同时得到编码后的低维表示。设计估计高斯分布的图神经网络预测模块,结合重构模块的低维表示和原始输入进行图结构学习,以捕捉特征间的结构依赖。模型采用异常评分模块联合重构和预测模块的损失,在考虑序列整体分布的基础上进行时空联合表征。为验证所提出模型的性能,在三个工业数据集上对模型进行了对比实验,与基线模型相比,所提出的模型在F1性能指标上表现良好。

一种基于GCN的光伏短期出力预测方法研究

为提高光伏短期出力预测精确问题,提出一种基于图卷积神经网络(GCN)的光伏短期出力预测方法。首先,构建考虑多气象影响因素的光伏短期出力模型,开展光伏出力影响因素和出力特性分析。其次,对光伏出力历史时序数据进行图形化转换和数据重构,构建邻接矩阵并提取光伏短期出力图形化特征数据。在多时间尺度场景下,建立基于GCN的光伏出力预测模型,并与基于长短期记忆网络(LSTM)、反向传播网络(BP)、图注意力模型(GAT)等算法的预测模型做比对分析。最后,以某地区光伏出力实测数据开展仿真验证研究,仿真结果表明所提方法具有良好的预测效果。

稠油开采注汽锅炉在线综合预警关键技术

为了对工作在高温高压下注汽锅炉的各项运行参数进行在线准确监测和异常预警,本文在对稠油开采注汽锅炉工况参数进行采集、处理、分析的基础上,提出对注汽锅炉显性故障和隐性故障进行检测的方案。采用长期短期记忆神经网络,利用锅炉的时序数据对系统进行分析和建模,完成锅炉显性故障检测和预警,并通过预测数据的方式缓解锅炉大时滞的特性;利用深度异常检测技术,将无故障判别标准的数据进行隐性故障分析和预警。本文提出的综合预警方案对克拉玛依油田注汽锅炉进行了实验验证,预测误差仅有0.08%,同时异常检测范围也在设定值范围内。

基于谱域超图卷积网络的交通流预测模型

针对传统图结构难以对节点间的隐含复杂关联关系建模的问题,利用超图对交通流数据进行高阶表示,提出基于谱域超图卷积网络的交通流预测方法。首先,通过动态超边刻画数据特征层面的关系,利用谱域超图卷积,包括基于傅里叶和图小波的超图卷积及门控时序卷积,在多尺度上提取交通流的时空特征,实现端到端的节点级交通流预测。然后,采用北京市以及美国加利福尼亚州真实历史数据集进行预测实验。消融实验通过孤立和重构网络模型验证了所提方法的有效性。全时段和早高峰交通流预测的实验结果表明,该方法预测准确率高于目前主流交通流预测模型。

基于深度学习的农产品期货价格预测研究

提高农产品期货价格的预测能力可为投资者的投资交易和政府宏观调控提供一定借鉴.在对LSTM、GRU、BiLSTM三种深度学习模型进行对比研究的基础上,通过添加随机种子稳定预测结果、使用一阶差分降低价格预测滞后性、用正则化、回调函数等方法解决过拟合问题,对LSTM模型进行优化.利用大连商品交易所农产品期货数据,将优化后的模型应用于玉米、黄大豆1号、鸡蛋三种农产品期货的价格预测.预测结果评价指标表明,优化LSTM模型的均方根误差为17.04,平均绝对误差为13.94,误差分别降低了38.6%和33.6%.优化的深度学习模型能够用于预测农产品期货价格,为投资交易提供借鉴.

声波移动障碍反散射问题研究

研究一种具有Dirichlet边界条件的时域声波移动障碍反散射问题。首先对时域波动方程进行求解,在有限观测时间内对移动障碍物采集动态声波近场数据;其次建立一个时域声波移动障碍反散射问题的神经网络模型,该模型将时域近场数据作为输入序列,已知形状的移动障碍物位置参数作为输出序列,求解该反散射问题;最后对于不同移动速度的情况,有效地反演出移动障碍物的位置和轨迹。数值实验表明了该方法的有效性。

地磁时变观测数据中高压直流输电干扰事件多尺度表示及识别方法

高效准确地识别地磁时变观测数据中受高压直流输电干扰的波形对于提高地磁时变观测数据质量具有重要意义.然而,由于高压直流输电干扰事件持续时间长短不一、干扰程度变化多样,给识别任务带来巨大困难.为了能自动识别长短不同的高压直流输电干扰事件,本文提出一种高压输电干扰事件的多尺度表示及识别方法:利用小波技术具有多尺度的特性,卷积神经网络具有自动特征提取的特性,将二者结合,设计了一个多输入卷积神经网络模型来识别地磁中的高压直流输电干扰事件.首先使用离散小波技术将地磁时变观测样本进行多尺度分解,得到原始样本的多尺度表示,再将分解后的多尺度地磁时变观测样本分别输入到含有多个输入分支的卷积神经网络中,每个分支分别自动提取不同尺度的特征,然后将多个尺度的特征融合,并加入注意力机制来自适应计算每个尺度特征的权重,对多尺度特征进行加权处理,再采用全连接层和SoftMax层进行分类,本文将该模型命名为CBAM-MCNN.在中国地震前兆台网中心提供的高压直流输电干扰样本上进行试验,并将本文所提出模型的识别效果与现有的全卷积网络、残差神经网络、多输入卷积神经网络、IICM-HVDCT-CNN-LSTM进行了对比,在5271条测试样本集上,本文所提出的CBAM-MCNN模型识别准确率达到了97.14%,F_1值达到了97.12%,远远高于其他4种对比模型.