TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

TCAS-PINN:Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

Physics-informed neural networks(PINNs)have become an attractive machine learning framework for obtaining solutions to partial differential equations(PDEs).PINNs embed initial,boundary,and PDE constraints into the loss function.The performance of PINNs is generally affected by both training and sampling.Specifically,training methods focus on how to overcome the training difficulties caused by the special PDE residual loss of PINNs,and sampling methods are concerned with the location and distribution of the sampling points upon which evaluations of PDE residual loss are accomplished.However,a common problem among these original PINNs is that they omit special temporal information utilization during the training or sampling stages when dealing with an important PDE category,namely,time-dependent PDEs,where temporal information plays a key role in the algorithms used.There is one method,called Causal PINN,that considers temporal causality at the training level but not special temporal utilization at the sampling level.Incorporating temporal knowledge into sampling remains to be studied.To fill this gap,we propose a novel temporal causality-based adaptive sampling method that dynamically determines the sampling ratio according to both PDE residual and temporal causality.By designing a sampling ratio determined by both residual loss and temporal causality to control the number and location of sampled points in each temporal sub-domain,we provide a practical solution by incorporating temporal information into sampling.Numerical experiments of several nonlinear time-dependent PDEs,including the Cahn–Hilliard,Korteweg–de Vries,Allen–Cahn and wave equations,show that our proposed sampling method can improve the performance.We demonstrate that using such a relatively simple sampling method can improve prediction performance by up to two orders of magnitude compared with the results from other methods,especially when points are limited.

Discrete Choice Analysis of Temporal Factors on Social Network Growth

Social networks like Facebook, X (Twitter), and LinkedIn provide an interaction and communication environment for users to generate and share content, allowing for the observation of social behaviours in the digital world. These networks can be viewed as a collection of nodes and edges, where users and their interactions are represented as nodes and the connections between them as edges. Understanding the factors that contribute to the formation of these edges is important for studying network structure and processes. This knowledge can be applied to various areas such as identifying communities, recommending friends, and targeting online advertisements. Several factors, including node popularity and friends-of-friends relationships, influence edge formation and network growth. This research focuses on the temporal activity of nodes and its impact on edge formation. Specifically, the study examines how the minimum age of friends-of-friends edges and the average age of all edges connected to potential target nodes influence the formation of network edges. Discrete choice analysis is used to analyse the combined effect of these temporal factors and other well-known attributes like node degree (i.e., the number of connections a node has) and network distance between nodes. The findings reveal that temporal properties have a similar impact as network proximity in predicting the creation of links. By incorporating temporal features into the models, the accuracy of link prediction can be further improved.

Modelling infrastructure interdependencies and cascading effects using temporal networks

Lifelines are critical infrastructure systems characterized by a high level of interdependency that can lead to cascading failures after any disaster.Many approaches can be used to analyze infrastructural interdependencies,but they are usually not able to describe the sequence of events during emergencies.Therefore,interdependencies need to be modeled also taking into account the time effects.The methodology proposed in this paper is based on a modified version of the Input-output Inoperability Model and returns the probabilities of failure for each node of the system.Lifelines are modeled using graph theory,while perturbations,representing a natural or man-made disaster,are applied to the elements of the network following predetermined rules.The cascading effects among interdependent networks have been simulated using a spatial multilayer approach,while the use of an adjacency tensor allows to consider the temporal dimension and its effects.The method has been tested on a case study based on the 2011 Fukushima Dai-ichi nuclear disaster.Different configurations of the system have been analyzed and their probability of occurrence evaluated.Two models of the nuclear power plant have been developed to evaluate how different spatial scales and levels of detail affect the results.

A Spatial-Temporal Network Perspective for the Propagation Dynamics of Air Traffic Delays

Intractable delays occur in air traffic due to the imbalance between ever-increasing air traffic demand and limited airspace capacity.As air traffic is associated with complex air transport systems,delays can be magnified and propagated throughout these systems,resulting in the emergent behavior known as delay propagation.An understanding of delay propagation dynamics is pertinent to modern air traffic management.In this work,we present a complex network perspective of delay propagation dynamics.Specifically,we model air traffic scenarios using spatial–temporal networks with airports as the nodes.To establish the dynamic edges between the nodes,we develop a delay propagation method and apply it to a given set of air traffic schedules.Based on the constructed spatial-temporal networks,we suggest three metrics-magnitude,severity,and speed-to gauge delay propagation dynamics.To validate the effectiveness of the proposed method,we carry out case studies on domestic flights in the Southeastern Asia region(SAR)and the United States.Experiments demonstrate that the propagation magnitude in terms of the number of flights affected by delay propagation and the amount of propagated delays for the US traffic are respectively five and ten times those of the SAR.Experiments further reveal that the propagation speed for US traffic is eight times faster than that of the SAR.The delay propagation dynamics reveal that about six hub airports in the SAR have significant propagated delays,while the situation in the United States is considerably worse,with a corresponding number of around 16.This work provides a potent tool for tracing the evolution of air traffic delays.

Spectrum Sensing via Temporal Convolutional Network

In this paper,we investigate a spectrumsensing system in the presence of a satellite,where the satellite works as a sensing node.Considering the conventional energy detection method is sensitive to the noise uncertainty,thus,a temporal convolutional network(TCN)based spectrum-sensing method is designed to eliminate the effect of the noise uncertainty and improve the performance of spectrum sensing,relying on the offline training and the online detection stages.Specifically,in the offline training stage,spectrum data captured by the satellite is sent to the TCN deployed on the gateway for training purpose.Moreover,in the online detection stage,the well trained TCN is utilized to perform real-time spectrum sensing,which can upgrade spectrum-sensing performance by exploiting the temporal features.Additionally,simulation results demonstrate that the proposed method achieves a higher probability of detection than that of the conventional energy detection(ED),the convolutional neural network(CNN),and deep neural network(DNN).Furthermore,the proposed method outperforms the CNN and the DNN in terms of a lower computational complexity.

Temporal Analysis of the Diffusion of Knowledge in Networks of Software Maintenance and Development Project Team

Different approaches have been established for applications of social and complex networks involving biological systems, passing through collaborative systems in knowledge networks and organizational systems. In this latter application, we highlight the studies focused on the diffusion of information and knowledge in networks. However, most of the time, the propagation of information in these networks and the resulting process of creation and diffusion of knowledge, have been studied from static perspectives. Additionally, the very concept of diffusion inevitably implies the inclusion of the temporal dimension, due to that it is an essentially dynamic process. Although static analysis provides an important perspective in structural terms, the behavioral view that reflects the evolution of the relationships of the members of these networks over time is best described by temporal networks. Thus, it is possible to analyze both the information flow and the structural changes that occur over time, which influences the dynamics of the creation and diffusion of knowledge. This article describes the computational modeling used to elucidate the creation and diffusion of knowledge in temporal networks formed to execute software maintenance and construction projects, for the period between 2007 and 2013, in the SERVIÇO FEDERAL DE PROCESSAMENTO DE DADOS (FEDERAL DATA PROCESSING SERVICE-SERPRO)—a public organization that provides information and communication technology services. The methodological approach adopted for the study was based on techniques for analyzing social and complex networks and on the complementary extensions that address temporal modeling of these networks. We present an exploratory longitudinal study that enabled a dynamic and structural analysis of the knowledge networks formed by members of software maintenance and development project teams between 2007 and 2013. The study enabled identification of knowledge categories throughout this period, in addition to the determination that the networks have a structure with small-world and scale-free models. Finally, we concluded that, in general, the topologies of the networks studies had characteristics for facilitating the flow of knowledge within the organization.

Performance Characterization and Receiver Design for Random Temporal Multiple Access in Non-Coordinated Networks

Random access is a well-known multiple access method for uncoordinated communication nodes.Existing work mainly focuses on optimizing iterative access protocols,assuming that packets are corrupted once they are collided,or that feedback is available and can be exploited.In practice,a packet may still be able to be recovered successfully even when collided with other packets.System design and performance analysis under such a situation,particularly when the details of collision are taken into consideration,are less known.In this paper,we provide a framework for analytically evaluating the actual detection performance in a random temporal multiple access system where nodes can only transmit.Explicit expressions are provided for collision probability and signal to interference and noise ratio(SINR)when different numbers of packets are collided.We then discuss and compare two receiver options for the AP,and provide detailed receiver design for the premium one.In particular,we propose a synchronization scheme which can largely reduce the preamble length.We also demonstrate that system performance could be a convex function of preamble length both analytically and via simulation,as well as the forward error correction(FEC)coding rate.

Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extract knowledge from these sources is imperative.Recently,the BlazePose system has been released for skeleton extraction from images oriented to mobile devices.With this skeleton graph representation in place,a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action.We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest,it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks.Hence,in this study,we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition.Moreover,we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor.Additionally,we propose different skeleton detection thresholds that can improve the accuracy performance even further.We reached a top-1 accuracy performance of 40.1%on the Kinetics dataset.For the NTU-RGB+D dataset,we achieved 87.59%and 92.1%accuracy for Cross-Subject and Cross-View evaluation criteria,respectively.

A production prediction method of single well in water flooding oilfield based on integrated temporal convolutional network model

Since the oil production of single well in water flooding reservoir varies greatly and is hard to predict, an oil production prediction method of single well based on temporal convolutional network(TCN) is proposed and verified. This method is started from data processing, the correspondence between water injectors and oil producers is determined according to the influence radius of the water injectors, the influence degree of a water injector on an oil producer in the month concerned is added as a model feature, and a Random Forest(RF) model is built to fill the dynamic data of water flooding. The single well history is divided into 4 stages according to its water cut, that is, low water cut, middle water cut, high water cut and extra-high water cut stages. In each stage, a TCN based prediction model is established, hyperparameters of the model are optimized by the Sparrow Search Algorithm(SSA). Finally, the models of the 4 stages are integrated into one whole-life model of the well for production prediction. The application of this method in Daqing Oilfield, NE China shows that:(1) Compared with conventional data processing methods, the data obtained by this processing method are more close to the actual production, and the data set obtained is more authentic and complete.(2) The TCN model has higher prediction accuracy than other 11 models such as Long Short Term Memory(LSTM).(3) Compared with the conventional full-life-cycle models, the model of integrated stages can significantly reduce the error of production prediction.

Temporal Data Mining Using Genetic Algorithm and Neural Network——A Case Study of Air Pollutant Forecasts

This paper integrates genetic algorithm and neura l network techniques to build new temporal predicting analysis tools for geographic information system (GIS). These new GIS tools can be readily applied in a practical and appropriate manner in spatial and temp oral research to patch the gaps in GIS data mining and knowledge discovery functions. The specific achievement here is the integration of related artificial intellig ent technologies into GIS software to establish a conceptual spatial and temporal analysis framework. And, by using this framework to develop an artificial intelligent spatial and tempor al information analyst (ASIA) system which then is fully utilized in the existin g GIS package. This study of air pollutants forecasting provides a geographical practical case to prove the rationalization and justness of the conceptual tempo ral analysis framework.

Continuous Sign Language Recognition Based on Spatial-Temporal Graph Attention Network

Continuous sign language recognition(CSLR)is challenging due to the complexity of video background,hand gesture variability,and temporal modeling difficulties.This work proposes a CSLR method based on a spatialtemporal graph attention network to focus on essential features of video series.The method considers local details of sign language movements by taking the information on joints and bones as inputs and constructing a spatialtemporal graph to reflect inter-frame relevance and physical connections between nodes.The graph-based multihead attention mechanism is utilized with adjacent matrix calculation for better local-feature exploration,and short-term motion correlation modeling is completed via a temporal convolutional network.We adopted BLSTM to learn the long-termdependence and connectionist temporal classification to align the word-level sequences.The proposed method achieves competitive results regarding word error rates(1.59%)on the Chinese Sign Language dataset and the mean Jaccard Index(65.78%)on the ChaLearn LAP Continuous Gesture Dataset.

A Lightweight Temporal Convolutional Network for Human Motion Prediction

A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain the spatial structure information of human motion and extract the correlation in the time series of human motion.The residual structure is applied to the proposed network model to alleviate the problem of gradient disappearance in the deep network.Experiments on the Human 3.6M dataset demonstrate that the proposed method effectively reduces the errors of motion prediction compared with previous methods,especially of long-term prediction.

An Influence Maximization Algorithm Based on Improved K-Shell in Temporal Social Networks

Influence maximization of temporal social networks(IMT)is a problem that aims to find the most influential set of nodes in the temporal network so that their information can be the most widely spread.To solve the IMT problem,we propose an influence maximization algorithm based on an improved K-shell method,namely improved K-shell in temporal social networks(KT).The algorithm takes into account the global and local structures of temporal social networks.First,to obtain the kernel value Ks of each node,in the global scope,it layers the network according to the temporal characteristic of nodes by improving the K-shell method.Then,in the local scope,the calculation method of comprehensive degree is proposed to weigh the influence of nodes.Finally,the node with the highest comprehensive degree in each core layer is selected as the seed.However,the seed selection strategy of KT can easily lose some influential nodes.Thus,by optimizing the seed selection strategy,this paper proposes an efficient heuristic algorithm called improved K-shell in temporal social networks for influence maximization(KTIM).According to the hierarchical distribution of cores,the algorithm adds nodes near the central core to the candidate seed set.It then searches for seeds in the candidate seed set according to the comprehensive degree.Experiments showthatKTIMis close to the best performing improved method for influence maximization of temporal graph(IMIT)algorithm in terms of effectiveness,but runs at least an order of magnitude faster than it.Therefore,considering the effectiveness and efficiency simultaneously in temporal social networks,the KTIM algorithm works better than other baseline algorithms.

Route Temporal⁃Spatial Information Based Residual Neural Networks for Bus Arrival Time Prediction

Bus arrival time prediction contributes to the quality improvement of public transport services.Passengers can arrange departure time effectively if they know the accurate bus arrival time in advance.We proposed a machine⁃learning approach,RTSI⁃ResNet,to forecast the bus arrival time at target stations.The residual neural network framework was employed to model the bus route temporal⁃spatial information.It was found that the bus travel time on a segment between two stations not only had correlation with the preceding buses,but also had common change trends with nearby downstream/upstream segments.Two features about bus travel time and headway were extracted from bus route including target section in both forward and reverse directions to constitute the route temporal⁃spatial information,which reflects the road traffic conditions comprehensively.Experiments on the bus trajectory data of route No.10 in Shenzhen public transport system demonstrated that the proposed RTSI⁃ResNet outperformed other well⁃known methods(e.g.,RNN/LSTM,SVM).Specifically,the advantage was more significant when the distance between bus and the target station was farther.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Occluded Gait Emotion Recognition Based on Multi-Scale Suppression Graph Convolutional Network

In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accuracy significantly declines when the data is occluded.To enhance the accuracy of gait emotion recognition under occlusion,this paper proposes a Multi-scale Suppression Graph ConvolutionalNetwork(MS-GCN).TheMS-GCN consists of three main components:Joint Interpolation Module(JI Moudle),Multi-scale Temporal Convolution Network(MS-TCN),and Suppression Graph Convolutional Network(SGCN).The JI Module completes the spatially occluded skeletal joints using the(K-Nearest Neighbors)KNN interpolation method.The MS-TCN employs convolutional kernels of various sizes to comprehensively capture the emotional information embedded in the gait,compensating for the temporal occlusion of gait information.The SGCN extracts more non-prominent human gait features by suppressing the extraction of key body part features,thereby reducing the negative impact of occlusion on emotion recognition results.The proposed method is evaluated on two comprehensive datasets:Emotion-Gait,containing 4227 real gaits from sources like BML,ICT-Pollick,and ELMD,and 1000 synthetic gaits generated using STEP-Gen technology,and ELMB,consisting of 3924 gaits,with 1835 labeled with emotions such as“Happy,”“Sad,”“Angry,”and“Neutral.”On the standard datasets Emotion-Gait and ELMB,the proposed method achieved accuracies of 0.900 and 0.896,respectively,attaining performance comparable to other state-ofthe-artmethods.Furthermore,on occlusion datasets,the proposedmethod significantly mitigates the performance degradation caused by occlusion compared to other methods,the accuracy is significantly higher than that of other methods.

Adaptive Modeling and Forecasting of Time Series by Combining the Methods of Temporal Differences with Neural Networks

This paper discusses the modeling method of time series with neural network. In order to improve the adaptability of direct multi-step prediction models, this paper proposes a method of combining the temporal differences methods with back-propagation algorithm for updating the parameters continuously on the basis of recent data. This method can make the neural network model fit the recent characteristic of the time series as close as possible, therefore improves the prediction accuracy. We built models and made predictions for the sunspot series. The prediction results of adaptive modeling method are better than that of non-adaptive modeling methods.

Spatiotemporal Input Control:Leveraging Temporal Variation in Network Dynamics

The number of available control sources is a limiting factor to many network control tasks.A lack of input sources can result in compromised controllability and/or sub-optimal network performance,as noted in engineering applications such as the smart grids.The mechanism can be explained by a linear timeinvariant model,where structural controllability sets a lower bound on the number of required sources.Inspired by the ubiquity of time-varying topologies in the real world,we propose the strategy of spatiotemporal input control to overcome the source-related limit by exploiting temporal variation of the network topology.We theoretically prove that under this regime,the required number of sources can always be reduced to 2.It is further shown that the cost of control depends on two hyperparameters,the numbers of sources and intervals,in a trade-off fashion.As a demonstration,we achieve controllability over a complex network resembling the nervous system of Caenorhabditis elegans using as few as 6%of the sources predicted by a static control model.This example underlines the potential of utilizing topological variation in complex network control problems.

Shortest path of temporal networks:An information spreading-based approach

The shortest path is a widely studied network science problem and has attracted great attention.Nevertheless,it draws little attention in temporal networks,in which temporal edges determine information dissemination.In this paper,we propose an information spreading-based method to calculate the shortest paths distribution in temporal networks.We verify our method on both artificial and real-world temporal networks and obtain a good agreement.We further generalize our method to identify influential nodes and found an effective method.Finally,we verify the influential nodes identifying method on four networks.