A Novel Parameter-Optimized Recurrent Attention Network for Pipeline Leakage Detection

Accurate detection of pipeline leakage is essential to maintain the safety of pipeline transportation.Recently,deep learning(DL)has emerged as a promising tool for pipeline leakage detection(PLD).However,most existing DL methods have difficulty in achieving good performance in identifying leakage types due to the complex time dynamics of pipeline data.On the other hand,the initial parameter selection in the detection model is generally random,which may lead to unstable recognition performance.For this reason,a hybrid DL framework referred to as parameter-optimized recurrent attention network(PRAN)is presented in this paper to improve the accuracy of PLD.First,a parameter-optimized long short-term memory(LSTM)network is introduced to extract effective and robust features,which exploits a particle swarm optimization(PSO)algorithm with cross-entropy fitness function to search for globally optimal parameters.With this framework,the learning representation capability of the model is improved and the convergence rate is accelerated.Moreover,an anomaly-attention mechanism(AM)is proposed to discover class discriminative information by weighting the hidden states,which contributes to amplifying the normalabnormal distinguishable discrepancy,further improving the accuracy of PLD.After that,the proposed PRAN not only implements the adaptive optimization of network parameters,but also enlarges the contribution of normal-abnormal discrepancy,thereby overcoming the drawbacks of instability and poor generalization.Finally,the experimental results demonstrate the effectiveness and superiority of the proposed PRAN for PLD.

A Recurrent Attention and Interaction Model for Pedestrian Trajectory Prediction

The movement of pedestrians involves temporal continuity,spatial interactivity,and random diversity.As a result,pedestrian trajectory prediction is rather challenging.Most existing trajectory prediction methods tend to focus on just one aspect of these challenges,ignoring the temporal information of the trajectory and making too many assumptions.In this paper,we propose a recurrent attention and interaction(RAI)model to predict pedestrian trajectories.The RAI model consists of a temporal attention module,spatial pooling module,and randomness modeling module.The temporal attention module is proposed to assign different weights to the input sequence of a target,and reduce the speed deviation of different pedestrians.The spatial pooling module is proposed to model not only the social information of neighbors in historical frames,but also the intention of neighbors in the current time.The randomness modeling module is proposed to model the uncertainty and diversity of trajectories by introducing random noise.We conduct extensive experiments on several public datasets.The results demonstrate that our method outperforms many that are state-ofthe-art.

Deep Multimodal Reinforcement Network with Contextually GuidedRecurrent Attention for Image Question Answering

Image question answering (IQA) has emerged as a promising interdisciplinary topic in computer vision and natural language processing fields. In this paper, we propose a contextually guided recurrent attention model for solving the IQA issues. It is a deep reinforcement learning based multimodal recurrent neural network. Based on compositional contextual information, it recurrently decides where to look using reinforcement learning strategy. Different from traditional 'static' soft attention, it is deemed as a kind of 'dynamic' attention whose objective is designed based on reinforcement rewards purposefully towards IQA. The finally learned compositional information incorporates both global context and local informative details, which is demonstrated to benefit for generating answers. The proposed method is compared with several state-of-the-art methods on two public IQA datasets, including COCO-QA and VQA from dataset MS COCO. The experimental results demonstrate that our proposed model outperforms those methods and achieves better performance.

Deep Learning Applied to Computational Mechanics:A Comprehensive Review,State of the Art,and the Classics

Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.