Effects of data smoothing and recurrent neural network(RNN)algorithms for real-time forecasting of tunnel boring machine(TBM)performance

Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.

Optimized Phishing Detection with Recurrent Neural Network and Whale Optimizer Algorithm

Phishing attacks present a persistent and evolving threat in the cybersecurity land-scape,necessitating the development of more sophisticated detection methods.Traditional machine learning approaches to phishing detection have relied heavily on feature engineering and have often fallen short in adapting to the dynamically changing patterns of phishingUniformResource Locator(URLs).Addressing these challenge,we introduce a framework that integrates the sequential data processing strengths of a Recurrent Neural Network(RNN)with the hyperparameter optimization prowess of theWhale Optimization Algorithm(WOA).Ourmodel capitalizes on an extensive Kaggle dataset,featuring over 11,000 URLs,each delineated by 30 attributes.The WOA’s hyperparameter optimization enhances the RNN’s performance,evidenced by a meticulous validation process.The results,encapsulated in precision,recall,and F1-score metrics,surpass baseline models,achieving an overall accuracy of 92%.This study not only demonstrates the RNN’s proficiency in learning complex patterns but also underscores the WOA’s effectiveness in refining machine learning models for the critical task of phishing detection.

Hyperparameter Tuning for Deep Neural Networks Based Optimization Algorithm

For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.

Remaining Useful Life Prediction for a Roller in a Hot Strip Mill Based on Deep Recurrent Neural Networks

Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.

New Stability Criteria for Recurrent Neural Networks with a Time-varying Delay

This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.

Robust exponential stability analysis of a larger class of discrete-time recurrent neural networks

The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.

Global stability of interval recurrent neural networks

The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robust stability of interval RNNs is transformed into a problem of solving a class of linear matrix inequalities.Thus,the robust stability of interval RNNs can be analyzed by directly using the linear matrix inequalities（LMI） toolbox of MATLAB.Numerical example is given to show the effectiveness of the obtained results.

A Prediction Method of Trend-Type Capacity Index Based on Recurrent Neural Network

Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes it very difficult to perform high-precision capacity prediction.In order to improve the forecasting efficiency of related indexes,this paper designs a classification method of capacity index data,which divides the capacity index data into trend type,periodic type and irregular type.Then for the prediction of trend data,it proposes a capacity index prediction model based on Recurrent Neural Network(RNN),denoted as RNN-LSTM-LSTM.This model includes a basic RNN,two Long Short-Term Memory(LSTM)networks and two Fully Connected layers.The experimental results show that,compared with the traditional Holt-Winters,Autoregressive Integrated Moving Average(ARIMA)and Back Propagation(BP)neural network prediction model,the mean square error(MSE)of the proposed RNN-LSTM-LSTM model are reduced by 11.82%and 20.34%on the order storage and data migration,which has greatly improved the efficiency of trend-type capacity index prediction.

Multimodal emotion recognition based on deep neural network

In order to increase the accuracy rate of emotion recognition in voiceand video,the mixed convolutional neural network(CNN)and recurrent neural network(RNN)ae used to encode and integrate the two information sources.For the audio signals,several frequency bands as well as some energy functions are extacted as low-level features by using a sophisticated audio technique,and then they are encoded w it a one-dimensional(I D)convolutional neural network to abstact high-level features.Finally,tiese are fed into a recurrent neural network for te sake of capturing dynamic tone changes in a temporal dimensionality.As a contrast,a two-dimensional(2D)convolutional neural network and a similar RNN are used to capture dynamic facial appearance changes of temporal sequences.The method was used in te Chinese Natral Audio-'Visual Emotion Database in te Chinese Conference on Pattern Recognition(CCPR)in2016.Experimental results demonstrate that te classification average precision of the proposed metiod is41.15%,which is increased by16.62%compaed with te baseline algorithm offered by the CCPR in2016.It is proved ta t te proposed method has higher accuracy in te identification of emotional information.

Deep-fake video detection approaches using convolutional–recurrent neural networks

Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural networks employed for Deep-Fake video detection.In the model,the features from the training data are extracted with the intended Convolution Neural Network model to form feature vectors which are further analysed using a dense layer,a Long Short-Term Memoryand Gated Recurrent by adopting transfer learning with fine tuning for training the models.The model is evaluated to detect Artificial Intelligence based Deep fakes images and videos using benchmark datasets.Comparative analysis shows that the detections are majorly biased towards domain of the dataset but there is a noteworthy improvement in the model performance parameters by using Transfer Learning whereas Convolutional-Recurrent Neural Network has benefits in sequence detection.

Two-Phase Rate Adaptation Strategy for Improving Real-Time Video QoE in Mobile Networks

With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience(Qo E) has made rate adaptation methods very attractive. In this paper, we propose a two-phase rate adaptation strategy to improve users' real-time video Qo E. First, to measure and assess video Qo E, we provide a continuous Qo E prediction engine modeled by RNN recurrent neural network. Different from traditional Qo E models which consider the Qo E-aware factors separately or incompletely, our RNN-Qo E model accounts for three descriptive factors(video quality, rebuffering, and rate change) and reflects the impact of cognitive memory and recency. Besides, the video playing is separated into the initial startup phase and the steady playback phase, and we takes different optimization goals for each phase: the former aims at shortening the startup delay while the latter ameliorates the video quality and the rebufferings. Simulation results have shown that RNN-Qo E can follow the subjective Qo E quite well, and the proposed strategy can effectively reduce the occurrence of rebufferings caused by the mismatch between the requested video rates and the fluctuated throughput and attains standout performance on real-time Qo E compared with classical rate adaption methods.

Fake News Classification: Past, Current, and Future

The proliferation of deluding data such as fake news and phony audits on news web journals,online publications,and internet business apps has been aided by the availability of the web,cell phones,and social media.Individuals can quickly fabricate comments and news on social media.The most difficult challenge is determining which news is real or fake.Accordingly,tracking down programmed techniques to recognize fake news online is imperative.With an emphasis on false news,this study presents the evolution of artificial intelligence techniques for detecting spurious social media content.This study shows past,current,and possible methods that can be used in the future for fake news classification.Two different publicly available datasets containing political news are utilized for performing experiments.Sixteen supervised learning algorithms are used,and their results show that conventional Machine Learning(ML)algorithms that were used in the past perform better on shorter text classification.In contrast,the currently used Recurrent Neural Network(RNN)and transformer-based algorithms perform better on longer text.Additionally,a brief comparison of all these techniques is provided,and it concluded that transformers have the potential to revolutionize Natural Language Processing(NLP)methods in the near future.

Continuous Mobile User Authentication Using a Hybrid CNN-Bi-LSTM Approach

Internet of Things (IoT) devices incorporate a large amount ofdata in several fields, including those of medicine, business, and engineering.User authentication is paramount in the IoT era to assure connecteddevices’ security. However, traditional authentication methods and conventionalbiometrics-based authentication approaches such as face recognition,fingerprints, and password are vulnerable to various attacks, including smudgeattacks, heat attacks, and shoulder surfing attacks. Behavioral biometrics isintroduced by the powerful sensing capabilities of IoT devices such as smartwearables and smartphones, enabling continuous authentication. ArtificialIntelligence (AI)-based approaches introduce a bright future in refining largeamounts of homogeneous biometric data to provide innovative user authenticationsolutions. This paper presents a new continuous passive authenticationapproach capable of learning the signatures of IoT users utilizing smartphonesensors such as a gyroscope, magnetometer, and accelerometer to recognizeusers by their physical activities. This approach integrates the convolutionalneural network (CNN) and recurrent neural network (RNN) models to learnsignatures of human activities from different users. A series of experiments areconducted using the MotionSense dataset to validate the effectiveness of theproposed method. Our technique offers a competitive verification accuracyequal to 98.4%.We compared the proposed method with several conventionalmachine learning and CNN models and found that our proposed modelachieves higher identification accuracy than the recently developed verificationsystems. The high accuracy achieved by the proposed method proves itseffectiveness in recognizing IoT users passively through their physical activitypatterns.

Treatment of Imbalance Dataset for Human Emotion Classification

Developments in biomedical science, signal processing technologies have led Electroencephalography (EEG) signals to be widely used in the diagnosis of brain disease and in the field of Brain-Computer Interface (BCI). The collected EEG signals are processed using Machine Learning-Random Forest and Naive Bayes- and Deep Learning-Recurrent Neural Network (RNN), Neural Network (NN) and Long Short Term Memory (LSTM)-Algorithms to obtain the recent mood of a person. The Algorithms mentioned above have been imposed on the data set in order to find out what the person is feeling at a particular moment. The following thesis is conducted to find out one of the following moods (happy, surprised, disgust, fear, anger and sadness) of a person at an instant, with an aim to obtain the result with least amount of time delay as the mood differs. It is pretty obvious that the accuracy of the output varies depending upon the algorithm used, time taken to process the data, so that it is easy for us to compare the reliability and dependency of a particular algorithm to another, prior to its practical implementation. The imbalance data sets that were used had an imbalanced class and thus, over fitting occurred. This problem was handled by generating Artificial Data sets with the use of SMOTE Oversampling Technique.

Recurrent Neural Network for Nonconvex Economic Emission Dispatch

In this paper, an economic emission dispatch(EED) model is developed to reduce fuel cost and environmental pollution emissions. Considering the development of new energy sources in recent years, the EED problem involves thermal units with the valve point effect and WTs. Meanwhile, it complies with demand constraint and generator capacity constraints. A recurrent neural network(RNN) is proposed to search for local optimal solution of the introduced nonconvex EED problem. The optimality and convergence of the proposed dynamic model are given. The RNN algorithm is verified on a power generation system for the optimization of scheduling and minimization of total cost. Moreover, a particle swarm optimization(PSO) algorithm is compared with RNN under the same problematic frame. Numerical simulation results demonstrate that the optimal scheduling given by RNN is more precise and has lower total cost than PSO. In addition, the dynamic variation of power load demand is considered and the power distribution of eight generators during 12 time periods is depicted.

Tunnel boring machine vibration-based deep learning for the ground identification of working faces

Tunnel boring machine(TBM) vibration induced by cutting complex ground contains essential information that can help engineers evaluate the interaction between a cutterhead and the ground itself.In this study,deep recurrent neural networks(RNNs) and convolutional neural networks(CNNs) were used for vibration-based working face ground identification.First,field monitoring was conducted to obtain the TBM vibration data when tunneling in changing geological conditions,including mixed-face,homogeneous,and transmission ground.Next,RNNs and CNNs were utilized to develop vibration-based prediction models,which were then validated using the testing dataset.The accuracy of the long short-term memory(LSTM) and bidirectional LSTM(Bi-LSTM) models was approximately 70% with raw data;however,with instantaneous frequency transmission,the accuracy increased to approximately 80%.Two types of deep CNNs,GoogLeNet and ResNet,were trained and tested with time-frequency scalar diagrams from continuous wavelet transformation.The CNN models,with an accuracy greater than 96%,performed significantly better than the RNN models.The ResNet-18,with an accuracy of 98.28%,performed the best.When the sample length was set as the cutterhead rotation period,the deep CNN and RNN models achieved the highest accuracy while the proposed deep CNN model simultaneously achieved high prediction accuracy and feedback efficiency.The proposed model could promptly identify the ground conditions at the working face without stopping the normal tunneling process,and the TBM working parameters could be adjusted and optimized in a timely manner based on the predicted results.

Automated Facial Expression Recognition and Age Estimation Using Deep Learning

With the advancement of computer vision techniques in surveillance systems,the need for more proficient,intelligent,and sustainable facial expressions and age recognition is necessary.The main purpose of this study is to develop accurate facial expressions and an age recognition system that is capable of error-free recognition of human expression and age in both indoor and outdoor environments.The proposed system first takes an input image pre-process it and then detects faces in the entire image.After that landmarks localization helps in the formation of synthetic face mask prediction.A novel set of features are extracted and passed to a classifier for the accurate classification of expressions and age group.The proposed system is tested over two benchmark datasets,namely,the Gallagher collection person dataset and the Images of Groups dataset.The system achieved remarkable results over these benchmark datasets about recognition accuracy and computational time.The proposed system would also be applicable in different consumer application domains such as online business negotiations,consumer behavior analysis,E-learning environments,and emotion robotics.

大尺度多视立体深度估计网络S-MVSNet

针对现有估计网络空间复杂度高、难以处理高分辨率影像的问题,提出了一种大尺度多视立体深度估计网络(scalable multi-view stereo network,S-MVSNet)。以R-MVSNet为基础,提出一种轻量级的多尺度特征提取网络MiniUNet来降低空间复杂度;结合多尺度循环神经网络(multi-scale recurrent neural network,MS-RNN)来提升正则化网络的感受野;设计半全局正则化(semi-global regularization,SGR)方法将3D正则化问题转化为两个方向上的2D序列正则化问题,以提升深度估计的精度。在DTU、ETH3D及Tanks and Temples数据集上进行测试,测试结果表明,S-MVSNet能在8 GB显存下处理2K影像,支持的最大输入分辨率是RMVSNet的2.25倍,平均精度较R-MVSNet提升11%。

Multi-View Feature Fusion Model for Software Bug Repair Pattern Prediction

Many search-based Automatic Program Repair(APR)techniques employ a set of repair patterns to generate candidate patches.Regarding repair pattern selection,existing search-based APR techniques either randomly select a repair pattern from the repair pattern set to apply or prioritize all repair patterns based on the bug's context information.In this paper,we introduce PatternNet,a multi-view feature f usion model capable of predicting the repair pattern for a reported software bug.To accomplish this task,PatternNet first extracts multiview features from the pair of buggy code and bug report using different models.Specifically,a transformer-based model(i.e.,UniXcoder)is utilized to obtain the bimodal feature representation of the buggy code and bug report.Additionally,an Abstract Syntax Tree(AST)-based neural model(i.e.,ASTNN)is employed to learn the feature representation of the buggy code.Second,a co-attention mechanism is adopted to capture the dependencies between the statement trees in the AST of the buggy code and the textual tokens of the reported bug,resulting in co-attentive features between statement trees and reported bug's textual tokens.Finally,these multi-view features are combined i nto a unified representation using a feature fusion network.We quantitatively demonstrate the effectiveness of PatternNet and the feature fusion network for predicting software bug repair patterns.

Online Sequential Extreme Multilayer Perception with Time Series Learning Machine Based Output Self Feedback for Prediction

This study presents a time series prediction model with output self feedback which is implemented based on online sequential extreme learning machine. The output variables derived from multilayer perception can feedback to the network input layer to create a temporal relation between the current node inputs and the lagged node outputs while overcoming the limitation of memory which is a vital port for any time-series prediction application. The model can overcome the static prediction problem with most time series prediction models and can effectively cope with the dynamic properties of time series data. A linear and a nonlinear forecasting algorithms based on online extreme learning machine are proposed to implement the output feedback forecasting model. They are both recursive estimator and have two distinct phases： Predict and Update. The proposed model was tested against different kinds of time series data and the results indicate that the model outperforms the original static model without feedback.