Effects of data smoothing and recurrent neural network(RNN)algorithms for real-time forecasting of tunnel boring machine(TBM)performance

Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.

Hyperparameter Tuning for Deep Neural Networks Based Optimization Algorithm

For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.

Remaining Useful Life Prediction for a Roller in a Hot Strip Mill Based on Deep Recurrent Neural Networks

Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.

New Stability Criteria for Recurrent Neural Networks with a Time-varying Delay

This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.

Global stability of interval recurrent neural networks

The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robust stability of interval RNNs is transformed into a problem of solving a class of linear matrix inequalities.Thus,the robust stability of interval RNNs can be analyzed by directly using the linear matrix inequalities（LMI） toolbox of MATLAB.Numerical example is given to show the effectiveness of the obtained results.

A Prediction Method of Trend-Type Capacity Index Based on Recurrent Neural Network

Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes it very difficult to perform high-precision capacity prediction.In order to improve the forecasting efficiency of related indexes,this paper designs a classification method of capacity index data,which divides the capacity index data into trend type,periodic type and irregular type.Then for the prediction of trend data,it proposes a capacity index prediction model based on Recurrent Neural Network(RNN),denoted as RNN-LSTM-LSTM.This model includes a basic RNN,two Long Short-Term Memory(LSTM)networks and two Fully Connected layers.The experimental results show that,compared with the traditional Holt-Winters,Autoregressive Integrated Moving Average(ARIMA)and Back Propagation(BP)neural network prediction model,the mean square error(MSE)of the proposed RNN-LSTM-LSTM model are reduced by 11.82%and 20.34%on the order storage and data migration,which has greatly improved the efficiency of trend-type capacity index prediction.

Deep-fake video detection approaches using convolutional–recurrent neural networks

Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural networks employed for Deep-Fake video detection.In the model,the features from the training data are extracted with the intended Convolution Neural Network model to form feature vectors which are further analysed using a dense layer,a Long Short-Term Memoryand Gated Recurrent by adopting transfer learning with fine tuning for training the models.The model is evaluated to detect Artificial Intelligence based Deep fakes images and videos using benchmark datasets.Comparative analysis shows that the detections are majorly biased towards domain of the dataset but there is a noteworthy improvement in the model performance parameters by using Transfer Learning whereas Convolutional-Recurrent Neural Network has benefits in sequence detection.

Fake News Classification: Past, Current, and Future

The proliferation of deluding data such as fake news and phony audits on news web journals,online publications,and internet business apps has been aided by the availability of the web,cell phones,and social media.Individuals can quickly fabricate comments and news on social media.The most difficult challenge is determining which news is real or fake.Accordingly,tracking down programmed techniques to recognize fake news online is imperative.With an emphasis on false news,this study presents the evolution of artificial intelligence techniques for detecting spurious social media content.This study shows past,current,and possible methods that can be used in the future for fake news classification.Two different publicly available datasets containing political news are utilized for performing experiments.Sixteen supervised learning algorithms are used,and their results show that conventional Machine Learning(ML)algorithms that were used in the past perform better on shorter text classification.In contrast,the currently used Recurrent Neural Network(RNN)and transformer-based algorithms perform better on longer text.Additionally,a brief comparison of all these techniques is provided,and it concluded that transformers have the potential to revolutionize Natural Language Processing(NLP)methods in the near future.

Continuous Mobile User Authentication Using a Hybrid CNN-Bi-LSTM Approach

Internet of Things (IoT) devices incorporate a large amount ofdata in several fields, including those of medicine, business, and engineering.User authentication is paramount in the IoT era to assure connecteddevices’ security. However, traditional authentication methods and conventionalbiometrics-based authentication approaches such as face recognition,fingerprints, and password are vulnerable to various attacks, including smudgeattacks, heat attacks, and shoulder surfing attacks. Behavioral biometrics isintroduced by the powerful sensing capabilities of IoT devices such as smartwearables and smartphones, enabling continuous authentication. ArtificialIntelligence (AI)-based approaches introduce a bright future in refining largeamounts of homogeneous biometric data to provide innovative user authenticationsolutions. This paper presents a new continuous passive authenticationapproach capable of learning the signatures of IoT users utilizing smartphonesensors such as a gyroscope, magnetometer, and accelerometer to recognizeusers by their physical activities. This approach integrates the convolutionalneural network (CNN) and recurrent neural network (RNN) models to learnsignatures of human activities from different users. A series of experiments areconducted using the MotionSense dataset to validate the effectiveness of theproposed method. Our technique offers a competitive verification accuracyequal to 98.4%.We compared the proposed method with several conventionalmachine learning and CNN models and found that our proposed modelachieves higher identification accuracy than the recently developed verificationsystems. The high accuracy achieved by the proposed method proves itseffectiveness in recognizing IoT users passively through their physical activitypatterns.

Treatment of Imbalance Dataset for Human Emotion Classification

Developments in biomedical science, signal processing technologies have led Electroencephalography (EEG) signals to be widely used in the diagnosis of brain disease and in the field of Brain-Computer Interface (BCI). The collected EEG signals are processed using Machine Learning-Random Forest and Naive Bayes- and Deep Learning-Recurrent Neural Network (RNN), Neural Network (NN) and Long Short Term Memory (LSTM)-Algorithms to obtain the recent mood of a person. The Algorithms mentioned above have been imposed on the data set in order to find out what the person is feeling at a particular moment. The following thesis is conducted to find out one of the following moods (happy, surprised, disgust, fear, anger and sadness) of a person at an instant, with an aim to obtain the result with least amount of time delay as the mood differs. It is pretty obvious that the accuracy of the output varies depending upon the algorithm used, time taken to process the data, so that it is easy for us to compare the reliability and dependency of a particular algorithm to another, prior to its practical implementation. The imbalance data sets that were used had an imbalanced class and thus, over fitting occurred. This problem was handled by generating Artificial Data sets with the use of SMOTE Oversampling Technique.

Recurrent Neural Network for Nonconvex Economic Emission Dispatch

In this paper, an economic emission dispatch(EED) model is developed to reduce fuel cost and environmental pollution emissions. Considering the development of new energy sources in recent years, the EED problem involves thermal units with the valve point effect and WTs. Meanwhile, it complies with demand constraint and generator capacity constraints. A recurrent neural network(RNN) is proposed to search for local optimal solution of the introduced nonconvex EED problem. The optimality and convergence of the proposed dynamic model are given. The RNN algorithm is verified on a power generation system for the optimization of scheduling and minimization of total cost. Moreover, a particle swarm optimization(PSO) algorithm is compared with RNN under the same problematic frame. Numerical simulation results demonstrate that the optimal scheduling given by RNN is more precise and has lower total cost than PSO. In addition, the dynamic variation of power load demand is considered and the power distribution of eight generators during 12 time periods is depicted.

Tunnel boring machine vibration-based deep learning for the ground identification of working faces

Tunnel boring machine(TBM) vibration induced by cutting complex ground contains essential information that can help engineers evaluate the interaction between a cutterhead and the ground itself.In this study,deep recurrent neural networks(RNNs) and convolutional neural networks(CNNs) were used for vibration-based working face ground identification.First,field monitoring was conducted to obtain the TBM vibration data when tunneling in changing geological conditions,including mixed-face,homogeneous,and transmission ground.Next,RNNs and CNNs were utilized to develop vibration-based prediction models,which were then validated using the testing dataset.The accuracy of the long short-term memory(LSTM) and bidirectional LSTM(Bi-LSTM) models was approximately 70% with raw data;however,with instantaneous frequency transmission,the accuracy increased to approximately 80%.Two types of deep CNNs,GoogLeNet and ResNet,were trained and tested with time-frequency scalar diagrams from continuous wavelet transformation.The CNN models,with an accuracy greater than 96%,performed significantly better than the RNN models.The ResNet-18,with an accuracy of 98.28%,performed the best.When the sample length was set as the cutterhead rotation period,the deep CNN and RNN models achieved the highest accuracy while the proposed deep CNN model simultaneously achieved high prediction accuracy and feedback efficiency.The proposed model could promptly identify the ground conditions at the working face without stopping the normal tunneling process,and the TBM working parameters could be adjusted and optimized in a timely manner based on the predicted results.

Automated Facial Expression Recognition and Age Estimation Using Deep Learning

With the advancement of computer vision techniques in surveillance systems,the need for more proficient,intelligent,and sustainable facial expressions and age recognition is necessary.The main purpose of this study is to develop accurate facial expressions and an age recognition system that is capable of error-free recognition of human expression and age in both indoor and outdoor environments.The proposed system first takes an input image pre-process it and then detects faces in the entire image.After that landmarks localization helps in the formation of synthetic face mask prediction.A novel set of features are extracted and passed to a classifier for the accurate classification of expressions and age group.The proposed system is tested over two benchmark datasets,namely,the Gallagher collection person dataset and the Images of Groups dataset.The system achieved remarkable results over these benchmark datasets about recognition accuracy and computational time.The proposed system would also be applicable in different consumer application domains such as online business negotiations,consumer behavior analysis,E-learning environments,and emotion robotics.

大尺度多视立体深度估计网络S-MVSNet

针对现有估计网络空间复杂度高、难以处理高分辨率影像的问题,提出了一种大尺度多视立体深度估计网络(scalable multi-view stereo network,S-MVSNet)。以R-MVSNet为基础,提出一种轻量级的多尺度特征提取网络MiniUNet来降低空间复杂度;结合多尺度循环神经网络(multi-scale recurrent neural network,MS-RNN)来提升正则化网络的感受野;设计半全局正则化(semi-global regularization,SGR)方法将3D正则化问题转化为两个方向上的2D序列正则化问题,以提升深度估计的精度。在DTU、ETH3D及Tanks and Temples数据集上进行测试,测试结果表明,S-MVSNet能在8 GB显存下处理2K影像,支持的最大输入分辨率是RMVSNet的2.25倍,平均精度较R-MVSNet提升11%。

Multi-View Feature Fusion Model for Software Bug Repair Pattern Prediction

Many search-based Automatic Program Repair(APR)techniques employ a set of repair patterns to generate candidate patches.Regarding repair pattern selection,existing search-based APR techniques either randomly select a repair pattern from the repair pattern set to apply or prioritize all repair patterns based on the bug's context information.In this paper,we introduce PatternNet,a multi-view feature f usion model capable of predicting the repair pattern for a reported software bug.To accomplish this task,PatternNet first extracts multiview features from the pair of buggy code and bug report using different models.Specifically,a transformer-based model(i.e.,UniXcoder)is utilized to obtain the bimodal feature representation of the buggy code and bug report.Additionally,an Abstract Syntax Tree(AST)-based neural model(i.e.,ASTNN)is employed to learn the feature representation of the buggy code.Second,a co-attention mechanism is adopted to capture the dependencies between the statement trees in the AST of the buggy code and the textual tokens of the reported bug,resulting in co-attentive features between statement trees and reported bug's textual tokens.Finally,these multi-view features are combined i nto a unified representation using a feature fusion network.We quantitatively demonstrate the effectiveness of PatternNet and the feature fusion network for predicting software bug repair patterns.

A Regularized LSTM Method for Predicting Remaining Useful Life of Rolling Bearings

Rotating machinery is important to industrial production. Any failure of rotating machinery, especially the failure of rolling bearings, can lead to equipment shutdown and even more serious incidents. Therefore, accurate residual life prediction plays a crucial role in guaranteeing machine operation safety and reliability and reducing maintenance cost. In order to increase the forecasting precision of the remaining useful life(RUL) of the rolling bearing, an advanced approach combining elastic net with long short-time memory network(LSTM) is proposed, and the new approach is referred to as E-LSTM. The E-LSTM algorithm consists of an elastic mesh and LSTM, taking temporal-spatial correlation into consideration to forecast the RUL through the LSTM. To solve the over-fitting problem of the LSTM neural network during the training process, the elastic net based regularization term is introduced to the LSTM structure.In this way, the change of the output can be well characterized to express the bearing degradation mode. Experimental results from the real-world data demonstrate that the proposed E-LSTM method can obtain higher stability and relevant values that are useful for the RUL forecasting of bearing. Furthermore, these results also indicate that E-LSTM can achieve better performance.

Precision measurement and compensation of kinematic errors for industrial robots using artifact and machine learning

Industrial robots are widely used in various areas owing to their greater degrees of freedom(DOFs)and larger operation space compared with traditional frame movement systems involving sliding and rotational stages.However,the geometrical transfer of joint kinematic errors and the relatively weak rigidity of industrial robots compared with frame movement systems decrease their absolute kinematic accuracy,thereby limiting their further application in ultraprecision manufacturing.This imposes a stringent requirement for improving the absolute kinematic accuracy of industrial robots in terms of the position and orientation of the robot arm end.Current measurement and compensation methods for industrial robots either require expensive measuring systems,producing positioning or orientation errors,or offer low measurement accuracy.Herein,a kinematic calibration method for an industrial robot using an artifact with a hybrid spherical and ellipsoid surface is proposed.A system with submicrometric precision for measuring the position and orientation of the robot arm end is developed using laser displacement sensors.Subsequently,a novel kinematic error compensating method involving both a residual learning algorithm and a neural network is proposed to compensate for nonlinear errors.A six-layer recurrent neural network(RNN)is designed to compensate for the kinematic nonlinear errors of a six-DOF industrial robot.The results validate the feasibility of the proposed method for measuring the kinematic errors of industrial robots,and the compensation method based on the RNN improves the accuracy via parameter fitting.Experimental studies show that the measuring system and compensation method can reduce motion errors by more than 30%.The present study provides a feasible and economic approach for measuring and improving the motion accuracy of an industrial robot at the submicrometric measurement level.

The efficacy of deep learning based LSTM model in forecasting the outbreak of contagious diseases

The coronavirus disease that outbreak in 2019 has caused various health issues.According to the WHO,the first positive case was detected in Bangladesh on 7th March 2020,but while writing this paper in June 2021,the total confirmed,recovered,and death cases were 826922,766266 and 13118,respectively.Due to the emergence of COVID-19 in Bangladesh,the country is facing a major public health crisis.Unfortunately,the country does not have a comprehensive health policy to address this issue.This makes it hard to predict how the pandemic will affect the population.Machine learning techniques can help us detect the disease's spread.To predict the trend,parameters,risks,and to take preventive measure in Bangladesh;this work utilized the Recurrent Neural Networks based Deep Learning methodologies like LongShort-Term Memory.Here,we aim to predict the epidemic's progression for a period of more than a year under various scenarios in Bangladesh.We extracted the data for daily confirmed,recovered,and death cases from March 2020 to August 2021.The obtained Root Mean Square Error(RMSE)values of confirmed,recovered,and death cases indicates that our result is more accurate than other contemporary techniques.This study indicates that the LSTM model could be used effectively in predicting contagious diseases.The obtained results could help in explaining the seriousness of the situation,also mayhelp the authorities to take precautionary steps to control the situation.

Non-Line-of-Sight Multipath Detection Method for BDS/GPS Fusion System Based on Deep Learning

Non-line-of-sight(NLOS)multipath effect is the main factor that restricts the application of global navigation satellite system(GNSS)in complex environments,especially in urban canyon.The effective avoidance of NLOS signals can significantly improve the positioning performance of GNSS receiver.In this paper,an NLOS/LOS classification model based on recurrent neural network is proposed to classify satellite signals received in urban canyon environments.The accuracy of classification is 91%,and the recognition rate of NLOS is 89%;the classification performance is better than that of traditional machine learning classification models such as support vector machine.For BeiDou navigation satellite system/global positioning system(BDS/GPS)fusion system,the least square algorithm and extended Kalman filter are used to estimate the position.The experimental results show that the three-dimensional positioning accuracy after NLOS recognition is improved about 60%on average compared with the traditional methods,and the positioning stability is also improved significantly.