Automated Machine Learning Algorithm Using Recurrent Neural Network to Perform Long-Term Time Series Forecasting

Long-term time series forecasting stands as a crucial research domain within the realm of automated machine learning(AutoML).At present,forecasting,whether rooted in machine learning or statistical learning,typically relies on expert input and necessitates substantial manual involvement.This manual effort spans model development,feature engineering,hyper-parameter tuning,and the intricate construction of time series models.The complexity of these tasks renders complete automation unfeasible,as they inherently demand human intervention at multiple junctures.To surmount these challenges,this article proposes leveraging Long Short-Term Memory,which is the variant of Recurrent Neural Networks,harnessing memory cells and gating mechanisms to facilitate long-term time series prediction.However,forecasting accuracy by particular neural network and traditional models can degrade significantly,when addressing long-term time-series tasks.Therefore,our research demonstrates that this innovative approach outperforms the traditional Autoregressive Integrated Moving Average(ARIMA)method in forecasting long-term univariate time series.ARIMA is a high-quality and competitive model in time series prediction,and yet it requires significant preprocessing efforts.Using multiple accuracy metrics,we have evaluated both ARIMA and proposed method on the simulated time-series data and real data in both short and long term.Furthermore,our findings indicate its superiority over alternative network architectures,including Fully Connected Neural Networks,Convolutional Neural Networks,and Nonpooling Convolutional Neural Networks.Our AutoML approach enables non-professional to attain highly accurate and effective time series forecasting,and can be widely applied to various domains,particularly in business and finance.

The Lightweight Edge-Side Fault Diagnosis Approach Based on Spiking Neural Network

Network fault diagnosis methods play a vital role in maintaining network service quality and enhancing user experience as an integral component of intelligent network management.Considering the unique characteristics of edge networks,such as limited resources,complex network faults,and the need for high real-time performance,enhancing and optimizing existing network fault diagnosis methods is necessary.Therefore,this paper proposes the lightweight edge-side fault diagnosis approach based on a spiking neural network(LSNN).Firstly,we use the Izhikevich neurons model to replace the Leaky Integrate and Fire(LIF)neurons model in the LSNN model.Izhikevich neurons inherit the simplicity of LIF neurons but also possess richer behavioral characteristics and flexibility to handle diverse data inputs.Inspired by Fast Spiking Interneurons(FSIs)with a high-frequency firing pattern,we use the parameters of FSIs.Secondly,inspired by the connection mode based on spiking dynamics in the basal ganglia(BG)area of the brain,we propose the pruning approach based on the FSIs of the BG in LSNN to improve computational efficiency and reduce the demand for computing resources and energy consumption.Furthermore,we propose a multiple iterative Dynamic Spike Timing Dependent Plasticity(DSTDP)algorithm to enhance the accuracy of the LSNN model.Experiments on two server fault datasets demonstrate significant precision,recall,and F1 improvements across three diagnosis dimensions.Simultaneously,lightweight indicators such as Params and FLOPs significantly reduced,showcasing the LSNN’s advanced performance and model efficiency.To conclude,experiment results on a pair of datasets indicate that the LSNN model surpasses traditional models and achieves cutting-edge outcomes in network fault diagnosis tasks.

Crack Fault Diagnosis and Location Method for a Dual-Disk Hollow Shaft Rotor System Based on the Radial Basis Function Network and Pattern Recognition Neural Network

The crack fault is one of the most common faults in the rotor system,and researchers have paid close attention to its fault diagnosis.However,most studies focus on discussing the dynamic response characteristics caused by the crack rather than estimating the crack depth and position based on the obtained vibration signals.In this paper,a novel crack fault diagnosis and location method for a dual-disk hollow shaft rotor system based on the Radial basis function(RBF)network and Pattern recognition neural network(PRNN)is presented.Firstly,a rotor system model with a breathing crack suitable for a short-thick hollow shaft rotor is established based on the finite element method,where the crack's periodic opening and closing pattern and different degrees of crack depth are considered.Then,the dynamic response is obtained by the harmonic balance method.By adjusting the crack parameters,the dynamic characteristics related to the crack depth and position are analyzed through the amplitude-frequency responses and waterfall plots.The analysis results show that the first critical speed,first subcritical speed,first critical speed amplitude,and super-harmonic resonance peak at the first subcritical speed can be utilized for the crack fault diagnosis.Based on this,the RBF network and PRNN are adopted to determine the depth and approximate location of the crack respectively by taking the above dynamic characteristics as input.Test results show that the proposed method has high fault diagnosis accuracy.This research proposes a crack detection method adequate for the hollow shaft rotor system,where the crack depth and position are both unknown.

Machine Learning and Artificial Neural Network for Predicting Heart Failure Risk

Heart failure is now widely spread throughout the world.Heart disease affects approximately 48%of the population.It is too expensive and also difficult to cure the disease.This research paper represents machine learning models to predict heart failure.The fundamental concept is to compare the correctness of various Machine Learning(ML)algorithms and boost algorithms to improve models’accuracy for prediction.Some supervised algorithms like K-Nearest Neighbor(KNN),Support Vector Machine(SVM),Decision Trees(DT),Random Forest(RF),Logistic Regression(LR)are considered to achieve the best results.Some boosting algorithms like Extreme Gradient Boosting(XGBoost)and Cat-Boost are also used to improve the prediction using Artificial Neural Networks(ANN).This research also focuses on data visualization to identify patterns,trends,and outliers in a massive data set.Python and Scikit-learns are used for ML.Tensor Flow and Keras,along with Python,are used for ANN model train-ing.The DT and RF algorithms achieved the highest accuracy of 95%among the classifiers.Meanwhile,KNN obtained a second height accuracy of 93.33%.XGBoost had a gratified accuracy of 91.67%,SVM,CATBoost,and ANN had an accuracy of 90%,and LR had 88.33%accuracy.

Machine learning and deep neural network-based learning in osteoarthritis knee

Osteoarthritis(OA)of the knee joint is considered the commonest musculoskeletal condition leading to marked disability for patients residing in various regions around the globe.Application of machine learning(ML)in doing research regarding OA has brought about various clinical advances viz,OA being diagnosed at preliminary stages,prediction of chances of development of OA among the population,discovering various phenotypes of OA,calculating the severity in OA structure and also discovering people with slow and fast progression of disease pathology,etc.Various publications are available regarding machine learning methods for the early detection of osteoarthritis.The key features are detected by morphology,molecular architecture,and electrical and mechanical functions.In addition,this particular technique was utilized to assess non-interfering,non-ionizing,and in-vivo techniques using magnetic resonance imaging.ML is being utilized in OA,chiefly with the formulation of large cohorts viz,the OA Initiative,a cohort observational study,the Multicentre Osteoarthritis Study,an observational,prospective longitudinal study and the Cohort Hip&Cohort Knee,an observational cohort prospective study of both hip and knee OA.Though ML has various contributions and enhancing applications,it remains an imminent field with high potential,also with its limitations.Many more studies are to be carried out to find more about the link between machine learning and knee osteoarthritis,which would help in the improvement of making decisions clinically,and expedite the necessary interventions.

An Interpretable Denoising Layer for Neural Networks Based on Reproducing Kernel Hilbert Space and its Application in Machine Fault Diagnosis

Deep learning algorithms based on neural networks make remarkable achievements in machine fault diagnosis,while the noise mixed in measured signals harms the prediction accuracy of networks.Existing denoising methods in neural networks,such as using complex network architectures and introducing sparse techniques,always suffer from the difficulty of estimating hyperparameters and the lack of physical interpretability.To address this issue,this paper proposes a novel interpretable denoising layer based on reproducing kernel Hilbert space(RKHS)as the first layer for standard neural networks,with the aim to combine the advantages of both traditional signal processing technology with physical interpretation and network modeling strategy with parameter adaption.By investigating the influencing mechanism of parameters on the regularization procedure in RKHS,the key parameter that dynamically controls the signal smoothness with low computational cost is selected as the only trainable parameter of the proposed layer.Besides,the forward and backward propagation algorithms of the designed layer are formulated to ensure that the selected parameter can be automatically updated together with other parameters in the neural network.Moreover,exponential and piecewise functions are introduced in the weight updating process to keep the trainable weight within a reasonable range and avoid the ill-conditioned problem.Experiment studies verify the effectiveness and compatibility of the proposed layer design method in intelligent fault diagnosis of machinery in noisy environments.

Fault Diagnosis of Valve Clearance in Diesel Engine Based on BP Neural Network and Support Vector Machine

Based on wavelet packet transformation(WPT), genetic algorithm(GA), back propagation neural network(BPNN)and support vector machine(SVM), a fault diagnosis method of diesel engine valve clearance is presented. With power spectral density analysis, the characteristic frequency related to the engine running conditions can be extracted from vibration signals. The biggest singular values(BSV)of wavelet coefficients and root mean square(RMS)values of vibration in characteristic frequency sub-bands are extracted at the end of third level decomposition of vibration signals, and they are used as input vectors of BPNN or SVM. To avoid being trapped in local minima, GA is adopted. The normal and fault vibration signals measured in different valve clearance conditions are analyzed. BPNN, GA back propagation neural network(GA-BPNN), SVM and GA-SVM are applied to the training and testing for the extraction of different features, and the classification accuracies and training time are compared to determine the optimum fault classifier and feature selection. Experimental results demonstrate that the proposed features and classification algorithms give classification accuracy of 100%.

FAULT DIAGNOSIS OF ROTATING MACHINERY USING KNOWLEDGE-BASED FUZZY NEURAL NETWORK

A novel knowledge-based fuzzy neural network （KBFNN） for fault diagnosis is presented. Crude rules were extracted and the corresponding dependent factors and antecedent coverage factors were calculated firstly from the diagnostic sample based on rough sets theory. Then the number of rules was used to construct partially the structure of a fuzzy neural network and those factors were implemented as initial weights, with fuzzy output parameters being optimized by genetic algorithm. Such fuzzy neural network was called KBFNN. This KBFNN was utilized to identify typical faults of rotating machinery. Diagnostic results show that it has those merits of shorter training time and higher right diagnostic level compared to general fuzzy neural networks.

Research on Gear-box Fault Diagnosis Method Based on Adjusting-learning-rate PSO Neural Network

Based on the research of Particle Swarm Optimization (PSO) learning rate, two learning rates are changed linearly with velocity-formula evolving in order to adjust the proportion of social part and cognitional part; then the methods are applied to BP neural network training, the convergence rate is heavily accelerated and locally optional solution is avoided. According to actual data of two levels compound-box in vibration lab, signals are analyzed and their characteristic values are abstracted. By applying the trained BP neural networks to compound-box fault diagnosis, it is indicated that the methods are sound effective.

Application of MBAM Neural Network in CNC Machine Fault Diagnosis

In order to improve the bidirectional associative memory(BAM) performance, a modified BAM model(MBAM) is used to enhance neural network(NN)’s memory capacity and error correction capability, theoretical analysis and experiment results illuminate that MBAM performs much better than the original BAM. The MBAM is used in computer numeric control(CNC) machine fault diagnosis, it not only can complete fault diagnosis correctly but also have fairly high error correction capability for disturbed Input Information sequence.Moreover MBAM model is a more convenient and effective method of solving the problem of CNC electric system fault diagnosis.

SIMULATION INVESTIGATION OF AEROENGINE FAULT DIAGNOSIS USING NEURAL NETWORKS

Traditional scheduled maintenance systems are costly, labor intensive, and typically provide noncomprehensive detection and diagnosis of engine faults. The engine monitoring system (EMS) on modern aircrafts has the potential to provide maintenance personnel with valuable information for detecting and diagnosing engine faults. In this paper, an RBF neural network approach is applied to aeroengine gas path fault diagnosis. It can detect multiple faults and quantify the amount of deterioration of the various engine components as a function of measured parameters. The results obtained demonstrate that the accuracy of diagnosis is consistent with practical requirements. The approach takes advantage of the nonlinear mapping feature of neural networks to capture the appropriate characteristics of an aeroengine. The methodology is generic and applicable to other similar plants having high complexity.

Long-Range Dependencies Learning Based on Nonlocal 1D-Convolutional Neural Network for Rolling Bearing Fault Diagnosis

In the field of data-driven bearing fault diagnosis,convolutional neural network(CNN)has been widely researched and applied due to its superior feature extraction and classification ability.However,the convolutional operation could only process a local neighborhood at a time and thus lack the ability of capturing long-range dependencies.Therefore,building an efficient learning method for long-range dependencies is crucial to comprehend and express signal features considering that the vibration signals obtained in a real industrial environment always have strong instability,periodicity,and temporal correlation.This paper introduces nonlocal mean to the CNN and presents a 1D nonlocal block(1D-NLB)to extract long-range dependencies.The 1D-NLB computes the response at a position as a weighted average value of the features at all positions.Based on it,we propose a nonlocal 1D convolutional neural network(NL-1DCNN)aiming at rolling bearing fault diagnosis.Furthermore,the 1D-NLB could be simply plugged into most existing deep learning architecture to improve their fault diagnosis ability.Under multiple noise conditions,the 1D-NLB improves the performance of the CNN on the wheelset bearing data set of high-speed train and the Case Western Reserve University bearing data set.The experiment results show that the NL-1DCNN exhibits superior results compared with six state-of-the-art fault diagnosis methods.

APPLICATION OF ASSOCIATIVE MEMORY NEURAL NETWORK IN HIGH VOLTAGE TRANSMISSIONLINE FAULT DIAGNOSIS

Effective methods of enhancing the fault-tolerance property are proposed for two kinds of associative memory (AM) neural network (NN) used in high voltage transmission line fault diagnosis. For feedforward NN (FNN),the conception of 'fake attaction region' is presented to expand the attraction region artificially,and for the feedback Hopfield bidirectional AM NN (BAM-NN),the measure to add redundant neurons is taken to enhance NN's memory capacity and fault-tolerance property. Study results show that the NNs built not only can complete fault diagnosis correctly but also have fairly high fault-tolerance ability for disturbed input information sequence. Moreover FNN is a more convenient and effective method of solving the problem of power system fault diagnosis.

Research on Rotating Machinery Fault Diagnosis Based on Improved Multi-target Domain Adversarial Network

Aiming at the problems of low efficiency,poor anti-noise and robustness of transfer learning model in intelligent fault diagnosis of rotating machinery,a new method of intelligent fault diagnosis of rotating machinery based on single source and multi-target domain adversarial network model(WDMACN)and Gram Angle Product field(GAPF)was proposed.Firstly,the original one-dimensional vibration signal is preprocessed using GAPF to generate the image data including all time series.Secondly,the residual network is used to extract data features,and the features of the target domain without labels are pseudo-labeled,and the transferable features among the feature extractors are shared through the depth parameter,and the feature extractors of the multi-target domain are updated anatomically to generate the features that the discriminator cannot distinguish.The modelt through adversarial domain adaptation,thus achieving fault classification.Finally,a large number of validations were carried out on the bearing data set of Case Western Reserve University(CWRU)and the gear data.The results show that the proposed method can greatly improve the diagnostic efficiency of the model,and has good noise resistance and generalization.

Label Recovery and Trajectory Designable Network for Transfer Fault Diagnosis of Machines With Incorrect Annotation

The success of deep transfer learning in fault diagnosis is attributed to the collection of high-quality labeled data from the source domain.However,in engineering scenarios,achieving such high-quality label annotation is difficult and expensive.The incorrect label annotation produces two negative effects:1)the complex decision boundary of diagnosis models lowers the generalization performance on the target domain,and2)the distribution of target domain samples becomes misaligned with the false-labeled samples.To overcome these negative effects,this article proposes a solution called the label recovery and trajectory designable network(LRTDN).LRTDN consists of three parts.First,a residual network with dual classifiers is to learn features from cross-domain samples.Second,an annotation check module is constructed to generate a label anomaly indicator that could modify the abnormal labels of false-labeled samples in the source domain.With the training of relabeled samples,the complexity of diagnosis model is reduced via semi-supervised learning.Third,the adaptation trajectories are designed for sample distributions across domains.This ensures that the target domain samples are only adapted with the pure-labeled samples.The LRTDN is verified by two case studies,in which the diagnosis knowledge of bearings is transferred across different working conditions as well as different yet related machines.The results show that LRTDN offers a high diagnosis accuracy even in the presence of incorrect annotation.

Application of neural network in heating network leakage fault diagnosis

In order to investigate the leak detection strategy of a heating network,a space-based simulation mathematical model for the heating network under leakage conditions is built by graph theory.The pressure changes of all the nodes in the heating network are obtained from node leak and pipe leak conditions.Then,a leakage diagnosis system based on the back propagation（BP）neural network is established.This diagnosis system can predict the leakage pipe by collecting the pressure change data of the monitoring points,which can preliminary estimate the leak location.The usefulness of this system is proved by an example.The experimental results show that the forecast accuracy by this diagnosis system can reach 100%.

An Intelligent Process Fault Diagnosis System based on Andrews Plot and Convolutional Neural Network

This paper proposes an intelligent process fault diagnosis system through integrating the techniques of Andrews plot and convolutional neural network.The proposed fault diagnosis method extracts features from the on-line process measurements using Andrews function.To address the uncertainty of setting the proper dimension of extracted features in Andrews function,a convolutional neural network is used to further extract diagnostic information from the Andrews function outputs.The outputs of the convolutional neural network are then fed to a single hidden layer neural network to obtain the final fault diagnosis result.The proposed fault diagnosis system is compared with a conventional neural network based fault diagnosis system and integrating Andrews function with neural network and manual selection of features in Andrews function outputs.Applications to a simulated CSTR process show that the proposed fault diagnosis system gives much better performance than the conventional neural network based fault diagnosis system and manual selection of features in Andrews function outputs.It reveals that the use of Andrews function and convolutional neural network can improve the diagnosis performance.

Bearings Intelligent Fault Diagnosis by 1-D Adder Neural Networks

Integrated with sensors,processors,and radio frequency(RF)communication modules,intelligent bearing could achieve the autonomous perception and autonomous decision-making,guarantying the safety and reliability during their use.However,because of the resource limitations of the end device,processors in the intelligent bearing are unable to carry the computational load of deep learning models like convolutional neural network(CNN),which involves a great amount of multiplicative operations.To minimize the computation cost of the conventional CNN,based on the idea of AdderNet,a 1-D adder neural network with a wide first-layer kernel(WAddNN)suitable for bearing fault diagnosis is proposed in this paper.The proposed method uses the l1-norm distance between filters and input features as the output response,thus making the whole network almost free of multiplicative operations.The whole model takes the original signal as the input,uses a wide kernel in the first adder layer to extract features and suppress the high frequency noise,and then uses two layers of small kernels for nonlinear mapping.Through experimental comparison with CNN models of the same structure,WAddNN is able to achieve a similar accuracy as CNN models with significantly reduced computational cost.The proposed model provides a new fault diagnosis method for intelligent bearings with limited resources.

Feature evaluation and extraction based on neural network in analog circuit fault diagnosis

Choosing the right characteristic parameter is the key to fault diagnosis in analog circuit. The feature evaluation and extraction methods based on neural network are presented. Parameter evaluation of circuit features is realized by training results from neural network; the superior nonlinear mapping capability is competent for extracting fault features which are normalized and compressed subsequently. The complex classification problem on fault pattern recognition in analog circuit is transferred into feature processing stage by feature extraction based on neural network effectively, which improves the diagnosis efficiency. A fault diagnosis illustration validated this method.

Sensor Fault Diagnosis and Reconstruction of Engine Control System Based on Autoassociative Neural Network

The topology and property of Autoassociative Neural Networks(AANN) and theAANN's application to sensor fault diagnosis and reconstruction of engine control system arestudied. The key feature of AANN is feature extract and noise filtering. Sensor fault detection isaccomplished by integrating the optimal estimation and fault detection logic. Digital simulationshows that the scheme can detect hard and soft failures of sensors at the absence of models forengines which have performance deteriorate in the service life, and can provide good analyticalredundancy.