Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

Hybrid model for BOF oxygen blowing time prediction based on oxygen balance mechanism and deep neural network

The amount of oxygen blown into the converter is one of the key parameters for the control of the converter blowing process,which directly affects the tap-to-tap time of converter. In this study, a hybrid model based on oxygen balance mechanism (OBM) and deep neural network (DNN) was established for predicting oxygen blowing time in converter. A three-step method was utilized in the hybrid model. First, the oxygen consumption volume was predicted by the OBM model and DNN model, respectively. Second, a more accurate oxygen consumption volume was obtained by integrating the OBM model and DNN model. Finally, the converter oxygen blowing time was calculated according to the oxygen consumption volume and the oxygen supply intensity of each heat. The proposed hybrid model was verified using the actual data collected from an integrated steel plant in China, and compared with multiple linear regression model, OBM model, and neural network model including extreme learning machine, back propagation neural network, and DNN. The test results indicate that the hybrid model with a network structure of 3 hidden layer layers, 32-16-8 neurons per hidden layer, and 0.1 learning rate has the best prediction accuracy and stronger generalization ability compared with other models. The predicted hit ratio of oxygen consumption volume within the error±300 m^(3)is 96.67%;determination coefficient (R^(2)) and root mean square error (RMSE) are0.6984 and 150.03 m^(3), respectively. The oxygen blow time prediction hit ratio within the error±0.6 min is 89.50%;R2and RMSE are0.9486 and 0.3592 min, respectively. As a result, the proposed model can effectively predict the oxygen consumption volume and oxygen blowing time in the converter.

Design of a novel hybrid quantum deep neural network in INEQR images classification

We redesign the parameterized quantum circuit in the quantum deep neural network, construct a three-layer structure as the hidden layer, and then use classical optimization algorithms to train the parameterized quantum circuit, thereby propose a novel hybrid quantum deep neural network(HQDNN) used for image classification. After bilinear interpolation reduces the original image to a suitable size, an improved novel enhanced quantum representation(INEQR) is used to encode it into quantum states as the input of the HQDNN. Multi-layer parameterized quantum circuits are used as the main structure to implement feature extraction and classification. The output results of parameterized quantum circuits are converted into classical data through quantum measurements and then optimized on a classical computer. To verify the performance of the HQDNN, we conduct binary classification and three classification experiments on the MNIST(Modified National Institute of Standards and Technology) data set. In the first binary classification, the accuracy of 0 and 4 exceeds98%. Then we compare the performance of three classification with other algorithms, the results on two datasets show that the classification accuracy is higher than that of quantum deep neural network and general quantum convolutional neural network.

Analysis of learnability of a novel hybrid quantum-classical convolutional neural network in image classification

We design a new hybrid quantum-classical convolutional neural network(HQCCNN)model based on parameter quantum circuits.In this model,we use parameterized quantum circuits(PQCs)to redesign the convolutional layer in classical convolutional neural networks,forming a new quantum convolutional layer to achieve unitary transformation of quantum states,enabling the model to more accurately extract hidden information from images.At the same time,we combine the classical fully connected layer with PQCs to form a new hybrid quantum-classical fully connected layer to further improve the accuracy of classification.Finally,we use the MNIST dataset to test the potential of the HQCCNN.The results indicate that the HQCCNN has good performance in solving classification problems.In binary classification tasks,the classification accuracy of numbers 5 and 7 is as high as 99.71%.In multivariate classification,the accuracy rate also reaches 98.51%.Finally,we compare the performance of the HQCCNN with other models and find that the HQCCNN has better classification performance and convergence speed.

HQNN-SFOP:Hybrid Quantum Neural Networks with Signal Feature Overlay Projection for Drone Detection Using Radar Return Signals-A Simulation

With the wide application of drone technology,there is an increasing demand for the detection of radar return signals from drones.Existing detection methods mainly rely on time-frequency domain feature extraction and classical machine learning algorithms for image recognition.This method suffers from the problem of large dimensionality of image features,which leads to large input data size and noise affecting learning.Therefore,this paper proposes to extract signal time-domain statistical features for radar return signals from drones and reduce the feature dimension from 512×4 to 16 dimensions.However,the downscaled feature data makes the accuracy of traditional machine learning algorithms decrease,so we propose a new hybrid quantum neural network with signal feature overlay projection(HQNN-SFOP),which reduces the dimensionality of the signal by extracting the statistical features in the time domain of the signal,introduces the signal feature overlay projection to enhance the expression ability of quantum computation on the signal features,and introduces the quantum circuits to improve the neural network’s ability to obtain the inline relationship of features,thus improving the accuracy and migration generalization ability of drone detection.In order to validate the effectiveness of the proposed method,we experimented with the method using the MM model that combines the real parameters of five commercial drones and random drones parameters to generate data to simulate a realistic environment.The results show that the method based on statistical features in the time domain of the signal is able to extract features at smaller scales and obtain higher accuracy on a dataset with an SNR of 10 dB.On the time-domain feature data set,HQNNSFOP obtains the highest accuracy compared to other conventional methods.In addition,HQNN-SFOP has good migration generalization ability on five commercial drones and random drones data at different SNR conditions.Our method verifies the feasibility and effectiveness of signal detection methods based on quantum computation and experimentally demonstrates that the advantages of quantum computation for information processing are still valid in the field of signal processing,it provides a highly efficient method for the drone detection using radar return signals.

Simultaneous Identification of Thermophysical Properties of Semitransparent Media Using a Hybrid Model Based on Artificial Neural Network and Evolutionary Algorithm

A hybrid identification model based on multilayer artificial neural networks(ANNs) and particle swarm optimization(PSO) algorithm is developed to improve the simultaneous identification efficiency of thermal conductivity and effective absorption coefficient of semitransparent materials.For the direct model,the spherical harmonic method and the finite volume method are used to solve the coupled conduction-radiation heat transfer problem in an absorbing,emitting,and non-scattering 2D axisymmetric gray medium in the background of laser flash method.For the identification part,firstly,the temperature field and the incident radiation field in different positions are chosen as observables.Then,a traditional identification model based on PSO algorithm is established.Finally,multilayer ANNs are built to fit and replace the direct model in the traditional identification model to speed up the identification process.The results show that compared with the traditional identification model,the time cost of the hybrid identification model is reduced by about 1 000 times.Besides,the hybrid identification model remains a high level of accuracy even with measurement errors.

Deep hybrid: Multi-graph neural network collaboration for hyperspectral image classification

With limited number of labeled samples,hyperspectral image(HSI)classification is a difficult Problem in current research.The graph neural network(GNN)has emerged as an approach to semi-supervised classification,and the application of GNN to hyperspectral images has attracted much attention.However,in the existing GNN-based methods a single graph neural network or graph filter is mainly used to extract HSI features,which does not take full advantage of various graph neural networks(graph filters).Moreover,the traditional GNNs have the problem of oversmoothing.To alleviate these shortcomings,we introduce a deep hybrid multi-graph neural network(DHMG),where two different graph filters,i.e.,the spectral filter and the autoregressive moving average(ARMA)filter,are utilized in two branches.The former can well extract the spectral features of the nodes,and the latter has a good suppression effect on graph noise.The network realizes information interaction between the two branches and takes good advantage of different graph filters.In addition,to address the problem of oversmoothing,a dense network is proposed,where the local graph features are preserved.The dense structure satisfies the needs of different classification targets presenting different features.Finally,we introduce a GraphSAGEbased network to refine the graph features produced by the deep hybrid network.Extensive experiments on three public HSI datasets strongly demonstrate that the DHMG dramatically outperforms the state-ofthe-art models.

Solar Radiation Estimation Based on a New Combined Approach of Artificial Neural Networks (ANN) and Genetic Algorithms (GA) in South Algeria

When designing solar systems and assessing the effectiveness of their many uses,estimating sun irradiance is a crucial first step.This study examined three approaches(ANN,GA-ANN,and ANFIS)for estimating daily global solar radiation(GSR)in the south of Algeria:Adrar,Ouargla,and Bechar.The proposed hybrid GA-ANN model,based on genetic algorithm-based optimization,was developed to improve the ANN model.The GA-ANN and ANFIS models performed better than the standalone ANN-based model,with GA-ANN being better suited for forecasting in all sites,and it performed the best with the best values in the testing phase of Coefficient of Determination(R=0.9005),Mean Absolute Percentage Error(MAPE=8.40%),and Relative Root Mean Square Error(rRMSE=12.56%).Nevertheless,the ANFIS model outperformed the GA-ANN model in forecasting daily GSR,with the best values of indicators when testing the model being R=0.9374,MAPE=7.78%,and rRMSE=10.54%.Generally,we may conclude that the initial ANN stand-alone model performance when forecasting solar radiation has been improved,and the results obtained after injecting the genetic algorithm into the ANN to optimize its weights were satisfactory.The model can be used to forecast daily GSR in dry climates and other climates and may also be helpful in selecting solar energy system installations and sizes.

Bottom hole pressure prediction based on hybrid neural networks and Bayesian optimization

Many scholars have focused on applying machine learning models in bottom hole pressure (BHP) prediction. However, the complex and uncertain conditions in deep wells make it difficult to capture spatial and temporal correlations of measurement while drilling (MWD) data with traditional intelligent models. In this work, we develop a novel hybrid neural network, which integrates the Convolution Neural Network (CNN) and the Gate Recurrent Unit (GRU) for predicting BHP fluctuations more accurately. The CNN structure is used to analyze spatial local dependency patterns and the GRU structure is used to discover depth variation trends of MWD data. To further improve the prediction accuracy, we explore two types of GRU-based structure: skip-GRU and attention-GRU, which can capture more long-term potential periodic correlation in drilling data. Then, the different model structures tuned by the Bayesian optimization (BO) algorithm are compared and analyzed. Results indicate that the hybrid models can extract spatial-temporal information of data effectively and predict more accurately than random forests, extreme gradient boosting, back propagation neural network, CNN and GRU. The CNN-attention-GRU model with BO algorithm shows great superiority in prediction accuracy and robustness due to the hybrid network structure and attention mechanism, having the lowest mean absolute percentage error of 0.025%. This study provides a reference for solving the problem of extracting spatial and temporal characteristics and guidance for managed pressure drilling in complex formations.

LaNets:Hybrid Lagrange Neural Networks for Solving Partial Differential Equations

We propose new hybrid Lagrange neural networks called LaNets to predict the numerical solutions of partial differential equations.That is,we embed Lagrange interpolation and small sample learning into deep neural network frameworks.Concretely,we first perform Lagrange interpolation in front of the deep feedforward neural network.The Lagrange basis function has a neat structure and a strong expression ability,which is suitable to be a preprocessing tool for pre-fitting and feature extraction.Second,we introduce small sample learning into training,which is beneficial to guide themodel to be corrected quickly.Taking advantages of the theoretical support of traditional numerical method and the efficient allocation of modern machine learning,LaNets achieve higher predictive accuracy compared to the state-of-the-artwork.The stability and accuracy of the proposed algorithmare demonstrated through a series of classical numerical examples,including one-dimensional Burgers equation,onedimensional carburizing diffusion equations,two-dimensional Helmholtz equation and two-dimensional Burgers equation.Experimental results validate the robustness,effectiveness and flexibility of the proposed algorithm.

Lightweight and highly robust memristor-based hybrid neural networks for electroencephalogram signal processing

Memristor-based neuromorphic computing shows great potential for high-speed and high-throughput signal processing applications,such as electroencephalogram(EEG)signal processing.Nonetheless,the size of one-transistor one-resistor(1T1R)memristor arrays is limited by the non-ideality of the devices,which prevents the hardware implementation of large and complex networks.In this work,we propose the depthwise separable convolution and bidirectional gate recurrent unit(DSC-BiGRU)network,a lightweight and highly robust hybrid neural network based on 1T1R arrays that enables efficient processing of EEG signals in the temporal,frequency and spatial domains by hybridizing DSC and BiGRU blocks.The network size is reduced and the network robustness is improved while ensuring the network classification accuracy.In the simulation,the measured non-idealities of the 1T1R array are brought into the network through statistical analysis.Compared with traditional convolutional networks,the network parameters are reduced by 95%and the network classification accuracy is improved by 21%at a 95%array yield rate and 5%tolerable error.This work demonstrates that lightweight and highly robust networks based on memristor arrays hold great promise for applications that rely on low consumption and high efficiency.

Improved results on synchronization in arrays of coupled delayed neural networks with hybrid coupling

In order to investigate the influence of hybrid coupling on the synchronization of delayed neural networks, by choosing an improved delay-dependent Lyapunov-Krasovskii functional, one less conservative asymptotical criterion based on linear matrix inequality （LMI） is established. The Kronecker product and convex combination techniques are employed. Also the bounds of time-varying delays and delay derivatives are fully considered. By adjusting the inner coupling matrix parameters and using the Matlab LMI toolbox, the design and applications of addressed coupled networks can be realized. Finally, the efficiency and applicability of the proposed results are illustrated by a numerical example with simulations.

Dynamics Modeling and Robust Trajectory Tracking Control for a Class of Hybrid Humanoid Arm Based on Neural Network

In order to solve the problem of trajectory tracking for a class of novel serial-parallel hybrid humanoid arm（HHA）, which has parameters uncertainty, frictions, disturbance, abrasion and pulse forces derived from motors, a multistep dynamics modeling strategy is proposed and a robust controller based on neural network（NN）-adaptive algorithm is designed. At the first step of dynamics modeling, the dynamics model of the reduced HHA is established by Lagrange method. At the second step of dynamics modeling, the parameter uncertain part resulting mainly from the idealization of the HHA is learned by adaptive algorithm. In the trajectory tracking controller, the radial basis function（RBF） NN, whose optimal weights are learned online by adaptive algorithm, is used to learn the upper limit function of the total uncertainties including frictions, disturbances, abrasion and pulse forces. To a great extent, the conservatism of this robust trajectory tracking controller is reduced, and by this controller the HHA can impersonate mostly human actions. The proof and simulation results testify the validity of the adaptive strategy for parameter learning and the neural network-adaptive strategy for the trajectory tracking control.

Hybrid Neural Network Model for RH Vacuum Refining Process Control

A hybrid neural network model,in which RH process(theoretical)model is combined organically with neural network(NN)and case-base reasoning(CBR),was established.The CBR method was used to select the operation mode and the RH operational guide parameters for different steel grades according to the initial conditions of molten steel,and a three-layer BP neural network was adopted to deal with nonlinear factors for improving and compensating the limitations of technological model for RH process control and end-point prediction.It was verified that the hybrid neural network is effective for improving the precision and calculation efficiency of the model.

基于CNN-BiLSTM混合神经网络的雷达信号调制方式识别

针对具有时频特性的雷达信号,传统的雷达信号识别方法已经无法满足对信号类型精准识别的需求,因此需要通过采集并分析雷达信号脉内的时频特征实现对目标雷达的具体信息进行有效评估。设计了一种卷积-双向长短时记忆(Convolution-Bidirectional Long Short-Term Memory,CNN-BiLSTM)混合神经网络模型,主要通过BiLSTM的时序记忆特性深度挖掘雷达信号的时域特征,结合权值共享特性和CNN层捕获雷达信号的时频特征,再利用二者信号特征联合完成对雷达信号调制方式的识别。通过对比实验验证,所提方法对若干种雷达信号的识别具有较高的准确度,平均值达到95.349%;优于只使用单一特征的网络和传统算法,具有良好的抗噪声能力。

A Hybrid Neural Network Model for ENSO Prediction in Combination with Principal Oscillation Pattern Analyses

El Niño-Southern Oscillation(ENSO)can be currently predicted reasonably well six months and longer,but large biases and uncertainties remain in its real-time prediction.Various approaches have been taken to improve understanding of ENSO processes,and different models for ENSO predictions have been developed,including linear statistical models based on principal oscillation pattern(POP)analyses,convolutional neural networks(CNNs),and so on.Here,we develop a novel hybrid model,named as POP-Net,by combining the POP analysis procedure with CNN-long short-term memory(LSTM)algorithm to predict the Niño-3.4 sea surface temperature(SST)index.ENSO predictions are compared with each other from the corresponding three models:POP model,CNN-LSTM model,and POP-Net,respectively.The POP-based pre-processing acts to enhance ENSO-related signals of interest while filtering unrelated noise.Consequently,an improved prediction is achieved in the POP-Net relative to others.The POP-Net shows a high-correlation skill for 17-month lead time prediction(correlation coefficients exceeding 0.5)during the 1994-2020 validation period.The POP-Net also alleviates the spring predictability barrier(SPB).It is concluded that value-added artificial neural networks for improved ENSO predictions are possible by including the process-oriented analyses to enhance signal representations.

ARTIFICIAL NEURAL NETWORKS BASED GEARS MATERIAL SELECTION HYBRID INTELLIGENT SYSTEM

An artificial neural networks(ANNs) based gear material selection hybrid intelligent system is established by analyzing the individual advantages and weakness of expert system (ES) and ANNs and the applications in material select of them. The system mainly consists of tow parts: ES and ANNs. By being trained with much data samples, the back propagation (BP) ANN gets the knowledge of gear materials selection, and is able to inference according to user input. The system realizes the complementing of ANNs and ES. Using this system, engineers without materials selection experience can conveniently deal with gear materials selection.

A hybrid neural network system for prediction and recognition of promoter regions in human genome

This paper proposes a high specificity and sensitivity algorithm called PromPredictor for recognizing promoter regions in the human genome. PromPredictor extracts compositional features and CpG islands information from genomic sequence,feeding these features as input for a hybrid neural network system (HNN) and then applies the HNN for prediction. It combines a novel promoter recognition model, coding theory, feature selection and dimensionality reduction with machine learning algorithm.Evaluation on Human chromosome 22 was ～66% in sensitivity and ～48% in specificity. Comparison with two other systems revealed that our method had superior sensitivity and specificity in predicting promoter regions. PromPredictor is written in MATLAB and requires Matlab to run. PromPredictor is freely available at http://www.whtelecom.com/Prompredictor.htm.

Exponential synchronization of general chaotic delayed neural networks via hybrid feedback

This paper investigates the exponential synchronization problem of some chaotic delayed neural networks based on the proposed general neural network model,which is the interconnection of a linear delayed dynamic system and a bounded static nonlinear operator,and covers several well-known neural networks,such as Hopfield neural networks,cellular neural networks(CNNs),bidirectional associative memory(BAM)networks,recurrent multilayer perceptrons(RMLPs).By virtue of Lyapunov-Krasovskii stability theory and linear matrix inequality(LMI)technique,some exponential synchronization criteria are derived.Using the drive-response concept,hybrid feedback controllers are designed to synchronize two identical chaotic neural networks based on those synchronization criteria.Finally,detailed comparisons with existing results are made and numerical simulations are carried out to demonstrate the effectiveness of the established synchronization laws.

A hybrid model for short-term rainstorm forecasting based on a back-propagation neural network and synoptic diagnosis

Rainstorms are one of the most important types of natural disaster in China.In order to enhance the ability to forecast rainstorms in the short term,this paper explores how to combine a back-propagation neural network(BPNN)with synoptic diagnosis for predicting rainstorms,and analyzes the hit rates of rainstorms for the above two methods using the county of Tianquan as a case study.Results showed that the traditional synoptic diagnosis method still has an important referential meaning for most rainstorm types through synoptic typing and statistics of physical quantities based on historical cases,and the threat score(TS)of rainstorms was more than 0.75.However,the accuracy for two rainstorm types influenced by low-level easterly inverted troughs was less than 40%.The BPNN method efficiently forecasted these two rainstorm types;the TS and equitable threat score(ETS)of rainstorms were 0.80 and 0.79,respectively.The TS and ETS of the hybrid model that combined the BPNN and synoptic diagnosis methods exceeded the forecast score of multi-numerical simulations over the Sichuan Basin without exception.This kind of hybrid model enhanced the forecasting accuracy of rainstorms.The findings of this study provide certain reference value for the future development of refined forecast models with local features.