Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

Design of a novel hybrid quantum deep neural network in INEQR images classification

We redesign the parameterized quantum circuit in the quantum deep neural network, construct a three-layer structure as the hidden layer, and then use classical optimization algorithms to train the parameterized quantum circuit, thereby propose a novel hybrid quantum deep neural network(HQDNN) used for image classification. After bilinear interpolation reduces the original image to a suitable size, an improved novel enhanced quantum representation(INEQR) is used to encode it into quantum states as the input of the HQDNN. Multi-layer parameterized quantum circuits are used as the main structure to implement feature extraction and classification. The output results of parameterized quantum circuits are converted into classical data through quantum measurements and then optimized on a classical computer. To verify the performance of the HQDNN, we conduct binary classification and three classification experiments on the MNIST(Modified National Institute of Standards and Technology) data set. In the first binary classification, the accuracy of 0 and 4 exceeds98%. Then we compare the performance of three classification with other algorithms, the results on two datasets show that the classification accuracy is higher than that of quantum deep neural network and general quantum convolutional neural network.

Analysis of learnability of a novel hybrid quantum-classical convolutional neural network in image classification

We design a new hybrid quantum-classical convolutional neural network(HQCCNN)model based on parameter quantum circuits.In this model,we use parameterized quantum circuits(PQCs)to redesign the convolutional layer in classical convolutional neural networks,forming a new quantum convolutional layer to achieve unitary transformation of quantum states,enabling the model to more accurately extract hidden information from images.At the same time,we combine the classical fully connected layer with PQCs to form a new hybrid quantum-classical fully connected layer to further improve the accuracy of classification.Finally,we use the MNIST dataset to test the potential of the HQCCNN.The results indicate that the HQCCNN has good performance in solving classification problems.In binary classification tasks,the classification accuracy of numbers 5 and 7 is as high as 99.71%.In multivariate classification,the accuracy rate also reaches 98.51%.Finally,we compare the performance of the HQCCNN with other models and find that the HQCCNN has better classification performance and convergence speed.

Hybrid model for BOF oxygen blowing time prediction based on oxygen balance mechanism and deep neural network

The amount of oxygen blown into the converter is one of the key parameters for the control of the converter blowing process,which directly affects the tap-to-tap time of converter. In this study, a hybrid model based on oxygen balance mechanism (OBM) and deep neural network (DNN) was established for predicting oxygen blowing time in converter. A three-step method was utilized in the hybrid model. First, the oxygen consumption volume was predicted by the OBM model and DNN model, respectively. Second, a more accurate oxygen consumption volume was obtained by integrating the OBM model and DNN model. Finally, the converter oxygen blowing time was calculated according to the oxygen consumption volume and the oxygen supply intensity of each heat. The proposed hybrid model was verified using the actual data collected from an integrated steel plant in China, and compared with multiple linear regression model, OBM model, and neural network model including extreme learning machine, back propagation neural network, and DNN. The test results indicate that the hybrid model with a network structure of 3 hidden layer layers, 32-16-8 neurons per hidden layer, and 0.1 learning rate has the best prediction accuracy and stronger generalization ability compared with other models. The predicted hit ratio of oxygen consumption volume within the error±300 m^(3)is 96.67%;determination coefficient (R^(2)) and root mean square error (RMSE) are0.6984 and 150.03 m^(3), respectively. The oxygen blow time prediction hit ratio within the error±0.6 min is 89.50%;R2and RMSE are0.9486 and 0.3592 min, respectively. As a result, the proposed model can effectively predict the oxygen consumption volume and oxygen blowing time in the converter.

Energy-Efficient Traffic Offloading for RSMA-Based Hybrid Satellite Terrestrial Networks with Deep Reinforcement Learning

As the demands of massive connections and vast coverage rapidly grow in the next wireless communication networks, rate splitting multiple access(RSMA) is considered to be the new promising access scheme since it can provide higher efficiency with limited spectrum resources. In this paper, combining spectrum splitting with rate splitting, we propose to allocate resources with traffic offloading in hybrid satellite terrestrial networks. A novel deep reinforcement learning method is adopted to solve this challenging non-convex problem. However, the neverending learning process could prohibit its practical implementation. Therefore, we introduce the switch mechanism to avoid unnecessary learning. Additionally, the QoS constraint in the scheme can rule out unsuccessful transmission. The simulation results validates the energy efficiency performance and the convergence speed of the proposed algorithm.

A CSMA/CA based MAC protocol for hybrid Power-line/Visible-light communication networks:Design and analysis

Hybrid Power-line/Visible-light Communication(HPVC)network has been one of the most promising Cooperative Communication(CC)technologies for constructing Smart Home due to its superior communication reliability and hardware efficiency.Current research on HPVC networks focuses on the performance analysis and optimization of the Physical(PHY)layer,where the Power Line Communication(PLC)component only serves as the backbone to provide power to light Emitting Diode(LED)devices.So designing a Media Access Control(MAC)protocol remains a great challenge because it allows both PLC and Visible Light Communication(VLC)components to operate data transmission,i.e.,to achieve a true HPVC network CC.To solve this problem,we propose a new HPC network MAC protocol(HPVC MAC)based on Carrier Sense Multiple Access/Collision Avoidance(CSMA/CA)by combining IEEE 802.15.7 and IEEE 1901 standards.Firstly,we add an Additional Assistance(AA)layer to provide the channel selection strategies for sensor stations,so that they can complete data transmission on the selected channel via the specified CSMA/CA mechanism,respectively.Based on this,we give a detailed working principle of the HPVC MAC,followed by the construction of a joint analytical model for mathematicalmathematical validation of the HPVC MAC.In the modeling process,the impacts of PHY layer settings(including channel fading types and additive noise feature),CSMA/CA mechanisms of 802.15.7 and 1901,and practical configurations(such as traffic rate,transit buffer size)are comprehensively taken into consideration.Moreover,we prove the proposed analytical model has the solvability.Finally,through extensive simulations,we characterize the HPVC MAC performance under different system parameters and verify the correctness of the corresponding analytical model with an average error rate of 4.62%between the simulation and analytical results.

Deep hybrid: Multi-graph neural network collaboration for hyperspectral image classification

With limited number of labeled samples,hyperspectral image(HSI)classification is a difficult Problem in current research.The graph neural network(GNN)has emerged as an approach to semi-supervised classification,and the application of GNN to hyperspectral images has attracted much attention.However,in the existing GNN-based methods a single graph neural network or graph filter is mainly used to extract HSI features,which does not take full advantage of various graph neural networks(graph filters).Moreover,the traditional GNNs have the problem of oversmoothing.To alleviate these shortcomings,we introduce a deep hybrid multi-graph neural network(DHMG),where two different graph filters,i.e.,the spectral filter and the autoregressive moving average(ARMA)filter,are utilized in two branches.The former can well extract the spectral features of the nodes,and the latter has a good suppression effect on graph noise.The network realizes information interaction between the two branches and takes good advantage of different graph filters.In addition,to address the problem of oversmoothing,a dense network is proposed,where the local graph features are preserved.The dense structure satisfies the needs of different classification targets presenting different features.Finally,we introduce a GraphSAGEbased network to refine the graph features produced by the deep hybrid network.Extensive experiments on three public HSI datasets strongly demonstrate that the DHMG dramatically outperforms the state-ofthe-art models.

Downlink Resource Allocation for NOMA-Based Hybrid Spectrum Access in Cognitive Network

To solve the contradiction between limited spectrum resources and increasing communication demand,this paper proposes a wireless resource allocation scheme based on the Deep Q Network(DQN)to allocate radio resources in a downlink multi-user cognitive radio(CR)network with slicing.Secondary users(SUs)are multiplexed using non-orthogonal multiple access(NOMA).The SUs use the hybrid spectrum access mode to improve the spectral efficiency(SE).Considering the demand for multiple services,the enhanced mobile broadband(eMBB)slice and ultrareliable low-latency communication(URLLC)slice were established.The proposed scheme can maximize the SE while ensuring Quality of Service(QoS)for the users.This study established a mapping relationship between resource allocation and the DQN algorithm in the CR-NOMA network.According to the signal-to-interference-plusnoise ratio(SINR)of the primary users(PUs),the proposed scheme can output the optimal channel selection and power allocation.The simulation results reveal that the proposed scheme can converge faster and obtain higher rewards compared with the Q-Learning scheme.Additionally,the proposed scheme has better SE than both the overlay and underlay only modes.

A New Hybrid Approach Using GWO and MFO Algorithms to Detect Network Attack

This paper addresses the urgent need to detect network security attacks,which have increased significantly in recent years,with high accuracy and avoid the adverse effects of these attacks.The intrusion detection system should respond seamlessly to attack patterns and approaches.The use of metaheuristic algorithms in attack detection can produce near-optimal solutions with low computational costs.To achieve better performance of these algorithms and further improve the results,hybridization of algorithms can be used,which leads to more successful results.Nowadays,many studies are conducted on this topic.In this study,a new hybrid approach using Gray Wolf Optimizer(GWO)and Moth-Flame Optimization(MFO)algorithms was developed and applied to widely used data sets such as NSL-KDD,UNSW-NB15,and CIC IDS 2017,as well as various benchmark functions.The ease of hybridization of the GWO algorithm,its simplicity,its ability to perform global optimal search,and the success of the MFO algorithm in obtaining the best solution suggested that an effective solution would be obtained by combining these two algorithms.For these reasons,the developed hybrid algorithm aims to achieve better results by using the good aspects of both the GWO algorithm and the MFO algorithm.In reviewing the results,it was found that a high level of success was achieved in the benchmark functions.It achieved better results in 12 of the 13 benchmark functions compared.In addition,the success rates obtained according to the evaluation criteria in the different data sets are also remarkable.Comparing the 97.4%,98.3%,and 99.2% classification accuracy results obtained in the NSL-KDD,UNSW-NB15,and CIC IDS 2017 data sets with the studies in the literature,they seem to be quite successful.

Bottom hole pressure prediction based on hybrid neural networks and Bayesian optimization

Many scholars have focused on applying machine learning models in bottom hole pressure (BHP) prediction. However, the complex and uncertain conditions in deep wells make it difficult to capture spatial and temporal correlations of measurement while drilling (MWD) data with traditional intelligent models. In this work, we develop a novel hybrid neural network, which integrates the Convolution Neural Network (CNN) and the Gate Recurrent Unit (GRU) for predicting BHP fluctuations more accurately. The CNN structure is used to analyze spatial local dependency patterns and the GRU structure is used to discover depth variation trends of MWD data. To further improve the prediction accuracy, we explore two types of GRU-based structure: skip-GRU and attention-GRU, which can capture more long-term potential periodic correlation in drilling data. Then, the different model structures tuned by the Bayesian optimization (BO) algorithm are compared and analyzed. Results indicate that the hybrid models can extract spatial-temporal information of data effectively and predict more accurately than random forests, extreme gradient boosting, back propagation neural network, CNN and GRU. The CNN-attention-GRU model with BO algorithm shows great superiority in prediction accuracy and robustness due to the hybrid network structure and attention mechanism, having the lowest mean absolute percentage error of 0.025%. This study provides a reference for solving the problem of extracting spatial and temporal characteristics and guidance for managed pressure drilling in complex formations.

Using Hybrid Penalty and Gated Linear Units to Improve Wasserstein Generative Adversarial Networks for Single-Channel Speech Enhancement

Recently,speech enhancement methods based on Generative Adversarial Networks have achieved good performance in time-domain noisy signals.However,the training of Generative Adversarial Networks has such problems as convergence difficulty,model collapse,etc.In this work,an end-to-end speech enhancement model based on Wasserstein Generative Adversarial Networks is proposed,and some improvements have been made in order to get faster convergence speed and better generated speech quality.Specifically,in the generator coding part,each convolution layer adopts different convolution kernel sizes to conduct convolution operations for obtaining speech coding information from multiple scales;a gated linear unit is introduced to alleviate the vanishing gradient problem with the increase of network depth;the gradient penalty of the discriminator is replaced with spectral normalization to accelerate the convergence rate of themodel;a hybrid penalty termcomposed of L1 regularization and a scale-invariant signal-to-distortion ratio is introduced into the loss function of the generator to improve the quality of generated speech.The experimental results on both TIMIT corpus and Tibetan corpus show that the proposed model improves the speech quality significantly and accelerates the convergence speed of the model.

LaNets:Hybrid Lagrange Neural Networks for Solving Partial Differential Equations

We propose new hybrid Lagrange neural networks called LaNets to predict the numerical solutions of partial differential equations.That is,we embed Lagrange interpolation and small sample learning into deep neural network frameworks.Concretely,we first perform Lagrange interpolation in front of the deep feedforward neural network.The Lagrange basis function has a neat structure and a strong expression ability,which is suitable to be a preprocessing tool for pre-fitting and feature extraction.Second,we introduce small sample learning into training,which is beneficial to guide themodel to be corrected quickly.Taking advantages of the theoretical support of traditional numerical method and the efficient allocation of modern machine learning,LaNets achieve higher predictive accuracy compared to the state-of-the-artwork.The stability and accuracy of the proposed algorithmare demonstrated through a series of classical numerical examples,including one-dimensional Burgers equation,onedimensional carburizing diffusion equations,two-dimensional Helmholtz equation and two-dimensional Burgers equation.Experimental results validate the robustness,effectiveness and flexibility of the proposed algorithm.

A Hybrid Approach to Neighbour Discovery in Wireless Sensor Networks

In the contemporary era of unprecedented innovations such as Internet of Things(IoT),modern applications cannot be imagined without the presence of Wireless Sensor Network(WSN).Nodes in WSN use neighbour discovery(ND)protocols to have necessary communication among the nodes.Neighbour discovery process is crucial as it is to be done with energy efficiency and minimize discovery latency and maximize percentage of neighbours discovered.The current ND approaches that are indirect in nature are categorized into methods of removal of active slots from wake-up schedules and intelligent addition of new slots.The two methods are found to have certain drawbacks.Thefirst category disturbs original integrity of wake-up schedules leading to reduced chances of discovering new nodes in WSN as neighbours.When second category is followed,it may have inefficient slots in the wake-up schedules leading to performance degradation.Therefore,the motivation behind the work in this paper is that by combining the two categories,it is possible to reap benefits of both and get rid of the limitations of the both.Making a hybrid is achieved by introducing virtual nodes that help maximize performance by ensuring original integrity of wake-up schedules and adding of efficient active slots.Thus a Hybrid Approach to Neighbour Discovery(HAND)protocol is realized in WSN.The simulation study revealed that HAND outperforms the existing indirect ND models.

Hyperparameter Optimization for Capsule Network Based Modified Hybrid Rice Optimization Algorithm

Hyperparameters play a vital impact in the performance of most machine learning algorithms.It is a challenge for traditional methods to con-figure hyperparameters of the capsule network to obtain high-performance manually.Some swarm intelligence or evolutionary computation algorithms have been effectively employed to seek optimal hyperparameters as a com-binatorial optimization problem.However,these algorithms are prone to get trapped in the local optimal solution as random search strategies are adopted.The inspiration for the hybrid rice optimization(HRO)algorithm is from the breeding technology of three-line hybrid rice in China,which has the advantages of easy implementation,less parameters and fast convergence.In the paper,genetic search is combined with the hybrid rice optimization algorithm(GHRO)and employed to obtain the optimal hyperparameter of the capsule network automatically,that is,a probability search technique and a hybridization strategy belong with the primary HRO.Thirteen benchmark functions are used to evaluate the performance of GHRO.Furthermore,the MNIST,Chest X-Ray(pneumonia),and Chest X-Ray(COVID-19&pneumonia)datasets are also utilized to evaluate the capsule network learnt by GHRO.The experimental results show that GHRO is an effective method for optimizing the hyperparameters of the capsule network,which is able to boost the performance of the capsule network on image classification.

Lightweight and highly robust memristor-based hybrid neural networks for electroencephalogram signal processing

Memristor-based neuromorphic computing shows great potential for high-speed and high-throughput signal processing applications,such as electroencephalogram(EEG)signal processing.Nonetheless,the size of one-transistor one-resistor(1T1R)memristor arrays is limited by the non-ideality of the devices,which prevents the hardware implementation of large and complex networks.In this work,we propose the depthwise separable convolution and bidirectional gate recurrent unit(DSC-BiGRU)network,a lightweight and highly robust hybrid neural network based on 1T1R arrays that enables efficient processing of EEG signals in the temporal,frequency and spatial domains by hybridizing DSC and BiGRU blocks.The network size is reduced and the network robustness is improved while ensuring the network classification accuracy.In the simulation,the measured non-idealities of the 1T1R array are brought into the network through statistical analysis.Compared with traditional convolutional networks,the network parameters are reduced by 95%and the network classification accuracy is improved by 21%at a 95%array yield rate and 5%tolerable error.This work demonstrates that lightweight and highly robust networks based on memristor arrays hold great promise for applications that rely on low consumption and high efficiency.

Hybrid Power Bank Deployment Model for Energy Supply Coverage Optimization in Industrial Wireless Sensor Network

Energy supply is one of the most critical challenges of wireless sensor networks(WSNs)and industrial wireless sensor networks(IWSNs).While research on coverage optimization problem(COP)centers on the network’s monitoring coverage,this research focuses on the power banks’energy supply coverage.The study of 2-D and 3-D spaces is typical in IWSN,with the realistic environment being more complex with obstacles(i.e.,machines).A 3-D surface is the field of interest(FOI)in this work with the established hybrid power bank deployment model for the energy supply COP optimization of IWSN.The hybrid power bank deployment model is highly adaptive and flexible for new or existing plants already using the IWSN system.The model improves the power supply to a more considerable extent with the least number of power bank deployments.The main innovation in this work is the utilization of a more practical surface model with obstacles and training while improving the convergence speed and quality of the heuristic algorithm.An overall probabilistic coverage rate analysis of every point on the FOI is provided,not limiting the scope to target points or areas.Bresenham’s algorithm is extended from 2-D to 3-D surface to enhance the probabilistic covering model for coverage measurement.A dynamic search strategy(DSS)is proposed to modify the artificial bee colony(ABC)and balance the exploration and exploitation ability for better convergence toward eliminating NP-hard deployment problems.Further,the cellular automata(CA)is utilized to enhance the convergence speed.The case study based on two typical FOI in the IWSN shows that the CA scheme effectively speeds up the optimization process.Comparative experiments are conducted on four benchmark functions to validate the effectiveness of the proposed method.The experimental results show that the proposed algorithm outperforms the ABC and gbest-guided ABC(GABC)algorithms.The results show that the proposed energy coverage optimization method based on the hybrid power bank deployment model generates more accurate results than the results obtained by similar algorithms(i.e.,ABC,GABC).The proposed model is,therefore,effective and efficient for optimization in the IWSN.

The Evolving Bipartite Network and Semi-Bipartite Network Models with Adjustable Scale and Hybrid Attachment Mechanisms

The bipartite graph structure exists in the connections of many objects in the real world, and the evolving modeling is a good method to describe and understand the generation and evolution within various real complex networks. Previous bipartite models were proposed to mostly explain the principle of attachments, and ignored the diverse growth speed of nodes of sets in different bipartite networks. In this paper, we propose an evolving bipartite network model with adjustable node scale and hybrid attachment mechanisms, which uses different probability parameters to control the scale of two disjoint sets of nodes and the preference strength of hybrid attachment respectively. The results show that the degree distribution of single set in the proposed model follows a shifted power-law distribution when parameter r and s are not equal to 0, or exponential distribution when r or s is equal to 0. Furthermore, we extend the previous model to a semi-bipartite network model, which embeds more user association information into the internal network, so that the model is capable of carrying and revealing more deep information of each user in the network. The simulation results of two models are in good agreement with the empirical data, which verifies that the models have a good performance on real networks from the perspective of degree distribution. We believe these two models are valuable for an explanation of the origin and growth of bipartite systems that truly exist.

Mechanism analysis of regulating Turing instability and Hopf bifurcation of malware propagation in mobile wireless sensor networks

A dynamical model is constructed to depict the spatial-temporal evolution of malware in mobile wireless sensor networks(MWSNs). Based on such a model, we design a hybrid control scheme combining parameter perturbation and state feedback to effectively manipulate the spatiotemporal dynamics of malware propagation. The hybrid control can not only suppress the Turing instability caused by diffusion factor but can also adjust the occurrence of Hopf bifurcation induced by time delay. Numerical simulation results show that the hybrid control strategy can efficiently manipulate the transmission dynamics to achieve our expected desired properties, thus reducing the harm of malware propagation to MWSNs.

Solar Radiation Estimation Based on a New Combined Approach of Artificial Neural Networks (ANN) and Genetic Algorithms (GA) in South Algeria

When designing solar systems and assessing the effectiveness of their many uses,estimating sun irradiance is a crucial first step.This study examined three approaches(ANN,GA-ANN,and ANFIS)for estimating daily global solar radiation(GSR)in the south of Algeria:Adrar,Ouargla,and Bechar.The proposed hybrid GA-ANN model,based on genetic algorithm-based optimization,was developed to improve the ANN model.The GA-ANN and ANFIS models performed better than the standalone ANN-based model,with GA-ANN being better suited for forecasting in all sites,and it performed the best with the best values in the testing phase of Coefficient of Determination(R=0.9005),Mean Absolute Percentage Error(MAPE=8.40%),and Relative Root Mean Square Error(rRMSE=12.56%).Nevertheless,the ANFIS model outperformed the GA-ANN model in forecasting daily GSR,with the best values of indicators when testing the model being R=0.9374,MAPE=7.78%,and rRMSE=10.54%.Generally,we may conclude that the initial ANN stand-alone model performance when forecasting solar radiation has been improved,and the results obtained after injecting the genetic algorithm into the ANN to optimize its weights were satisfactory.The model can be used to forecast daily GSR in dry climates and other climates and may also be helpful in selecting solar energy system installations and sizes.

Optimizing wind farm layout for enhanced electricity extraction using a new hybrid PSO-ANN method

With the growing need for renewable energy,wind farms are playing an important role in generating clean power from wind resources.The best wind turbine architecture in a wind farm has a major influence on the energy extraction efficiency.This paper describes a unique strategy for optimizing wind turbine locations on a wind farm that combines the capabilities of particle swarm optimization(PSO)and artificial neural networks(ANNs).The PSO method was used to explore the solution space and develop preliminary turbine layouts,and the ANN model was used to fine-tune the placements based on the predicted energy generation.The proposed hybrid technique seeks to increase energy output while considering site-specific wind patterns and topographical limits.The efficacy and superiority of the hybrid PSO-ANN methodology are proved through comprehensive simulations and comparisons with existing approaches,giving exciting prospects for developing more efficient and sustainable wind farms.The integration of ANNs and PSO in our methodology is of paramount importance because it leverages the complementary strengths of both techniques.Furthermore,this novel methodology harnesses historical data through ANNs to identify optimal turbine positions that align with the wind speed and direction and enhance energy extraction efficiency.A notable increase in power generation is observed across various scenarios.The percentage increase in the power generation ranged from approximately 7.7%to 11.1%.Owing to its versatility and adaptability to site-specific conditions,the hybrid model offers promising prospects for advancing the field of wind farm layout optimization and contributing to a greener and more sustainable energy future.