Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.

Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

A data-driven model of drop size prediction based on artificial neural networks using small-scale data sets

An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.

Development of a convolutional neural network based geomechanical upscaling technique for heterogeneous geological reservoir

Geomechanical assessment using coupled reservoir-geomechanical simulation is becoming increasingly important for analyzing the potential geomechanical risks in subsurface geological developments.However,a robust and efficient geomechanical upscaling technique for heterogeneous geological reservoirs is lacking to advance the applications of three-dimensional(3D)reservoir-scale geomechanical simulation considering detailed geological heterogeneities.Here,we develop convolutional neural network(CNN)proxies that reproduce the anisotropic nonlinear geomechanical response caused by lithological heterogeneity,and compute upscaled geomechanical properties from CNN proxies.The CNN proxies are trained using a large dataset of randomly generated spatially correlated sand-shale realizations as inputs and simulation results of their macroscopic geomechanical response as outputs.The trained CNN models can provide the upscaled shear strength(R^(2)>0.949),stress-strain behavior(R^(2)>0.925),and volumetric strain changes(R^(2)>0.958)that highly agree with the numerical simulation results while saving over two orders of magnitude of computational time.This is a major advantage in computing the upscaled geomechanical properties directly from geological realizations without the need to perform local numerical simulations to obtain the geomechanical response.The proposed CNN proxybased upscaling technique has the ability to(1)bridge the gap between the fine-scale geocellular models considering geological uncertainties and computationally efficient geomechanical models used to assess the geomechanical risks of large-scale subsurface development,and(2)improve the efficiency of numerical upscaling techniques that rely on local numerical simulations,leading to significantly increased computational time for uncertainty quantification using numerous geological realizations.

An End-To-End Hyperbolic Deep Graph Convolutional Neural Network Framework

Graph Convolutional Neural Networks(GCNs)have been widely used in various fields due to their powerful capabilities in processing graph-structured data.However,GCNs encounter significant challenges when applied to scale-free graphs with power-law distributions,resulting in substantial distortions.Moreover,most of the existing GCN models are shallow structures,which restricts their ability to capture dependencies among distant nodes and more refined high-order node features in scale-free graphs with hierarchical structures.To more broadly and precisely apply GCNs to real-world graphs exhibiting scale-free or hierarchical structures and utilize multi-level aggregation of GCNs for capturing high-level information in local representations,we propose the Hyperbolic Deep Graph Convolutional Neural Network(HDGCNN),an end-to-end deep graph representation learning framework that can map scale-free graphs from Euclidean space to hyperbolic space.In HDGCNN,we define the fundamental operations of deep graph convolutional neural networks in hyperbolic space.Additionally,we introduce a hyperbolic feature transformation method based on identity mapping and a dense connection scheme based on a novel non-local message passing framework.In addition,we present a neighborhood aggregation method that combines initial structural featureswith hyperbolic attention coefficients.Through the above methods,HDGCNN effectively leverages both the structural features and node features of graph data,enabling enhanced exploration of non-local structural features and more refined node features in scale-free or hierarchical graphs.Experimental results demonstrate that HDGCNN achieves remarkable performance improvements over state-ofthe-art GCNs in node classification and link prediction tasks,even when utilizing low-dimensional embedding representations.Furthermore,when compared to shallow hyperbolic graph convolutional neural network models,HDGCNN exhibits notable advantages and performance enhancements.

Application of Convolutional Neural Networks in Classification of GBM for Enhanced Prognosis

The lethal brain tumor “Glioblastoma” has the propensity to grow over time. To improve patient outcomes, it is essential to classify GBM accurately and promptly in order to provide a focused and individualized treatment plan. Despite this, deep learning methods, particularly Convolutional Neural Networks (CNNs), have demonstrated a high level of accuracy in a myriad of medical image analysis applications as a result of recent technical breakthroughs. The overall aim of the research is to investigate how CNNs can be used to classify GBMs using data from medical imaging, to improve prognosis precision and effectiveness. This research study will demonstrate a suggested methodology that makes use of the CNN architecture and is trained using a database of MRI pictures with this tumor. The constructed model will be assessed based on its overall performance. Extensive experiments and comparisons with conventional machine learning techniques and existing classification methods will also be made. It will be crucial to emphasize the possibility of early and accurate prediction in a clinical workflow because it can have a big impact on treatment planning and patient outcomes. The paramount objective is to not only address the classification challenge but also to outline a clear pathway towards enhancing prognosis precision and treatment effectiveness.

A Review of Computing with Spiking Neural Networks

Artificial neural networks(ANNs)have led to landmark changes in many fields,but they still differ significantly fromthemechanisms of real biological neural networks and face problems such as high computing costs,excessive computing power,and so on.Spiking neural networks(SNNs)provide a new approach combined with brain-like science to improve the computational energy efficiency,computational architecture,and biological credibility of current deep learning applications.In the early stage of development,its poor performance hindered the application of SNNs in real-world scenarios.In recent years,SNNs have made great progress in computational performance and practicability compared with the earlier research results,and are continuously producing significant results.Although there are already many pieces of literature on SNNs,there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results.Starting from this issue,this paper elaborates on SNNs along the complete usage process of SNNs including network construction,data processing,model training,development,and deployment,aiming to provide more comprehensive and practical guidance to promote the development of SNNs.Therefore,the connotation and development status of SNNcomputing is reviewed systematically and comprehensively from four aspects:composition structure,data set,learning algorithm,software/hardware development platform.Then the development characteristics of SNNs in intelligent computing are summarized,the current challenges of SNNs are discussed and the future development directions are also prospected.Our research shows that in the fields of machine learning and intelligent computing,SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks.The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited.And the development of programming and deployment tools has lowered the threshold for the use of SNNs.SNNs show a broad development prospect for brain-like computing.

Expression Recognition Method Based on Convolutional Neural Network and Capsule Neural Network

Convolutional neural networks struggle to accurately handle changes in angles and twists in the direction of images,which affects their ability to recognize patterns based on internal feature levels. In contrast, CapsNet overcomesthese limitations by vectorizing information through increased directionality and magnitude, ensuring that spatialinformation is not overlooked. Therefore, this study proposes a novel expression recognition technique calledCAPSULE-VGG, which combines the strengths of CapsNet and convolutional neural networks. By refining andintegrating features extracted by a convolutional neural network before introducing theminto CapsNet, ourmodelenhances facial recognition capabilities. Compared to traditional neural network models, our approach offersfaster training pace, improved convergence speed, and higher accuracy rates approaching stability. Experimentalresults demonstrate that our method achieves recognition rates of 74.14% for the FER2013 expression dataset and99.85% for the CK+ expression dataset. By contrasting these findings with those obtained using conventionalexpression recognition techniques and incorporating CapsNet’s advantages, we effectively address issues associatedwith convolutional neural networks while increasing expression identification accuracy.

Unveiling the brain’s symphony:exploring the necessity and sufficiency of neural networks in behavior control

Since the pioneering work by Broca and Wernicke in the 19th century,who examined individuals with brain lesions to associate them with specific behaviors,it was evident that behaviors are complex and cannot be fully attributable to specific brain areas alone.Instead,they involve connectivity among brain areas,whether close or distant.At that time,this approach was considered the optimal way to dissect brain circuitry and function.These pioneering efforts opened the field to explore the necessity or sufficiency of brain areas in controlling behavior and hence dissecting brain function.However,the connectivity of the brain and the mechanisms through which various brain regions regulate specific behaviors,either individually or collaboratively,remain largely elusive.Utilizing animal models,researchers have endeavored to unravel the necessity or sufficiency of specific brain areas in influencing behavior;however,no clear associations have been firmly established.

Evidence of the Great Attractor and Great Repeller from Artificial Neural Network Imputation of Sloan Digital Sky Survey

The Sloane Digital Sky Survey (SDSS) has been in the process of creating a 3D digital map of the Universe, since 2000AD. However, it has not been able to map that portion of the sky which is occluded by the dust gas and stars of our own Milkyway Galaxy. This research builds on work from a previous paper that sought to impute this missing galactic information using Inpainting, polar transforms and Linear Regression ANNs. In that paper, the author only attempted to impute the data in the Northern hemisphere using the ANN model, which subsequently confirmed the existence of the Great Attractor and the homogeneity of the Universe. In this paper, the author has imputed the Southern Hemisphere and discovered a region that is mostly devoid of stars. Since this area appears to be the counterpart to the Great Attractor, the author refers to it as the Great Repeller and postulates that it is an area of physical repulsion, inline with the work of GerdPommerenke and others. Finally, the paper investigates large scale structures in the imputed galaxies.

NeurstrucEnergy:A bi-directional GNN model for energy prediction of neural networks in IoT

A significant demand rises for energy-efficient deep neural networks to support power-limited embedding devices with successful deep learning applications in IoT and edge computing fields.An accurate energy prediction approach is critical to provide measurement and lead optimization direction.However,the current energy prediction approaches lack accuracy and generalization ability due to the lack of research on the neural network structure and the excessive reliance on customized training dataset.This paper presents a novel energy prediction model,NeurstrucEnergy.NeurstrucEnergy treats neural networks as directed graphs and applies a bi-directional graph neural network training on a randomly generated dataset to extract structural features for energy prediction.NeurstrucEnergy has advantages over linear approaches because the bi-directional graph neural network collects structural features from each layer's parents and children.Experimental results show that NeurstrucEnergy establishes state-of-the-art results with mean absolute percentage error of 2.60%.We also evaluate NeurstrucEnergy in a randomly generated dataset,achieving the mean absolute percentage error of 4.83%over 10 typical convolutional neural networks in recent years and 7 efficient convolutional neural networks created by neural architecture search.Our code is available at https://github.com/NEUSoftGreenAI/NeurstrucEnergy.git.

Prediction of Porous Media Fluid Flow with Spatial Heterogeneity Using Criss-Cross Physics-Informed Convolutional Neural Networks

Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsinformedneural network(PINN)is currently the most general framework,which is more popular due to theconvenience of constructing NNs and excellent generalization ability.The automatic differentiation(AD)-basedPINN model is suitable for the homogeneous scientific problem;however,it is unclear how AD can enforce fluxcontinuity across boundaries between cells of different properties where spatial heterogeneity is represented bygrid cells with different physical properties.In this work,we propose a criss-cross physics-informed convolutionalneural network(CC-PINN)learning architecture,aiming to learn the solution of parametric PDEs with spatialheterogeneity of physical properties.To achieve the seamless enforcement of flux continuity and integration ofphysicalmeaning into CNN,a predefined 2D convolutional layer is proposed to accurately express transmissibilitybetween adjacent cells.The efficacy of the proposedmethodwas evaluated through predictions of several petroleumreservoir problems with spatial heterogeneity and compared against state-of-the-art(PINN)through numericalanalysis as a benchmark,which demonstrated the superiority of the proposed method over the PINN.

Double integral-enhanced Zeroing neural network with linear noise rejection for time-varying matrix inverse

In engineering fields,time-varying matrix inversion(TVMI)issue is often encountered.Zeroing neural network(ZNN)has been extensively employed to resolve the TVMI problem.Nevertheless,the original ZNN(OZNN)and the integral-enhanced ZNN(IEZNN)usually fail to deal with the TVMI problem under unbounded noises,such as linear noises.Therefore,a neural network model that can handle the TVMI under linear noise interference is urgently needed.This paper develops a double integral-enhanced ZNN(DIEZNN)model based on a novel integral-type design formula with inherent linear-noise tolerance.Moreover,its convergence and robustness are verified by deriva-tion strictly.For comparison and verification,the OZNN and the IEZNN models are adopted to resolve the TVMI under multiple identical noise environments.The experi-ments proved that the DIEZNN model has excellent advantages in solving TVMI problems under linear noises.In general,the DIEZNN model is an innovative work and is proposed for the first time.Satisfyingly,the errors of DIEZNN are always less than 1�10−3 under linear noises,whereas the error norms of OZNN and IEZNN models are not convergent to zero.In addition,these models are applied to the control of the controllable permanent magnet synchronous motor chaotic system to indicate the superiority of the DIEZNN.

Beam based alignment using a neural network

Beams typically do not travel through the magnet centers because of errors in storage rings.The beam deviating from the quadrupole centers is affected by additional dipole fields due to magnetic field feed-down.Beam-based alignment(BBA)is often performed to determine a golden orbit where the beam circulates around the quadrupole center axes.For storage rings with many quadrupoles,the conventional BBA procedure is time-consuming,particularly in the commissioning phase,because of the necessary iterative process.In addition,the conventional BBA method can be affected by strong coupling and the nonlinearity of the storage ring optics.In this study,a novel method based on a neural network was proposed to determine the golden orbit in a much shorter time with reasonable accuracy.This golden orbit can be used directly for operation or adopted as a starting point for conventional BBA.The method was demonstrated in the HLS-II storage ring for the first time through simulations and online experiments.The results of the experiments showed that the golden orbit obtained using this new method was consistent with that obtained using the conventional BBA.The development of this new method and the corresponding experiments are reported in this paper.

Improved training framework in a neural network model for disruption prediction and its application on EXL-50

A neural network model with a classical annotation method has been used on the EXL-50tokamak to predict impending disruption.However,the results revealed issues of overfitting and overconfidence in predictions caused by inaccurate labeling.To mitigate these issues,an improved training framework has been proposed.In this approach,soft labels from previous training serve as teachers to supervise the further learning process;this has lead to a significant improvement in predictive model performance.Notably,this enhancement is primarily attributed to the coupling effect of the soft labels and correction mechanism.This improved training framework introduces an instance-specific label smoothing method,which reflects a more nuanced model assessment on the likelihood of a disruption.It presents a possible solution to effectively address the challenges associated with accurate labeling across different machines.

A Denoiser for Correlated Noise Channel Decoding: Gated-Neural Network

This letter proposes a sliced-gated-convolutional neural network with belief propagation(SGCNN-BP) architecture for decoding long codes under correlated noise. The basic idea of SGCNNBP is using Neural Networks(NN) to transform the correlated noise into white noise, setting up the optimal condition for a standard BP decoder that takes the output from the NN. A gate-controlled neuron is used to regulate information flow and an optional operation—slicing is adopted to reduce parameters and lower training complexity. Simulation results show that SGCNN-BP has much better performance(with the largest gap being 5dB improvement) than a single BP decoder and achieves a nearly 1dB improvement compared to Fully Convolutional Networks(FCN).

Quick Weighing of Passing Vehicles Using the Transfer-Learning-Enhanced Convolutional Neural Network

Transfer learning could reduce the time and resources required by the training of new models and be therefore important for generalized applications of the trainedmachine learning algorithms.In this study,a transfer learningenhanced convolutional neural network(CNN)was proposed to identify the gross weight and the axle weight of moving vehicles on the bridge.The proposed transfer learning-enhanced CNN model was expected to weigh different bridges based on a small amount of training datasets and provide high identification accuracy.First of all,a CNN algorithm for bridge weigh-in-motion(B-WIM)technology was proposed to identify the axle weight and the gross weight of the typical two-axle,three-axle,and five-axle vehicles as they crossed the bridge with different loading routes and speeds.Then,the pre-trained CNN model was transferred by fine-tuning to weigh themoving vehicle on another bridge.Finally,the identification accuracy and the amount of training data required were compared between the two CNN models.Results showed that the pre-trained CNN model using transfer learning for B-WIM technology could be successfully used for the identification of the axle weight and the gross weight for moving vehicles on another bridge while reducing the training data by 63%.Moreover,the recognition accuracy of the pre-trained CNN model using transfer learning was comparable to that of the original model,showing its promising potentials in the actual applications.

Noise-tolerate and adaptive coefficient zeroing neural network for solving dynamic matrix square root

The solving of dynamic matrix square root(DMSR)problems is frequently encountered in many scientific and engineering fields.Although the original zeroing neural network is powerful for solving the DMSR,it cannot vanish the influence of the noise perturbations,and its constant-coefficient design scheme cannot accelerate the convergence speed.Therefore,a noise-tolerate and adaptive coefficient zeroing neural network(NTACZNN)is raised to enhance the robust noise immunity performance and accelerate the conver-gence speed simultaneously.Then,the global convergence and robustness of the pro-posed NTACZNN are theoretically analysed under an ideal environment and noise-perturbed circumstances.Furthermore,some illustrative simulation examples are designed and performed in order to substantiate the efficacy and advantage of the NTACZNN for the DMSR problem solution.Compared with some existing ZNNs,the proposed NTACZNN possesses advanced performance in terms of noise tolerance,solution accuracy,and convergence rate.

Performance of physical-informed neural network (PINN) for the key parameter inference in Langmuir turbulence parameterization scheme

The Stokes production coefficient(E_(6))constitutes a critical parameter within the Mellor-Yamada type(MY-type)Langmuir turbulence(LT)parameterization schemes,significantly affecting the simulation of turbulent kinetic energy,turbulent length scale,and vertical diffusivity coefficient for turbulent kinetic energy in the upper ocean.However,the accurate determination of its value remains a pressing scientific challenge.This study adopted an innovative approach by leveraging deep learning technology to address this challenge of inferring the E_(6).Through the integration of the information of the turbulent length scale equation into a physical-informed neural network(PINN),we achieved an accurate and physically meaningful inference of E_(6).Multiple cases were examined to assess the feasibility of PINN in this task,revealing that under optimal settings,the average mean squared error of the E_(6) inference was only 0.01,attesting to the effectiveness of PINN.The optimal hyperparameter combination was identified using the Tanh activation function,along with a spatiotemporal sampling interval of 1 s and 0.1 m.This resulted in a substantial reduction in the average bias of the E_(6) inference,ranging from O(10^(1))to O(10^(2))times compared with other combinations.This study underscores the potential application of PINN in intricate marine environments,offering a novel and efficient method for optimizing MY-type LT parameterization schemes.

Optimized operation scheme of flash-memory-based neural network online training with ultra-high endurance

With the rapid development of machine learning,the demand for high-efficient computing becomes more and more urgent.To break the bottleneck of the traditional Von Neumann architecture,computing-in-memory(CIM)has attracted increasing attention in recent years.In this work,to provide a feasible CIM solution for the large-scale neural networks(NN)requiring continuous weight updating in online training,a flash-based computing-in-memory with high endurance(10^(9) cycles)and ultrafast programming speed is investigated.On the one hand,the proposed programming scheme of channel hot electron injection(CHEI)and hot hole injection(HHI)demonstrate high linearity,symmetric potentiation,and a depression process,which help to improve the training speed and accuracy.On the other hand,the low-damage programming scheme and memory window(MW)optimizations can suppress cell degradation effectively with improved computing accuracy.Even after 109 cycles,the leakage current(I_(off))of cells remains sub-10pA,ensuring the large-scale computing ability of memory.Further characterizations are done on read disturb to demonstrate its robust reliabilities.By processing CIFAR-10 tasks,it is evident that~90%accuracy can be achieved after 109 cycles in both ResNet50 and VGG16 NN.Our results suggest that flash-based CIM has great potential to overcome the limitations of traditional Von Neumann architectures and enable high-performance NN online training,which pave the way for further development of artificial intelligence(AI)accelerators.