Artificial Neural Network Modeling for Predicting Thermal Conductivity of EG/Water-Based CNC Nanofluid for Engine Cooling Using Different Activation Functions

A vehicle engine cooling system is of utmost importance to ensure that the engine operates in a safe temperature range.In most radiators that are used to cool an engine,water serves as a cooling fluid.The performance of a radiator in terms of heat transmission is significantly influenced by the incorporation of nanoparticles into the cooling water.Concentration and uniformity of nanoparticle distribution are the two major factors for the practical use of nanofluids.The shape and size of nanoparticles also have a great impact on the performance of heat transfer.Many researchers are investigating the impact of nanoparticles on heat transfer.This study aims to develop an artificial neural network(ANN)model for predicting the thermal conductivity of an ethylene glycol(EG)/waterbased crystalline nanocellulose(CNC)nanofluid for cooling internal combustion engine.The implementation of an artificial neural network considering different activation functions in the hidden layer is made to find the bestmodel for the cooling of an engine using the nanofluid.Accuracies of the model with different activation functions in artificial neural networks are analyzed for different nanofluid concentrations and temperatures.In artificial neural networks,Levenberg–Marquardt is an optimization approach used with activation functions,including Tansig and Logsig functions in the training phase.The findings of each training,testing,and validation phase are presented to demonstrate the network that provides the highest level of accuracy.The best result was obtained with Tansig,which has a correlation of 0.99903 and an error of 3.7959×10^(–8).It has also been noticed that the Logsig function can also be a good model due to its correlation of 0.99890 and an error of 4.9218×10^(–8).Thus ourANNwith Tansig and Logsig functions demonstrates a high correlation between the actual output and the predicted output.

Fusion of Activation Functions: An Alternative to Improving Prediction Accuracy in Artificial Neural Networks

The purpose of this study was to address the challenges in predicting and classifying accuracy in modeling Container Dwell Time (CDT) using Artificial Neural Networks (ANN). This objective was driven by the suboptimal outcomes reported in previous studies and sought to apply an innovative approach to improve these results. To achieve this, the study applied the Fusion of Activation Functions (FAFs) to a substantial dataset. This dataset included 307,594 container records from the Port of Tema from 2014 to 2022, encompassing both import and transit containers. The RandomizedSearchCV algorithm from Python’s Scikit-learn library was utilized in the methodological approach to yield the optimal activation function for prediction accuracy. The results indicated that “ajaLT”, a fusion of the Logistic and Hyperbolic Tangent Activation Functions, provided the best prediction accuracy, reaching a high of 82%. Despite these encouraging findings, it’s crucial to recognize the study’s limitations. While Fusion of Activation Functions is a promising method, further evaluation is necessary across different container types and port operations to ascertain the broader applicability and generalizability of these findings. The original value of this study lies in its innovative application of FAFs to CDT. Unlike previous studies, this research evaluates the method based on prediction accuracy rather than training time. It opens new avenues for machine learning engineers and researchers in applying FAFs to enhance prediction accuracy in CDT modeling, contributing to a previously underexplored area.

A Universal Activation Function for Deep Learning

Recently,deep learning has achieved remarkable results in fields that require human cognitive ability,learning ability,and reasoning ability.Activation functions are very important because they provide the ability of artificial neural networks to learn complex patterns through nonlinearity.Various activation functions are being studied to solve problems such as vanishing gradients and dying nodes that may occur in the deep learning process.However,it takes a lot of time and effort for researchers to use the existing activation function in their research.Therefore,in this paper,we propose a universal activation function(UA)so that researchers can easily create and apply various activation functions and improve the performance of neural networks.UA can generate new types of activation functions as well as functions like traditional activation functions by properly adjusting three hyperparameters.The famous Convolutional Neural Network(CNN)and benchmark datasetwere used to evaluate the experimental performance of the UA proposed in this study.We compared the performance of the artificial neural network to which the traditional activation function is applied and the artificial neural network to which theUA is applied.In addition,we evaluated the performance of the new activation function generated by adjusting the hyperparameters of theUA.The experimental performance evaluation results showed that the classification performance of CNNs improved by up to 5%through the UA,although most of them showed similar performance to the traditional activation function.

Multistability of delayed complex-valued recurrent neural networks with discontinuous real-imaginarytype activation functions

In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results.

Activation Functions Effect on Fractal Coding Using Neural Networks

Activation functions play an essential role in converting the output of the artificial neural network into nonlinear results,since without this nonlinearity,the results of the network will be less accurate.Nonlinearity is the mission of all nonlinear functions,except for polynomials.The activation function must be dif-ferentiable for backpropagation learning.This study’s objective is to determine the best activation functions for the approximation of each fractal image.Different results have been attained using Matlab and Visual Basic programs,which indi-cate that the bounded function is more helpful than other functions.The non-lin-earity of the activation function is important when using neural networks for coding fractal images because the coefficients of the Iterated Function System are different according to the different types of fractals.The most commonly cho-sen activation function is the sigmoidal function,which produces a positive value.Other functions,such as tansh or arctan,whose values can be positive or negative depending on the network input,tend to train neural networks faster.The coding speed of the fractal image is different depending on the appropriate activation function chosen for each fractal shape.In this paper,we have provided the appro-priate activation functions for each type of system of iterated functions that help the network to identify the transactions of the system.

Neural Networks on an FPGA and Hardware-Friendly Activation Functions

This paper describes our implementation of several neural networks built on a field programmable gate array (FPGA) and used to recognize a handwritten digit dataset—the Modified National Institute of Standards and Technology (MNIST) database. We also propose a novel hardware-friendly activation function called the dynamic Rectifid Linear Unit (ReLU)—D-ReLU function that achieves higher performance than traditional activation functions at no cost to accuracy. We built a 2-layer online training multilayer perceptron (MLP) neural network on an FPGA with varying data width. Reducing the data width from 8 to 4 bits only reduces prediction accuracy by 11%, but the FPGA area decreases by 41%. Compared to networks that use the sigmoid functions, our proposed D-ReLU function uses 24% - 41% less area with no loss to prediction accuracy. Further reducing the data width of the 3-layer networks from 8 to 4 bits, the prediction accuracies only decrease by 3% - 5%, with area being reduced by 9% - 28%. Moreover, FPGA solutions have 29 times faster execution time, even despite running at a 60× lower clock rate. Thus, FPGA implementations of neural networks offer a high-performance, low power alternative to traditional software methods, and our novel D-ReLU activation function offers additional improvements to performance and power saving.

H_(∞)/Passive Synchronization of Semi-Markov Jump Neural Networks Subject to Hybrid Attacks via an Activation Function Division Approach

In this work,an H_(∞)/passive-based secure synchronization control problem is investigated for continuous-time semi-Markov neural networks subject to hybrid attacks,in which hybrid attacks are the combinations of denial-of-service attacks and deception attacks,and they are described by two groups of independent Bernoulli distributions.On this foundation,via the Lyapunov stability theory and linear matrix inequality technology,the H_(∞)/passive-based performance criteria for semi-Markov jump neural networks are obtained.Additionally,an activation function division approach for neural networks is adopted to further reduce the conservatism of the criteria.Finally,a simulation example is provided to verify the validity and feasibility of the proposed method.

GAAF:Searching Activation Functions for Binary Neural Networks Through Genetic Algorithm

Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.

Performance analysis of ASR system in hybrid DNN-HMM framework using a PWL euclidean activation function

Automatic Speech Recognition(ASR)is the process of mapping an acoustic speech signal into a human readable text format.Traditional systems exploit the Acoustic Component of ASR using the Gaussian Mixture Model-Hidden Markov Model(GMM-HMM)approach.Deep NeuralNetwork(DNN)opens up new possibilities to overcome the shortcomings of conventional statistical algorithms.Recent studies modeled the acoustic component of ASR system using DNN in the so called hybrid DNN-HMM approach.In the context of activation functions used to model the non-linearity in DNN,Rectified Linear Units(ReLU)and maxout units are mostly used in ASR systems.This paper concentrates on the acoustic component of a hybrid DNN-HMM system by proposing an efficient activation function for the DNN network.Inspired by previous works,euclidean norm activation function is proposed to model the non-linearity of the DNN network.Such non-linearity is shown to belong to the family of Piecewise Linear(PWL)functions having distinct features.These functions can capture deep hierarchical features of the pattern.The relevance of the proposal is examined in depth both theoretically and experimentally.The performance of the developed ASR system is evaluated in terms of Phone Error Rate(PER)using TIMIT database.Experimental results achieve a relative increase in performance by using the proposed function over conventional activation functions.

Global Exponential Periodicity of a Class of Recurrent Neural Networks with Non-Monotone Activation Functions and Time-Varying Delays

The stability of a periodic oscillation and the global exponential class of recurrent neural networks with non-monotone activation functions and time-varying delays are analyzed. For two sets of activation functions, some algebraic criteria for ascertaining global exponential periodicity and global exponential stability of the class of recurrent neural networks are derived by using the comparison principle and the theory of monotone operator. These conditions are easy to check in terms of system parameters. In addition, we provide a new and efficacious method for the qualitative analysis of various neural networks.

Learning Specialized Activation Functions for Physics-Informed Neural Networks

Physics-informed neural networks(PINNs)are known to suffer from optimization difficulty.In this work,we reveal the connection between the optimization difficulty of PINNs and activation functions.Specifically,we show that PINNs exhibit high sensitivity to activation functions when solving PDEs with distinct properties.Existing works usually choose activation functions by inefficient trial-and-error.To avoid the inefficient manual selection and to alleviate the optimization difficulty of PINNs,we introduce adaptive activation functions to search for the optimal function when solving different problems.We compare different adaptive activation functions and discuss their limitations in the context of PINNs.Furthermore,we propose to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization,which has a higher requirement for the smoothness and diversity on learned functions.This is achieved by removing activation functions which cannot provide higher-order derivatives from the candidate set and incorporating elementary functions with different properties according to our prior knowledge about the PDE at hand.We further enhance the search space with adaptive slopes.The proposed adaptive activation function can be used to solve different PDE systems in an interpretable way.Its effectiveness is demonstrated on a series of benchmarks.Code is available at https://github.com/LeapLabTHU/AdaAFforPINNs.

Robust stability of mixed Cohen–Grossberg neural networks with discontinuous activation functions

Purpose–The purpose of this paper is to develop a method for the existence,uniqueness and globally robust stability of the equilibrium point for Cohen–Grossberg neural networks with time-varying delays,continuous distributed delays and a kind of discontinuous activation functions.Design/methodology/approach–Basedonthe Leray–Schauderalternativetheoremand chainrule,by using a novel integral inequality dealing with monotone non-decreasing function,the authors obtain a delay-dependent sufficient condition with less conservativeness for robust stability of considered neural networks.Findings–Itturns out thattheauthors’delay-dependent sufficientcondition canbeformed intermsof linear matrix inequalities conditions.Two examples show the effectiveness of the obtained results.Originality/value–The novelty of the proposed approach lies in dealing with a new kind of discontinuous activation functions by using the Leray–Schauder alternative theorem,chain rule and a novel integral inequality on monotone non-decreasing function.

General Decay Synchronization of Competitive Fuzzy Neural Networks Involving Time Delays and Right-Hand Discontinuous Activation

This paper discusses the general decay synchronization problem for a class of fuzzy competitive neural networks with time-varying delays and discontinuous activation functions. Firstly, based on the concept of Filippov solutions for right-hand discontinuous systems, some sufficient conditions for general decay synchronization of the considered system are obtained via designing a nonlinear feedback controller and applying discontinuous differential equation theory, Lyapunov functional methods and some inequality techniques. Finally, one numerical example is given to verify the effectiveness of the proposed theoretical results. The general decay synchronization considered in this article can better estimate the convergence rate of the system, and the exponential synchronization and polynomial synchronization can be seen as its special cases.

Elevated brain temperature under severe heat exposure impairs cortical motor activity and executive function

Background:Excessive heat exposure can lead to hyperthermia in humans,which impairs physical performance and disrupts cognitive function.While heat is a known physiological stressor,it is unclear how severe heat stress affects brain physiology and function.Methods:Eleven healthy participants were subjected to heat stress from prolonged exercise or warm water immersion until their rectal temperatures(T_(re))attained 39.5℃,inducing exertional or passive hyperthermia,respectively.In a separate trial,blended ice was ingested before and during exercise as a cooling strategy.Data were compared to a control condition with seated rest(normothermic).Brain temperature(T_(br)),cerebral perfusion,and task-based brain activity were assessed using magnetic resonance imaging techniques.Results:T_(br)in motor cortex was found to be tightly regulated at rest(37.3℃±0.4℃(mean±SD))despite fluctuations in T_(re).With the development of hyperthermia,T_(br)increases and dovetails with the rising T_(re).Bilateral motor cortical activity was suppressed during high-intensity plantarflexion tasks,implying a reduced central motor drive in hyperthermic participants(T_(re)=38.5℃±0.1℃).Global gray matter perfusion and regional perfusion in sensorimotor cortex were reduced with passive hyperthermia.Executive function was poorer under a passive hyperthermic state,and this could relate to compromised visual processing as indicated by the reduced activation of left lateral-occipital cortex.Conversely,ingestion of blended ice before and during exercise alleviated the rise in both T_(re)and T_(bc)and mitigated heat-related neural perturbations.Conclusion:Severe heat exposure elevates T_(br),disrupts motor cortical activity and executive function,and this can lead to impairment of physical and cognitive performance.

Theoretical Study on the C-H Activation in Decarbonylation of Acetaldehyde by NiL_2(L=SO_3CH_3) Using Density Functional Theory

Density functional theory calculations were carried out to explore the potential energy surface（PES） associated with the gas-phase reaction of Ni L2（L=SO3CH3） with acetone. The geometries and energies of the reactants, intermediates, products and transition states of the triplet ground potential energy surfaces of [Ni, O, C2, H4] were obtained at the B3LYP/6-311＋＋G（d,p） levels in C,H,O atoms and B3LYP/ Lanl2 dz in Ni atom. It was found through our calculations that the decabonylation of acetaldehyde contains four steps including encounter complexation, C-C activation, aldehyde H-shift and nonreactive dissociation. The results revealed that C-C activation induced by Ni L2（L=SO3CH3） led to the decarbonylation of acetaldehyde.

Convolution-Based Heterogeneous Activation Facility for Effective Machine Learning of ECG Signals

Machine Learning(ML)and Deep Learning(DL)technologies are revolutionizing the medical domain,especially with Electrocardiogram(ECG),by providing new tools and techniques for diagnosing,treating,and preventing diseases.However,DL architectures are computationally more demanding.In recent years,researchers have focused on combining the computationally less intensive portion of the DL architectures with ML approaches,say for example,combining the convolutional layer blocks of Convolution Neural Networks(CNNs)into ML algorithms such as Extreme Gradient Boosting(XGBoost)and K-Nearest Neighbor(KNN)resulting in CNN-XGBoost and CNN-KNN,respectively.However,these approaches are homogenous in the sense that they use a fixed Activation Function(AFs)in the sequence of convolution and pooling layers,thereby limiting the ability to capture unique features.Since various AFs are readily available and each could capture unique features,we propose a Convolutionbased Heterogeneous Activation Facility(CHAF)which uses multiple AFs in the convolution layer blocks,one for each block,with a motivation of extracting features in a better manner to improve the accuracy.The proposed CHAF approach is validated on PTB and shown to outperform the homogeneous approaches such as CNN-KNN and CNN-XGBoost.For PTB dataset,proposed CHAF-KNN has an accuracy of 99.55%and an F1 score of 99.68%in just 0.008 s,outperforming the state-of-the-art CNN-XGBoost which has an accuracy of 99.38%and an F1 score of 99.32%in 1.23 s.To validate the generality of the proposed CHAF,experiments were repeated on MIT-BIH dataset,and the proposed CHAF-KNN is shown to outperform CNN-KNN and CNN-XGBoost.

Microbial community structure and functional metabolic diversity are associated with organic carbon availability in an agricultural soil

Exploration of soil environmental characteristics governing soil microbial community structure and activity may improve our understanding of biogeochemical processes and soil quality. The impact of soil environmental characteristics especially organic carbon availability after 15-yr different organic and inorganic fertilizer inputs on soil bacterial community structure and functional metabolic diversity of soil microbial communities were evaluated in a 15-yr fertilizer experiment in Changping County, Beijing, China. The experiment was a wheat-maize rotation system which was established in 1991 including four different fertilizer treatments. These treatments included： a non-amended control（CK）, a commonly used application rate of inorganic fertilizer treatment（NPK）; a commonly used application rate of inorganic fertilizer with swine manure incorporated treatment（NPKM）, and a commonly used application rate of inorganic fertilizer with maize straw incorporated treatment（NPKS）. Denaturing gradient gel electrophoresis（DGGE） of the 16 S r RNA gene was used to determine the bacterial community structure and single carbon source utilization profiles were determined to characterize the microbial community functional metabolic diversity of different fertilizer treatments using Biolog Eco plates. The results indicated that long-term fertilized treatments significantly increased soil bacterial community structure compared to CK. The use of inorganic fertilizer with organic amendments incorporated for long term（NPKM, NPKS） significantly promoted soil bacterial structure than the application of inorganic fertilizer only（NPK）, and NPKM treatment was the most important driver for increases in the soil microbial community richness（S） and structural diversity（H）. Overall utilization of carbon sources by soil microbial communities（average well color development, AWCD） and microbial substrate utilization diversity and evenness indices（H＇ and E） indicated that long-term inorganic fertilizer with organic amendments incorporated（NPKM, NPKS） could significantly stimulate soil microbial metabolic activity and functional diversity relative to CK, while no differences of them were found between NPKS and NPK treatments. Principal component analysis（PCA） based on carbon source utilization profiles also showed significant separation of soil microbial community under long-term fertilization regimes and NPKM treatment was significantly separated from the other three treatments primarily according to the higher microbial utilization of carbohydrates, carboxylic acids, polymers, phenolic compounds, and amino acid, while higher utilization of amines/amides differed soil microbial community in NPKS treatment from those in the other three treatments. Redundancy analysis（RDA） indicated that soil organic carbon（SOC） availability, especially soil microbial biomass carbon（Cmic） and Cmic/SOC ratio are the key factors of soil environmental characteristics contributing to the increase of both soil microbial community structure and functional metabolic diversity in the long-term fertilization trial. Our results showed that long-term inorganic fertilizer and swine manure application could significantly improve soil bacterial community structure and soil microbial metabolic activity through the increases in SOC availability, which could provide insights into the sustainable management of China＇s soil resource.

Altered intrinsic functional connectivity of the primary visual cortex in youth patients with comitant exotropia: a resting state fMRI study

AIM： To evaluate the differences in the functional connectivity（FC） of the primary visual cortex（V1） between the youth comitant exotropia（CE） patients and health subjects using resting functional magnetic resonance imaging（f MRI） data.METHODS： Totally, 32 CEs（25 males and 7 females） and 32 healthy control subjects（HCs）（25 males and 7 females） were enrolled in the study and underwent the MRI scanning. Two-sample t-test was used to examine differences in FC maps between the CE patients and HCs. RESULTS： The CE patients showed significantly less FC between the left brodmann area（BA17） and left lingual gyrus/cerebellum posterior lobe, right middle occipital gyrus, left precentral gyrus/postcentral gyrus and right inferior parietal lobule/postcentral gyrus. Meanwhile, CE patients showed significantly less FC between right BA17 and right middle occipital gyrus（BA19, 37）.CONCLUSION： Our findings show that CE involves abnormal FC in primary visual cortex in many regions, which may underlie the pathologic mechanism of impaired fusion and stereoscopic vision in CEs.

The cortical activation pattern during bilateral arm raising movements

Bilateral arm raising movements have been used in brain rehabilitation for a long time. However, no study has been reported on the effect of these movements on the cerebral cortex. In this study, using functional near infrared spectroscopy（f NIRS）, we attempted to investigate cortical activation generated during bilateral arm raising movements. Ten normal subjects were recruited for this study. f NIRS was performed using an f NIRS system with 49 channels. Bilateral arm raising movements were performed in sitting position at the rate of 0.5 Hz. We measured values of oxyhemoglobin and total hemoglobin in five regions of interest： the primary sensorimotor cortex, premotor cortex, supplementary motor area, prefrontal cortex, and posterior parietal cortex. During performance of bilateral arm raising movements, oxyhemoglobin and total hemoglobin values in the primary sensorimotor cortex, premotor cortex, supplementary motor area, and prefrontal cortex were similar, but higher in these regions than those in the prefrontal cortex. We observed activation of the arm somatotopic areas of the primary sensorimotor cortex and premotor cortex in both hemispheres during bilateral arm raising movements. According to this result, bilateral arm raising movements appeared to induce large-scale neuronal activation and therefore arm raising movements would be good exercise for recovery of brain functions.

Effect of Preparation Process of Functional Polymer Active Materials on Properties of Gadolinium Ion Selective Electrode

Recently we have studied the rare earth ion-selective electrodes with active materials of the func-tional polymers and found that the process chosen for the functional polymers had an effect on the propertiesof gadolinium ion selective electrode besides the effects of their structures.1.Effect of preparation process of the grafted polymers on the properties ofgadolinium ion selective electrodesThe electrode membranes which consist of functional polymers as active materials were prepared by re-action of gadolinium chloride with the radiation grafted clmer of acrlic acid and polystyrene of which