Advancing COVID-19 Diagnosis with CNNs: An Empirical Study of Learning Rates and Optimization Strategies

The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convolutional Neural Networks (CNNs) in the diagnosis of COVID-19 from chest X-ray and CT images, focusing on the impact of varying learning rates and optimization strategies. Despite the abundance of chest X-ray datasets from various institutions, the lack of a dedicated COVID-19 dataset for computational analysis presents a significant challenge. Our work introduces an empirical analysis across four distinct learning rate policies—Cyclic, Step Based, Time-Based, and Epoch Based—each tested with four different optimizers: Adam, Adagrad, RMSprop, and Stochastic Gradient Descent (SGD). The performance of these configurations was evaluated in terms of training and validation accuracy over 100 epochs. Our results demonstrate significant differences in model performance, with the Cyclic learning rate policy combined with SGD optimizer achieving the highest validation accuracy of 83.33%. This study contributes to the existing body of knowledge by outlining effective CNN configurations for COVID-19 image dataset analysis, offering insights into the optimization of machine learning models for the diagnosis of infectious diseases. Our findings underscore the potential of CNNs in supplementing traditional PCR tests, providing a computational approach to identify patterns in chest X-rays and CT scans indicative of COVID-19, thereby aiding in the swift and accurate diagnosis of the virus.

LEARNING RATES OF KERNEL-BASED ROBUST CLASSIFICATION

This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship between the excess misclassification error and the excess generalization error is provided;from this,along with the convex analysis theory,a kind of learning rate is derived.The results show that the performance of the classifier is effected by the outliers,and the extent of impact can be controlled by choosing the homotopy parameters properly.

Adaptive learning rate GMM for moving object detection in outdoor surveillance for sudden illumination changes

A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence of sudden illumination changes.The GMM is mostly used for detecting objects in complex scenes for intelligent monitoring systems.To solve this problem,a mixture Gaussian model has been built for each pixel in the video frame,and according to the scene change from the frame difference,the learning rate of GMM can be dynamically adjusted.The experiments show that the proposed method gives good results with an adaptive GMM learning rate when we compare it with GMM method with a fixed learning rate.The method was tested on a certain dataset,and tests in the case of sudden natural light changes show that our method has a better accuracy and lower false alarm rate.

Assessment of dairy cow feed intake based on BP neural network with polynomial decay learning rate

To overcome the shortcomings of traditional dairy cow feed intake assessment model andBP neural network, this paper proposes a method of optimizing BP neural network usingpolynomial decay learning rate, taking the cow’s body weight, lying duration, lying times,walking steps, foraging duration and concentrate-roughage ratio as input variables andtaking the actual feed intake is the output variable to establish a dairy cow feed intakeassessment model, and the model is trained and verified by experimental data collectedon site. For the sake of comparative study, feed intake is simultaneously assessed by SVRmodel, KNN logistic regression model, traditional BP neural network model, and multilayerBP neural network model. The results show that the established BP model using the polynomial decay learning rate has the highest assessment accuracy, the MSPE, RMSE, MAE,MAPE and R2 are 0.043 kg2/d and 0.208 kg/d, 0.173 kg/d, 1.37% and 0.94 respectively. Compared with SVR model and KNN mode, the RMSE value reduced by 43.9% and 26.5%, it isalso found that the model designed in this paper has many advantages in comparison withthe BP model and multilayer BP model in terms of precision and generalization. Therefore,this method is ready to be applied for accurately evaluating the dairy cow feed intake, andit can provide theoretical guidance and technical support for the precise-feeding and canalso be of high significance in the improvement of dairy precise-breeding.

The Learning Rate of l_2-Coefcient Regularized Classifcation with Strong Loss

In the present paper, we give an investigation on the learning rate of l2-coefficient regularized classification with strong loss and the data dependent kernel functional spaces. The results show that the learning rate is influenced by the strong convexity.

Tuning the Learning Rate for Stochastic Variational Inference

Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback-Leibler （KL） divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models： latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.

Neurocomputing van der Pauw function for the measurement of a semiconductor's resistivity without use of the learning rate of weight vector regulation

Van der Pauw＇s function is often used in the measurement of a semiconductor＇s resistivity. However, it is difficult to obtain its value from voltage measurements because it has an implicit form. If it can be expressed as a polynomial, a semiconductor＇s resistivity can be obtained from such measurements. Normally, five orders of the abscissa can provide sufficient precision during the expression of any non-linear function. Therefore, the key is to determine the coefficients of the polynomial. By taking five coefficients as weights to construct a neuronetwork, neurocomputing has been used to solve this problem. Finally, the polynomial expression for van der Pauw＇s function is obtained.

Learning sparse and smooth functions by deep Sigmoid nets

To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.

Data Fusion Architecture Empowered with Deep Learning for Breast Cancer Classification

Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection and therapy can help women receive effective treatment and,as a result,decrease the rate of breast cancer disease.The cancer tumor develops when cells grow improperly and attack the healthy tissue in the human body.Tumors are classified as benign or malignant,and the absence of cancer in the breast is considered normal.Deep learning,machine learning,and transfer learning models are applied to detect and identify cancerous tissue like BC.This research assists in the identification and classification of BC.We implemented the pre-trained model AlexNet and proposed model Breast cancer identification and classification(BCIC),which are machine learning-based models,by evaluating them in the form of comparative research.We used 3 datasets,A,B,and C.We fuzzed these datasets and got 2 datasets,A2C and B3C.Dataset A2C is the fusion of A,B,and C with 2 classes categorized as benign and malignant.Dataset B3C is the fusion of datasets A,B,and C with 3 classes classified as benign,malignant,and normal.We used customized AlexNet according to our datasets and BCIC in our proposed model.We achieved an accuracy of 86.5%on Dataset B3C and 76.8%on Dataset A2C by using AlexNet,and we achieved the optimum accuracy of 94.5%on Dataset B3C and 94.9%on Dataset A2C by using proposed model BCIC at 40 epochs with 0.00008 learning rate.We proposed fuzzed dataset model using transfer learning.We fuzzed three datasets to get more accurate results and the proposed model achieved the highest prediction accuracy using fuzzed dataset transfer learning technique.

Investigation on the Chinese Text Sentiment Analysis Based on Convolutional Neural Networks in Deep Learning

Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is a crucial task in the web monitoring area.The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data.Deep learning is a hot research topic of the artificial intelligence in the recent years.By now,several research groups have studied the sentiment analysis of English texts using deep learning methods.In contrary,relatively few works have so far considered the Chinese text sentiment analysis toward this direction.In this paper,a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network(CNN)in deep learning in order to improve the analysis accuracy.The feature values of the CNN after the training process are nonuniformly distributed.In order to overcome this problem,a method for normalizing the feature values is proposed.Moreover,the dimensions of the text features are optimized through simulations.Finally,a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances.Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods,e.g.,the support vector machine method.

Learning from regularized regression algorithms with p-order Markov chain sampling

Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically distributed（i.i.d.） samples has been researched by a large number of references. In the present paper we provide the convergence rates for the performance of regularized regression based on the inputs of p-order Markov chains.

Approximating and learning by Lipschitz kernel on the sphere

This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, is obtained, where n is the number of parameters needed in the approximation. By means of the approximation, a learning rate of regularized least square algorithm with the Lipschitz kernel on the sphere is also deduced.

Hyperparameter Tuning for Deep Neural Networks Based Optimization Algorithm

For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.

A Review of Three Misconceptions about Age and L2 Learning(Marinova‐Todd,et al.,2000)and Comments on Stefka H.Marinova-Todd,D.Bradford Marshall,and Catherine E.Snow’s“Three Misconceptions About Age and L2 Learning.”(Hyltenstam&Abrahamsson,2001)

Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 learning.Hyltenstam&Abrahamsson refutes Marinova‐Todd’s“three misconceptions”.By contrasting and comparing the views and evidence from the two papers,the present author analyzes the differences among various claims on the rate of learning and different brain organization,as well as the possible causes for the disagreements.

On the Feasibility of Early-age English learning

Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has explored the Critical Period Hypothesis and discussed the younger learners' disadvantages and older learners' advantages when learning English. and concludes that early-age English learning is not feasible.

Health-related fitness knowledge growth in middle school years:Individual-and school-level correlates

Background:Health-related fitness knowledge(HRFK)has been an essential concept for many health and physical education programs.There has been limited understanding and longitudinal investigation on HRFK growth.This longitudinal study examined HRFK growth and its individual-and school-level correlates in middle school years under 1 curriculum condition:Five for Life.Methods:Participants were 12,044 students from 47 middle schools.Data were collected at both individual/participant and school/institution levels.Individual-level variables included gender,grade,and HRFK test scores.School-level variables included percentage of students receiving free and reduced meals(FARM),student-to-faculty ratio for physical education,and school academic performance(SAP).We used hierarchical linear modeling to examine HRFK 3-year growth in relation to individual-and school-level correlates.Results:The average HRFK score at 6th grade for females was 42.81%±1.32%.The predicted HRFK growth was 17.06%±1.02%per year,holding other factors constant.A 1-standard deviation increase in FARM correlated with a 14.68%-point decrease in predicted test score(p=0.02).A 1-standard deviation increase in SAP was associated with an 11.90%-point increase in HRFK score.Males had a significantly lower growth rate than females during the middle school years(0.78%/year,p=0.02).Conclusion:The result showed that both individual-and school-level variables such as gender,FARM,and SAP influenced HRFK growth.Educators should heed gender differences in growth curves and recognize the correlates of school-level variables.

An Optimized Deep Residual Network with a Depth Concatenated Block for Handwritten Characters Classification

Even though much advancements have been achieved with regards to the recognition of handwritten characters,researchers still face difficulties with the handwritten character recognition problem,especially with the advent of new datasets like the Extended Modified National Institute of Standards and Technology dataset(EMNIST).The EMNIST dataset represents a challenge for both machine-learning and deep-learning techniques due to inter-class similarity and intra-class variability.Inter-class similarity exists because of the similarity between the shapes of certain characters in the dataset.The presence of intra-class variability is mainly due to different shapes written by different writers for the same character.In this research,we have optimized a deep residual network to achieve higher accuracy vs.the published state-of-the-art results.This approach is mainly based on the prebuilt deep residual network model ResNet18,whose architecture has been enhanced by using the optimal number of residual blocks and the optimal size of the receptive field of the first convolutional filter,the replacement of the first max-pooling filter by an average pooling filter,and the addition of a drop-out layer before the fully connected layer.A distinctive modification has been introduced by replacing the final addition layer with a depth concatenation layer,which resulted in a novel deep architecture having higher accuracy vs.the pure residual architecture.Moreover,the dataset images’sizes have been adjusted to optimize their visibility in the network.Finally,by tuning the training hyperparameters and using rotation and shear augmentations,the proposed model outperformed the state-of-the-art models by achieving average accuracies of 95.91%and 90.90%for the Letters and Balanced dataset sections,respectively.Furthermore,the average accuracies were improved to 95.9%and 91.06%for the Letters and Balanced sections,respectively,by using a group of 5 instances of the trained models and averaging the output class probabilities.

The adaptive control using BP neural networks for a nonlinear servo-motor

The servo-motor possesses a strongly nonlinear property due to the effect of the stimulating input voltage, load-torque and environmental operating conditions. So it is rather difficult to derive a traditional mathematical model which is capable of expressing both its dynamics and steady-state characteristics. A neural network-based adaptive control strategy is proposed in this paper. In this method, two neural networks have been adopted for system identification （NNI） and control （NNC）, respectively. Then, the commonly-used specialized learning has been modified, by taking the NNI output as the approximation output of the servo-motor during the weights training to get sensitivity information. Moreover, the rule for choosing the learning rate is given on the basis of the analysis of Lyapunov stability. Finally, an example of applying the proposed control strategy on a servo-motor is presented to show its effectiveness.

Adaptive Generalized Eigenvector Estimating Algorithm for Hermitian Matrix Pencil

Generalized eigenvector plays an essential role in the signal processing field.In this paper,we present a novel neural network learning algorithm for estimating the generalized eigenvector of a Hermitian matrix pencil.Differently from some traditional algorithms,which need to select the proper values of learning rates before using,the proposed algorithm does not need a learning rate and is very suitable for real applications.Through analyzing all of the equilibrium points,it is proven that if and only if the weight vector of the neural network is equal to the generalized eigenvector corresponding to the largest generalized eigenvalue of a Hermitian matrix pencil,the proposed algorithm reaches to convergence status.By using the deterministic discretetime(DDT)method,some convergence conditions,which can be satisfied with probability 1,are also obtained to guarantee its convergence.Simulation results show that the proposed algorithm has a fast convergence speed and good numerical stability.The real application demonstrates its effectiveness in tracking the optimal vector of beamforming.

Study on Speeding up the Back-Propagation Algorithm

Slow convergence of back-propagation (BP) algorithm is a limiting factor in its practical applications. A new learning algorithm which can adaptively adjust its learning rate on the basis of gradient information of the error function is put forward. its convergence performance is also tested by the XOR problem compared with the standard BP algorithm.