Evolution and Role of Optimizers in Training Deep Learning Models

TO perform well,deep learning(DL)models have to be trained well.Which optimizer should be adopted?We answer this question by discussing how optimizers have evolved from traditional methods like gradient descent to more advanced techniques to address challenges posed by highdimensional and non-convex problem space.Ongoing challenges include their hyperparameter sensitivity,balancing between convergence and generalization performance,and improving interpretability of optimization processes.Researchers continue to seek robust,efficient,and universally applicable optimizers to advance the field of DL across various domains.

Pruning method for dendritic neuron model based on dendrite layer significance constraints

The dendritic neural model(DNM)mimics the non-linearity of synapses in the human brain to simulate the information processing mechanisms and procedures of neurons.This enhances the understanding of biological nervous systems and the applicability of the model in various fields.However,the existing DNM suffers from high complexity and limited generalisation capability.To address these issues,a DNM pruning method with dendrite layer significance constraints is proposed.This method not only evaluates the significance of dendrite layers but also allocates the significance of a few dendrite layers in the trained model to a few dendrite layers,allowing the removal of low-significance dendrite layers.The simulation experiments on six UCI datasets demonstrate that our method surpasses existing pruning methods in terms of network size and generalisation performance.

Pleniglacial millennium-scale climate variations in northern China based on records from the Salawusu River Valley

Situated in the Salawusu River Valley, southeast of China＇s Mu Us Desert, the MGS2 （Milanggouwan section） portion of the Milanggouwan stratigraphic section records 5.5 sedimentary cycles consisting of alternations between dune sand deposits and fluvial or lacustrine facies. We analyzed the grain-size and CaCO3 distributions in MGS2, and found that Mz （mean particle diameter） and o （standard deviation） displayed clear variations in peaks and valleys within different sedimentary facies. The CaCO3 content averaged 0.4% in the dune sand deposits, 1.43% in the fluvial facies, and 8.82% in the lacustrine facies. Both the grain-size distribution and CaCO3 contents, which equal the indicators for the alternation among the sedimentary facies, suggest the occurrence of 5.5 cycles. These results suggest that the observed cycles mainly resulted from fluctuations between a cold and dry winter monsoon climate and a warm and humid summer monsoon climate, and that the MGS2 portion experienced at least 5.5 fluctuations between these two extremes. This high-frequency climatic fluctuation indicates a strong influence of millennium-scale variations in the strength of the East Asian winter and summer monsoons in our study area during the Pleniglacial.