Assessments of Data-Driven Deep Learning Models on One-Month Predictions of Pan-Arctic Sea Ice Thickness

In recent years,deep learning methods have gradually been applied to prediction tasks related to Arctic sea ice concentration,but relatively little research has been conducted for larger spatial and temporal scales,mainly due to the limited time coverage of observations and reanalysis data.Meanwhile,deep learning predictions of sea ice thickness(SIT)have yet to receive ample attention.In this study,two data-driven deep learning(DL)models are built based on the ConvLSTM and fully convolutional U-net(FC-Unet)algorithms and trained using CMIP6 historical simulations for transfer learning and fine-tuned using reanalysis/observations.These models enable monthly predictions of Arctic SIT without considering the complex physical processes involved.Through comprehensive assessments of prediction skills by season and region,the results suggest that using a broader set of CMIP6 data for transfer learning,as well as incorporating multiple climate variables as predictors,contribute to better prediction results,although both DL models can effectively predict the spatiotemporal features of SIT anomalies.Regarding the predicted SIT anomalies of the FC-Unet model,the spatial correlations with reanalysis reach an average level of 89%over all months,while the temporal anomaly correlation coefficients are close to unity in most cases.The models also demonstrate robust performances in predicting SIT and SIE during extreme events.The effectiveness and reliability of the proposed deep transfer learning models in predicting Arctic SIT can facilitate more accurate pan-Arctic predictions,aiding climate change research and real-time business applications.

A game-theoretic approach for federated learning:A trade-off among privacy,accuracy and energy

Benefiting from the development of Federated Learning(FL)and distributed communication systems,large-scale intelligent applications become possible.Distributed devices not only provide adequate training data,but also cause privacy leakage and energy consumption.How to optimize the energy consumption in distributed communication systems,while ensuring the privacy of users and model accuracy,has become an urgent challenge.In this paper,we define the FL as a 3-layer architecture including users,agents and server.In order to find a balance among model training accuracy,privacy-preserving effect,and energy consumption,we design the training process of FL as game models.We use an extensive game tree to analyze the key elements that influence the players’decisions in the single game,and then find the incentive mechanism that meet the social norms through the repeated game.The experimental results show that the Nash equilibrium we obtained satisfies the laws of reality,and the proposed incentive mechanism can also promote users to submit high-quality data in FL.Following the multiple rounds of play,the incentive mechanism can help all players find the optimal strategies for energy,privacy,and accuracy of FL in distributed communication systems.

Knowledge-reused transfer learning for molecular and materials science

Leveraging big data analytics and advanced algorithms to accelerate and optimize the process of molecular and materials design, synthesis, and application has revolutionized the field of molecular and materials science, allowing researchers to gain a deeper understanding of material properties and behaviors,leading to the development of new materials that are more efficient and reliable. However, the difficulty in constructing large-scale datasets of new molecules/materials due to the high cost of data acquisition and annotation limits the development of conventional machine learning(ML) approaches. Knowledgereused transfer learning(TL) methods are expected to break this dilemma. The application of TL lowers the data requirements for model training, which makes TL stand out in researches addressing data quality issues. In this review, we summarize recent progress in TL related to molecular and materials. We focus on the application of TL methods for the discovery of advanced molecules/materials, particularly, the construction of TL frameworks for different systems, and how TL can enhance the performance of models. In addition, the challenges of TL are also discussed.

Identification of Software Bugs by Analyzing Natural Language-Based Requirements Using Optimized Deep Learning Features

Software project outcomes heavily depend on natural language requirements,often causing diverse interpretations and issues like ambiguities and incomplete or faulty requirements.Researchers are exploring machine learning to predict software bugs,but a more precise and general approach is needed.Accurate bug prediction is crucial for software evolution and user training,prompting an investigation into deep and ensemble learning methods.However,these studies are not generalized and efficient when extended to other datasets.Therefore,this paper proposed a hybrid approach combining multiple techniques to explore their effectiveness on bug identification problems.The methods involved feature selection,which is used to reduce the dimensionality and redundancy of features and select only the relevant ones;transfer learning is used to train and test the model on different datasets to analyze how much of the learning is passed to other datasets,and ensemble method is utilized to explore the increase in performance upon combining multiple classifiers in a model.Four National Aeronautics and Space Administration(NASA)and four Promise datasets are used in the study,showing an increase in the model’s performance by providing better Area Under the Receiver Operating Characteristic Curve(AUC-ROC)values when different classifiers were combined.It reveals that using an amalgam of techniques such as those used in this study,feature selection,transfer learning,and ensemble methods prove helpful in optimizing the software bug prediction models and providing high-performing,useful end mode.

Position-Aware and Subgraph Enhanced Dynamic Graph Contrastive Learning on Discrete-Time Dynamic Graph

Unsupervised learning methods such as graph contrastive learning have been used for dynamic graph represen-tation learning to eliminate the dependence of labels.However,existing studies neglect positional information when learning discrete snapshots,resulting in insufficient network topology learning.At the same time,due to the lack of appropriate data augmentation methods,it is difficult to capture the evolving patterns of the network effectively.To address the above problems,a position-aware and subgraph enhanced dynamic graph contrastive learning method is proposed for discrete-time dynamic graphs.Firstly,the global snapshot is built based on the historical snapshots to express the stable pattern of the dynamic graph,and the random walk is used to obtain the position representation by learning the positional information of the nodes.Secondly,a new data augmentation method is carried out from the perspectives of short-term changes and long-term stable structures of dynamic graphs.Specifically,subgraph sampling based on snapshots and global snapshots is used to obtain two structural augmentation views,and node structures and evolving patterns are learned by combining graph neural network,gated recurrent unit,and attention mechanism.Finally,the quality of node representation is improved by combining the contrastive learning between different structural augmentation views and between the two representations of structure and position.Experimental results on four real datasets show that the performance of the proposed method is better than the existing unsupervised methods,and it is more competitive than the supervised learning method under a semi-supervised setting.

Privacy-Preserving Large-Scale AI Models for Intelligent Railway Transportation Systems:Hierarchical Poisoning Attacks and Defenses in Federated Learning

The development of Intelligent Railway Transportation Systems necessitates incorporating privacy-preserving mechanisms into AI models to protect sensitive information and enhance system efficiency.Federated learning offers a promising solution by allowing multiple clients to train models collaboratively without sharing private data.However,despite its privacy benefits,federated learning systems are vulnerable to poisoning attacks,where adversaries alter local model parameters on compromised clients and send malicious updates to the server,potentially compromising the global model’s accuracy.In this study,we introduce PMM(Perturbation coefficient Multiplied by Maximum value),a new poisoning attack method that perturbs model updates layer by layer,demonstrating the threat of poisoning attacks faced by federated learning.Extensive experiments across three distinct datasets have demonstrated PMM’s ability to significantly reduce the global model’s accuracy.Additionally,we propose an effective defense method,namely CLBL(Cluster Layer By Layer).Experiment results on three datasets have confirmed CLBL’s effectiveness.

Securing Cloud-Encrypted Data:Detecting Ransomware-as-a-Service(RaaS)Attacks through Deep Learning Ensemble

Data security assurance is crucial due to the increasing prevalence of cloud computing and its widespread use across different industries,especially in light of the growing number of cybersecurity threats.A major and everpresent threat is Ransomware-as-a-Service(RaaS)assaults,which enable even individuals with minimal technical knowledge to conduct ransomware operations.This study provides a new approach for RaaS attack detection which uses an ensemble of deep learning models.For this purpose,the network intrusion detection dataset“UNSWNB15”from the Intelligent Security Group of the University of New South Wales,Australia is analyzed.In the initial phase,the rectified linear unit-,scaled exponential linear unit-,and exponential linear unit-based three separate Multi-Layer Perceptron(MLP)models are developed.Later,using the combined predictive power of these three MLPs,the RansoDetect Fusion ensemble model is introduced in the suggested methodology.The proposed ensemble technique outperforms previous studieswith impressive performance metrics results,including 98.79%accuracy and recall,98.85%precision,and 98.80%F1-score.The empirical results of this study validate the ensemble model’s ability to improve cybersecurity defenses by showing that it outperforms individual MLPmodels.In expanding the field of cybersecurity strategy,this research highlights the significance of combined deep learning models in strengthening intrusion detection systems against sophisticated cyber threats.

Machine learning-guided accelerated discovery of structure-property correlations in lean magnesium alloys for biomedical applications

Magnesium alloys are emerging as promising alternatives to traditional orthopedic implant materials thanks to their biodegradability,biocompatibility,and impressive mechanical characteristics.However,their rapid in-vivo degradation presents challenges,notably in upholding mechanical integrity over time.This study investigates the impact of high-temperature thermal processing on the mechanical and degradation attributes of a lean Mg-Zn-Ca-Mn alloy,ZX10.Utilizing rapid,cost-efficient characterization methods like X-ray diffraction and optical microscopy,we swiftly examine microstructural changes post-thermal treatment.Employing Pearson correlation coefficient analysis,we unveil the relationship between microstructural properties and critical targets(properties):hardness and corrosion resistance.Additionally,leveraging the least absolute shrinkage and selection operator(LASSO),we pinpoint the dominant microstructural factors among closely correlated variables.Our findings underscore the significant role of grain size refinement in strengthening and the predominance of the ternary Ca_(2)Mg_(6)Zn_(3)phase in corrosion behavior.This suggests that achieving an optimal blend of strength and corrosion resistance is attainable through fine grains and reduced concentration of ternary phases.This thorough investigation furnishes valuable insights into the intricate interplay of processing,structure,and properties in magnesium alloys,thereby advancing the development of superior biodegradable implant materials.

Integrating Ontology-Based Approaches with Deep Learning Models for Fine-Grained Sentiment Analysis

Although sentiment analysis is pivotal to understanding user preferences,existing models face significant challenges in handling context-dependent sentiments,sarcasm,and nuanced emotions.This study addresses these challenges by integrating ontology-based methods with deep learning models,thereby enhancing sentiment analysis accuracy in complex domains such as film reviews and restaurant feedback.The framework comprises explicit topic recognition,followed by implicit topic identification to mitigate topic interference in subsequent sentiment analysis.In the context of sentiment analysis,we develop an expanded sentiment lexicon based on domainspecific corpora by leveraging techniques such as word-frequency analysis and word embedding.Furthermore,we introduce a sentiment recognition method based on both ontology-derived sentiment features and sentiment lexicons.We evaluate the performance of our system using a dataset of 10,500 restaurant reviews,focusing on sentiment classification accuracy.The incorporation of specialized lexicons and ontology structures enables the framework to discern subtle sentiment variations and context-specific expressions,thereby improving the overall sentiment-analysis performance.Experimental results demonstrate that the integration of ontology-based methods and deep learning models significantly improves sentiment analysis accuracy.

Fundamental error in tree-based machine learning model selection for reservoir characterisation

Over the past two decades,machine learning techniques have been extensively used in predicting reservoir properties.While this approach has significantly contributed to the industry,selecting an appropriate model is still challenging for most researchers.Relying solely on statistical metrics to select the best model for a particular problem may not always be the most effective approach.This study encourages researchers to incorporate data visualization in their analysis and model selection process.To evaluate the suitability of different models in predicting horizontal permeability in the Volve field,wireline logs were used to train Extra-Trees,Ridge,Bagging,and XGBoost models.The Random Forest feature selection technique was applied to select the relevant logs as inputs for the models.Based on statistical metrics,the Extra-Trees model achieved the highest test accuracy of 0.996,RMSE of 19.54 mD,and MAE of 3.18 mD,with XGBoost coming in second.However,when the results were visualised,it was discovered that the XGBoost model was more suitable for the problem being tackled.The XGBoost model was a better predictor within the sandstone interval,while the Extra-Trees model was more appropriate in non-sandstone intervals.Since this study aims to predict permeability in the reservoir interval,the XGBoost model is the most suitable.These contrasting results demonstrate the importance of incorporating data visualisation techniques as an evaluation metric.Given the heterogeneity of the subsurface,relying solely on statistical metrics may not be sufficient to determine which model is best suited for a particular problem.

A deep learning framework for suppressing prestack seismic random noise without noise-free labels

Random noise attenuation is significant in seismic data processing.Supervised deep learning-based denoising methods have been widely developed and applied in recent years.In practice,it is often time-consuming and laborious to obtain noise-free data for supervised learning.Therefore,we propose a novel deep learning framework to denoise prestack seismic data without clean labels,which trains a high-resolution residual neural network(SRResnet)with noisy data for input and the same valid data with different noise for output.Since valid signals in noisy sample pairs are spatially correlated and random noise is spatially independent and unpredictable,the model can learn the features of valid data while suppressing random noise.Noisy data targets are generated by a simple conventional method without fine-tuning parameters.The initial estimates allow signal or noise leakage as the network does not require clean labels.The Monte Carlo strategy is applied to select training patches for increasing valid patches and expanding training datasets.Transfer learning is used to improve the generalization of real data processing.The synthetic and real data tests perform better than the commonly used state-of-the-art denoising methods.

Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Machine-learning-assisted efficient reconstruction of the quantum states generated from the Sagnac polarization-entangled photon source

Neural networks are becoming ubiquitous in various areas of physics as a successful machine learning(ML)technique for addressing different tasks.Based on ML technique,we propose and experimentally demonstrate an efficient method for state reconstruction of the widely used Sagnac polarization-entangled photon source.By properly modeling the target states,a multi-output fully connected neural network is well trained using only six of the sixteen measurement bases in standard tomography technique,and hence our method reduces the resource consumption without loss of accuracy.We demonstrate the ability of the neural network to predict state parameters with a high precision by using both simulated and experimental data.Explicitly,the mean absolute error for all the parameters is below 0.05 for the simulated data and a mean fidelity of 0.99 is achieved for experimentally generated states.Our method could be generalized to estimate other kinds of states,as well as other quantum information tasks.

Olive Leaf Disease Detection via Wavelet Transform and Feature Fusion of Pre-Trained Deep Learning Models

Olive trees are susceptible to a variety of diseases that can cause significant crop damage and economic losses.Early detection of these diseases is essential for effective management.We propose a novel transformed wavelet,feature-fused,pre-trained deep learning model for detecting olive leaf diseases.The proposed model combines wavelet transforms with pre-trained deep-learning models to extract discriminative features from olive leaf images.The model has four main phases:preprocessing using data augmentation,three-level wavelet transformation,learning using pre-trained deep learning models,and a fused deep learning model.In the preprocessing phase,the image dataset is augmented using techniques such as resizing,rescaling,flipping,rotation,zooming,and contrasting.In wavelet transformation,the augmented images are decomposed into three frequency levels.Three pre-trained deep learning models,EfficientNet-B7,DenseNet-201,and ResNet-152-V2,are used in the learning phase.The models were trained using the approximate images of the third-level sub-band of the wavelet transform.In the fused phase,the fused model consists of a merge layer,three dense layers,and two dropout layers.The proposed model was evaluated using a dataset of images of healthy and infected olive leaves.It achieved an accuracy of 99.72%in the diagnosis of olive leaf diseases,which exceeds the accuracy of other methods reported in the literature.This finding suggests that our proposed method is a promising tool for the early detection of olive leaf diseases.

Applications of deep learning for detecting ophthalmic diseases with ultrawide-field fundus

AIM:To summarize the application of deep learning in detecting ophthalmic disease with ultrawide-field fundus images and analyze the advantages,limitations,and possible solutions common to all tasks.METHODS:We searched three academic databases,including PubMed,Web of Science,and Ovid,with the date of August 2022.We matched and screened according to the target keywords and publication year and retrieved a total of 4358 research papers according to the keywords,of which 23 studies were retrieved on applying deep learning in diagnosing ophthalmic disease with ultrawide-field images.RESULTS:Deep learning in ultrawide-field images can detect various ophthalmic diseases and achieve great performance,including diabetic retinopathy,glaucoma,age-related macular degeneration,retinal vein occlusions,retinal detachment,and other peripheral retinal diseases.Compared to fundus images,the ultrawide-field fundus scanning laser ophthalmoscopy enables the capture of the ocular fundus up to 200°in a single exposure,which can observe more areas of the retina.CONCLUSION:The combination of ultrawide-field fundus images and artificial intelligence will achieve great performance in diagnosing multiple ophthalmic diseases in the future.

Exploring the Core-shell Structure of BaTiO3-based Dielectric Ceramics Using Machine Learning Models and Interpretability Analysis

A machine learning(ML)-based random forest(RF)classification model algorithm was employed to investigate the main factors affecting the formation of the core-shell structure of BaTiO_(3)-based ceramics and their interpretability was analyzed by using Shapley additive explanations(SHAP).An F1-score changed from 0.8795 to 0.9310,accuracy from 0.8450 to 0.9070,precision from 0.8714 to 0.9000,recall from 0.8929 to 0.9643,and ROC/AUC value of 0.97±0.03 was achieved by the RF classification with the optimal set of features containing only 5 features,demonstrating the high accuracy of our model and its high robustness.During the interpretability analysis of the model,it was found that the electronegativity,melting point,and sintering temperature of the dopant contribute highly to the formation of the core-shell structure,and based on these characteristics,specific ranges were delineated and twelve elements were finally obtained that met all the requirements,namely Si,Sc,Mn,Fe,Co,Ni,Pd,Er,Tm,Lu,Pa,and Cm.In the process of exploring the structure of the core-shell,the doping elements can be effectively localized to be selected by choosing the range of features.

Fault Estimation for a Class of Markov Jump Piecewise-Affine Systems: Current Feedback Based Iterative Learning Approach

In this paper, the issues of stochastic stability analysis and fault estimation are investigated for a class of continuoustime Markov jump piecewise-affine(PWA) systems against actuator and sensor faults. Firstly, a novel mode-dependent PWA iterative learning observer with current feedback is designed to estimate the system states and faults, simultaneously, which contains both the previous iteration information and the current feedback mechanism. The auxiliary feedback channel optimizes the response speed of the observer, therefore the estimation error would converge to zero rapidly. Then, sufficient conditions for stochastic stability with guaranteed performance are demonstrated for the estimation error system, and the equivalence relations between the system information and the estimated information can be established via iterative accumulating representation.Finally, two illustrative examples containing a class of tunnel diode circuit systems are presented to fully demonstrate the effectiveness and superiority of the proposed iterative learning observer with current feedback.

Unveiling the Re,Cr,and I diffusion in saturated compacted bentonite using machine-learning methods

The safety assessment of high-level radioactive waste repositories requires a high predictive accuracy for radionuclide diffusion and a comprehensive understanding of the diffusion mechanism.In this study,a through-diffusion method and six machine-learning methods were employed to investigate the diffusion of ReO_(4)^(−),HCrO_(4)^(−),and I−in saturated compacted bentonite under different salinities and compacted dry densities.The machine-learning models were trained using two datasets.One dataset contained six input features and 293 instances obtained from the diffusion database system of the Japan Atomic Energy Agency(JAEA-DDB)and 15 publications.The other dataset,comprising 15,000 pseudo-instances,was produced using a multi-porosity model and contained eight input features.The results indicate that the former dataset yielded a higher predictive accuracy than the latter.Light gradient-boosting exhibited a higher prediction accuracy(R2=0.92)and lower error(MSE=0.01)than the other machine-learning algorithms.In addition,Shapley Additive Explanations,Feature Importance,and Partial Dependence Plot analysis results indicate that the rock capacity factor and compacted dry density had the two most significant effects on predicting the effective diffusion coefficient,thereby offering valuable insights.

A Stacking Machine Learning Model for Student Performance Prediction Based on Class Activities in E-Learning

After the spread of COVID-19,e-learning systems have become crucial tools in educational systems worldwide,spanning all levels of education.This widespread use of e-learning platforms has resulted in the accumulation of vast amounts of valuable data,making it an attractive resource for predicting student performance.In this study,we aimed to predict student performance based on the analysis of data collected from the OULAD and Deeds datasets.The stacking method was employed for modeling in this research.The proposed model utilized weak learners,including nearest neighbor,decision tree,random forest,enhanced gradient,simple Bayes,and logistic regression algorithms.After a trial-and-error process,the logistic regression algorithm was selected as the final learner for the proposed model.The results of experiments with the above algorithms are reported separately for the pass and fail classes.The findings indicate that the accuracy of the proposed model on the OULAD dataset reached 98%.Overall,the proposed method improved accuracy by 4%on the OULAD dataset.

Sentiment Analysis Using E-Commerce Review Keyword-Generated Image with a Hybrid Machine Learning-Based Model

In the context of the accelerated pace of daily life and the development of e-commerce,online shopping is a mainstreamway for consumers to access products and services.To understand their emotional expressions in facing different shopping experience scenarios,this paper presents a sentiment analysis method that combines the ecommerce reviewkeyword-generated imagewith a hybrid machine learning-basedmodel,inwhich theWord2Vec-TextRank is used to extract keywords that act as the inputs for generating the related images by generative Artificial Intelligence(AI).Subsequently,a hybrid Convolutional Neural Network and Support Vector Machine(CNNSVM)model is applied for sentiment classification of those keyword-generated images.For method validation,the data randomly comprised of 5000 reviews from Amazon have been analyzed.With superior keyword extraction capability,the proposedmethod achieves impressive results on sentiment classification with a remarkable accuracy of up to 97.13%.Such performance demonstrates its advantages by using the text-to-image approach,providing a unique perspective for sentiment analysis in the e-commerce review data compared to the existing works.Thus,the proposed method enhances the reliability and insights of customer feedback surveys,which would also establish a novel direction in similar cases,such as social media monitoring and market trend research.