A Dual Discriminator Method for Generalized Zero-Shot Learning

Zero-shot learning enables the recognition of new class samples by migrating models learned from semanticfeatures and existing sample features to things that have never been seen before. The problems of consistencyof different types of features and domain shift problems are two of the critical issues in zero-shot learning. Toaddress both of these issues, this paper proposes a new modeling structure. The traditional approach mappedsemantic features and visual features into the same feature space;based on this, a dual discriminator approachis used in the proposed model. This dual discriminator approach can further enhance the consistency betweensemantic and visual features. At the same time, this approach can also align unseen class semantic features andtraining set samples, providing a portion of information about the unseen classes. In addition, a new feature fusionmethod is proposed in the model. This method is equivalent to adding perturbation to the seen class features,which can reduce the degree to which the classification results in the model are biased towards the seen classes.At the same time, this feature fusion method can provide part of the information of the unseen classes, improvingits classification accuracy in generalized zero-shot learning and reducing domain bias. The proposed method isvalidated and compared with othermethods on four datasets, and fromthe experimental results, it can be seen thatthe method proposed in this paper achieves promising results.

A Real-time Prediction System for Molecular-level Information of Heavy Oil Based on Machine Learning

Acquiring accurate molecular-level information about petroleum is crucial for refining and chemical enterprises to implement the“selection of the optimal processing route”strategy.With the development of data prediction systems represented by machine learning,it has become possible for real-time prediction systems of petroleum fraction molecular information to replace analyses such as gas chromatography and mass spectrometry.However,the biggest difficulty lies in acquiring the data required for training the neural network.To address these issues,this work proposes an innovative method that utilizes the Aspen HYSYS and full two-dimensional gas chromatography-time-of-flight mass spectrometry to establish a comprehensive training database.Subsequently,a deep neural network prediction model is developed for heavy distillate oil to predict its composition in terms of molecular structure.After training,the model accurately predicts the molecular composition of catalytically cracked raw oil in a refinery.The validation and test sets exhibit R2 values of 0.99769 and 0.99807,respectively,and the average relative error of molecular composition prediction for raw materials of the catalytic cracking unit is less than 7%.Finally,the SHAP(SHapley Additive ExPlanation)interpretation method is used to disclose the relationship among different variables by performing global and local weight comparisons and correlation analyses.

Machine learning-based stiffness optimization of digital composite metamaterials with desired positive or negative Poisson's ratio

Mechanical metamaterials such as auxetic materials have attracted great interest due to their unusual properties that are dictated by their architectures.However,these architected materials usually have low stiffness because of the bending or rotation deformation mechanisms in the microstructures.In this work,a convolutional neural network(CNN)based self-learning multi-objective optimization is performed to design digital composite materials.The CNN models have undergone rigorous training using randomly generated two-phase digital composite materials,along with their corresponding Poisson's ratios and stiffness values.Then the CNN models are used for designing composite material structures with the minimum Poisson's ratio at a given volume fraction constraint.Furthermore,we have designed composite materials with optimized stiffness while exhibiting a desired Poisson's ratio(negative,zero,or positive).The optimized designs have been successfully and efficiently obtained,and their validity has been confirmed through finite element analysis results.This self-learning multi-objective optimization model offers a promising approach for achieving comprehensive multi-objective optimization.

Rapid, accurate and serotype independent pipeline for in silico epitope mapping of SARS-CoV-2 antigens: a combined machine learning and Chou’s pseudo amino acid composition method

Here,a new integrated machine learning and Chou’s pseudo amino acid composition method has been proposed for in silico epitope mapping of severe acute respiratorysyndrome-like coronavirus antigens.For this,a training dataset including 266 linear B-cell epitopes,1,267 T-cell epitopes and 1,280 non-epitopes were prepared.The epitope sequences were then converted to numerical vectors using Chou’s pseudo amino acid composition method.The vectors were then introduced to the support vector machine,random forest,artificial neural network,and K-nearest neighbor algorithms for the classification process.The algorithm with the highest performance was selected for the epitope mapping procedure.Based on the obtained results,the random forest algorithm was the most accurate classifier with an accuracy of 0.934 followed by K-nearest neighbor,artificial neural network,and support vector machine respectively.Furthermore,the efficacies of predicted epitopes by the trained random forest algorithm were assessed through their antigenicity potential as well as affinity to human B cell receptor and MHC-I/II alleles using the VaxiJen score and molecular docking,respectively.It was also clear that the predicted epitopes especially the B-cell epitopes had high antigenicity potentials and good affinities to the protein targets.According to the results,the suggested method can be considered for developing specific epitope predictor software as well as an accelerator pipeline for designing serotype independent vaccine against the virus.

基于反向投影的zero-shot learning目标分类算法研究

Zero-shot learning(ZSL)是针对没有训练样本的类别进行分类的问题。传统回归方法的核心是将视觉特征投影到语义空间,没有充分利用视觉特征自身包含的样本信息,同时训练计算量大。提出基于反向投影的ZSL目标分类方法,将类别原型投影到视觉空间,利用视觉特征的语义性学习出映射函数,参数优化过程仅通过解析解就可以获得。在两个基准数据集的实验结果表明,提出的反向投影方法分类结果较传统回归方法和其他现有方法有大幅提升,并且训练时间大大减少,可以更好地推广到未知类别的分类问题上。

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image.The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered,and dissimilar images are separated in the low embedding space.Previous works primarily focused on defining local structure loss functions like triplet loss,pairwise loss,etc.However,training via these approaches takes a long training time,and they have poor accuracy.Additionally,representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes.This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues.In the proposed work,class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters.And other instances are treated as negatives within the same cluster.Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space.The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results(85.38%recall@1 for CARS-196%and 70.13%recall@1 for CUB-200)compared to other existing methods.

Gasification of Organic Waste:Parameters,Mechanism and Prediction with the Machine Learning Approach

Gasification of organic waste represents one of the most effective valorization pathways for renewable energy and resources recovery,while this process can be affected by multi-factors like temperature,feedstock,and steam content,making the product’s prediction problematic.With the popularization and promotion of artificial intelligence such as machine learning(ML),traditional artificial neural networks have been paid more attention by researchers from the data science field,which provides scientific and engineering communities with flexible and rapid prediction frameworks in the field of organic waste gasification.In this work,critical parameters including temperature,steam ratio,and feedstock during gasification of organic waste were reviewed in three scenarios including steam gasification,air gasification,and oxygen-riched gasification,and the product distribution and involved mechanism were elaborated.Moreover,we presented the details of ML methods like regression analysis,artificial neural networks,decision trees,and related methods,which are expected to revolutionize data analysis and modeling of the gasification of organic waste.Typical outputs including the syngas yield,composition,and HHVs were discussed with a better understanding of the gasification process and ML application.This review focused on the combination of gasification and ML,and it is of immediate significance for the resource and energy utilization of organic waste.

Breed identification using breed‑informative SNPs and machine learning based on whole genome sequence data and SNP chip data

Background Breed identification is useful in a variety of biological contexts.Breed identification usually involves two stages,i.e.,detection of breed-informative SNPs and breed assignment.For both stages,there are several methods proposed.However,what is the optimal combination of these methods remain unclear.In this study,using the whole genome sequence data available for 13 cattle breeds from Run 8 of the 1,000 Bull Genomes Project,we compared the combinations of three methods(Delta,FST,and In)for breed-informative SNP detection and five machine learning methods(KNN,SVM,RF,NB,and ANN)for breed assignment with respect to different reference population sizes and difference numbers of most breed-informative SNPs.In addition,we evaluated the accuracy of breed identification using SNP chip data of different densities.Results We found that all combinations performed quite well with identification accuracies over 95%in all scenarios.However,there was no combination which performed the best and robust across all scenarios.We proposed to inte-grate the three breed-informative detection methods,named DFI,and integrate the three machine learning methods,KNN,SVM,and RF,named KSR.We found that the combination of these two integrated methods outperformed the other combinations with accuracies over 99%in most cases and was very robust in all scenarios.The accuracies from using SNP chip data were only slightly lower than that from using sequence data in most cases.Conclusions The current study showed that the combination of DFI and KSR was the optimal strategy.Using sequence data resulted in higher accuracies than using chip data in most cases.However,the differences were gener-ally small.In view of the cost of genotyping,using chip data is also a good option for breed identification.

Length matters:Scalable fast encrypted internet traffic service classification based on multiple protocol data unit length sequence with composite deep learning

As an essential function of encrypted Internet traffic analysis,encrypted traffic service classification can support both coarse-grained network service traffic management and security supervision.However,the traditional plaintext-based Deep Packet Inspection(DPI)method cannot be applied to such a classification.Moreover,machine learning-based existing methods encounter two problems during feature selection:complex feature overcost processing and Transport Layer Security(TLS)version discrepancy.In this paper,we consider differences between encryption network protocol stacks and propose a composite deep learning-based method in multiprotocol environments using a sliding multiple Protocol Data Unit(multiPDU)length sequence as features by fully utilizing the Markov property in a multiPDU length sequence and maintaining suitability with a TLS-1.3 environment.Control experiments show that both Length-Sensitive(LS)composite deep learning model using a capsule neural network and LS-long short time memory achieve satisfactory effectiveness in F1-score and performance.Owing to faster feature extraction,our method is suitable for actual network environments and superior to state-of-the-art methods.

A Composite Transformer-Based Multi-Stage Defect Detection Architecture for Sewer Pipes

Urban sewer pipes are a vital infrastructure in modern cities,and their defects must be detected in time to prevent potential malfunctioning.In recent years,to relieve the manual efforts by human experts,models based on deep learning have been introduced to automatically identify potential defects.However,these models are insufficient in terms of dataset complexity,model versatility and performance.Our work addresses these issues with amulti-stage defect detection architecture using a composite backbone Swin Transformer.Themodel based on this architecture is trained using a more comprehensive dataset containingmore classes of defects.By ablation studies on the modules of combined backbone Swin Transformer,multi-stage detector,test-time data augmentation and model fusion,it is revealed that they all contribute to the improvement of detection accuracy from different aspects.The model incorporating all these modules achieves the mean Average Precision(mAP)of 78.6% at an Intersection over Union(IoU)threshold of 0.5.This represents an improvement of 14.1% over the ResNet50 Faster Region-based Convolutional Neural Network(R-CNN)model and a 6.7% improvement over You Only Look Once version 6(YOLOv6)-large,the highest in the YOLO methods.In addition,for other defect detection models for sewer pipes,although direct comparison with themis infeasible due to the unavailability of their private datasets,our results are obtained from a more comprehensive dataset and have superior generalization capabilities.

Automated body composition analysis system based on chest CT for evaluating content of muscle and adipose

Objective To establish a body composition analysis system based on chest CT,and to observe its value for evaluating content of chest muscle and adipose.Methods T7—T8 layer CT images of 108 pneumonia patients were collected(segmented dataset),and chest CT data of 984 patients were screened from the COVID 19-CT dataset(10 cases were randomly selected as whole test dataset,the remaining 974 cases were selected as layer selection dataset).T7—T8 layer was classified based on convolutional neural network(CNN)derived networks,including ResNet,ResNeXt,MobileNet,ShuffleNet,DenseNet,EfficientNet and ConvNeXt,then the accuracy,precision,recall and specificity were used to evaluate the performance of layer selection dataset.The skeletal muscle(SM),subcutaneous adipose tissue(SAT),intermuscular adipose tissue(IMAT)and visceral adipose tissue(VAT)were segmented using classical fully CNN(FCN)derived network,including FCN,SegNet,UNet,Attention UNet,UNET++,nnUNet,UNeXt and CMUNeXt,then Dice similarity coefficient(DSC),intersection over union(IoU)and 95 Hausdorff distance(HD)were used to evaluate the performance of segmented dataset.The automatic body composition analysis system was constructed based on optimal layer selection network and segmentation network,the mean absolute error(MAE),root mean squared error(RMSE)and standard deviation(SD)of MAE were used to evaluate the performance of automatic system for testing the whole test dataset.Results The accuracy,precision,recall and specificity of DenseNet network for automatically classifying T7—T8 layer from chest CT images was 95.06%,84.83%,92.27%and 95.78%,respectively,which were all higher than those of the other layer selection networks.In segmentation of SM,SAT,IMAT and overall,DSC and IoU of UNet++network were all higher,while 95HD of UNet++network were all lower than those of the other segmentation networks.Using DenseNet as the layer selection network and UNet++as the segmentation network,MAE of the automatic body composition analysis system for predicting SM,SAT,IMAT,VAT and MAE was 27.09,6.95,6.65 and 3.35 cm 2,respectively.Conclusion The body composition analysis system based on chest CT could be used to assess content of chest muscle and adipose.Among them,the UNet++network had better segmentation performance in adipose tissue than SM.

A Service Composition Approach Based on Sequence Mining for Migrating E-learning Legacy System to SOA

With the fast development of business logic and information technology, today＇s best solutions are tomorrow＇s legacy systems. In China, the situation in the education domain follows the same path. Currently, there exists a number of e-learning legacy assets with accumulated practical business experience, such as program resource, usage behaviour data resource, and so on. In order to use these legacy assets adequately and efficiently, we should not only utilize the explicit assets but also discover the hidden assets. The usage behaviour data resource is the set of practical operation sequences requested by all users. The hidden patterns in this data resource will provide users＇ practical experiences, which can benefit the service composition in service-oriented architecture （SOA） migration. Namely, these discovered patterns will be the candidate composite services （coarse-grained） in SOA systems. Although data mining techniques have been used for software engineering tasks, little is known about how they can be used for service composition of migrating an e-learning legacy system （MELS） to SOA. In this paper, we propose a service composition approach based on sequence mining techniques for MELS. Composite services found by this approach will be the complementation of business logic analysis results of MELS. The core of this approach is to develop an appropriate sequence mining algorithm for mining related data collected from an e-learning legacy system. According to the features of execution trace data on usage behaviour from this e-learning legacy system and needs of further pattern analysis, we propose a sequential mining algorithm to mine this kind of data of tile legacy system. For validation, this approach has been applied to the corresponding real data, which was collected from the e-learning legacy system; meanwhile, some investigation questionnaires were set up to collect satisfaction data. The investigation result is 90% the same with the result obtained through our approach.

A geospatial service composition approach based on MCTS with temporal-difference learning

With the complexity of the composition process and the rapid growth of candidate services,realizing optimal or near-optimal service composition is an urgent problem.Currently,the static service composition chain is rigid and cannot be easily adapted to the dynamic Web environment.To address these challenges,the geographic information service composition(GISC) problem as a sequential decision-making task is modeled.In addition,the Markov decision process(MDP),as a universal model for the planning problem of agents,is used to describe the GISC problem.Then,to achieve self-adaptivity and optimization in a dynamic environment,a novel approach that integrates Monte Carlo tree search(MCTS) and a temporal-difference(TD) learning algorithm is proposed.The concrete services of abstract services are determined with optimal policies and adaptive capability at runtime,based on the environment and the status of component services.The simulation experiment is performed to demonstrate the effectiveness and efficiency through learning quality and performance.

Explanatory Multi-Scale Adversarial Semantic Embedding Space Learning for Zero-Shot Recognition

The goal of zero-shot recognition is to classify classes it has never seen before, which needs to build a bridge between seen and unseen classes through semantic embedding space. Therefore, semantic embedding space learning plays an important role in zero-shot recognition. Among existing works, semantic embedding space is mainly taken by user-defined attribute vectors. However, the discriminative information included in the user-defined attribute vector is limited. In this paper, we propose to learn an extra latent attribute space automatically to produce a more generalized and discriminative semantic embedded space. To prevent the bias problem, both user-defined attribute vector and latent attribute space are optimized by adversarial learning with auto-encoders. We also propose to reconstruct semantic patterns produced by explanatory graphs, which can make semantic embedding space more sensitive to usefully semantic information and less sensitive to useless information. The proposed method is evaluated on the AwA2 and CUB dataset. These results show that our proposed method achieves superior performance.

A machine-learning-enabled approach for bridging multiscale simulations of CNTs/PDMS composites

Benefitting from the interlaced networking structure of carbon nanotubes(CNTs),the composites of CNTs/polydimethylsiloxane(PDMS)have found extensive applications in wearable electronics.While hierarchical multiscale simulation frameworks exist to optimize the structure parameters,their wide applications were hindered by the high computational cost.In this study,a machine learning model based on the artificial neural networks(ANN)embedded graph attention network,termed as AGAT,was proposed.The datasets collected from the micro-scale and the macro-scale simulations are utilized to train the model.The ANN layer within the model framework is trained to pass the information from micro-scale to macro-scale,while the whole model is aimed to predict the electro-mechanical behavior of the CNTs/PDMS composites.By comparing the AGAT model with the original multiscale simulation results,the data-driven strategy is shown to be promising with high accuracy,demonstrating the potential of the machine-learning-enabled approach for the structure optimization of CNT-based composites.

Modeling and prediction of tribological properties of copper/aluminum-graphite self-lubricating composites using machine learning algorithms

The tribological properties of self-lubricating composites are influenced by many variables and complex mechanisms.Data-driven methods,including machine learning(ML)algorithms,can yield a better comprehensive understanding of complex problems under the influence of multiple parameters,typically for how tribological performances and material properties correlate.Correlation of friction coefficients and wear rates of copper/aluminum-graphite(Cu/Al-graphite)self-lubricating composites with their inherent material properties(composition,lubricant content,particle size,processing process,and interfacial bonding strength)and the variables related to the testing method(normal load,sliding speed,and sliding distance)were analyzed using traditional approaches,followed by modeling and prediction of tribological properties through five different ML algorithms,namely support vector machine(SVM),K-Nearest neighbor(KNN),random forest(RF),eXtreme gradient boosting(XGBoost),and least-squares boosting(LSBoost),based on the tribology experimental data.Results demonstrated that ML models could satisfactorily predict friction coefficient and wear rate from the material properties and testing method variables data.Herein,the LSBoost model based on the integrated learning algorithm presented the best prediction performance for friction coefficients and wear rates,with R^(2) of 0.9219 and 0.9243,respectively.Feature importance analysis also revealed that the content of graphite and the hardness of the matrix have the greatest influence on the friction coefficients,and the normal load,the content of graphite,and the hardness of the matrix influence the wear rates the most.

Machine Learning-Based Research on Tensile Strength of SiC-Reinforced Magnesium Matrix Composites via Stir Casting

SiC is the most common reinforcement in magnesium matrix composites,and the tensile strength of SiC-reinforced magnesium matrix composites is closely related to the distribution of SiC.Achieving a uniform distribution of SiC requires fine control over the parameters of SiC and the processing and preparation process.However,due to the numerous adjustable parameters,using traditional experimental methods requires a considerable amount of experimentation to obtain a uniformly distributed composite material.Therefore,this study adopts a machine learning approach to explore the tensile strength of SiC-reinforced magnesium matrix composites in the mechanical stirring casting process.By analyzing the influence of SiC parameters and processing parameters on composite material performance,we have established an effective predictive model.Furthermore,six different machine learning regression models have been developed to predict the tensile strength of SiC-reinforced magnesium matrix composites.Through validation and comparison,our models demonstrate good accuracy and reliability in predicting the tensile strength of the composite material.The research findings indicate that hot extrusion treatment,SiC content,and stirring time have a significant impact on the tensile strength.

Observer-based Adaptive Iterative Learning Control for Nonlinear Systems with Time-varying Delays

An observer-based adaptive iterative learning control （AILC） scheme is developed for a class of nonlinear systems with unknown time-varying parameters and unknown time-varying delays. The linear matrix inequality （LMI） method is employed to design the nonlinear observer. The designed controller contains a proportional-integral-derivative （PID） feedback term in time domain. The learning law of unknown constant parameter is differential-difference-type, and the learning law of unknown time-varying parameter is difference-type. It is assumed that the unknown delay-dependent uncertainty is nonlinearly parameterized. By constructing a Lyapunov-Krasovskii-like composite energy function （CEF）, we prove the boundedness of all closed-loop signals and the convergence of tracking error. A simulation example is provided to illustrate the effectiveness of the control algorithm proposed in this paper.

An Exploration on Adaptive Iterative Learning Control for a Class of Commensurate High-order Uncertain Nonlinear Fractional Order Systems

This paper explores the adaptive iterative learning control method in the control of fractional order systems for the first time. An adaptive iterative learning control(AILC) scheme is presented for a class of commensurate high-order uncertain nonlinear fractional order systems in the presence of disturbance.To facilitate the controller design, a sliding mode surface of tracking errors is designed by using sufficient conditions of linear fractional order systems. To relax the assumption of the identical initial condition in iterative learning control(ILC), a new boundary layer function is proposed by employing MittagLeffler function. The uncertainty in the system is compensated for by utilizing radial basis function neural network. Fractional order differential type updating laws and difference type learning law are designed to estimate unknown constant parameters and time-varying parameter, respectively. The hyperbolic tangent function and a convergent series sequence are used to design robust control term for neural network approximation error and bounded disturbance, simultaneously guaranteeing the learning convergence along iteration. The system output is proved to converge to a small neighborhood of the desired trajectory by constructing Lyapnov-like composite energy function(CEF)containing new integral type Lyapunov function, while keeping all the closed-loop signals bounded. Finally, a simulation example is presented to verify the effectiveness of the proposed approach.

Machine learning technique for prediction of magnetocaloric effect in La(Fe,Si/Al)_(13)-based materials

Data-mining techniques using machine learning are powerful and efficient for materials design, possessing great potential for discovering new materials with good characteristics. Here, this technique has been used on composition design for La（Fe,Si/Al）（13）-based materials, which are regarded as one of the most promising magnetic refrigerants in practice. Three prediction models are built by using a machine learning algorithm called gradient boosting regression tree（GBRT） to essentially find the correlation between the Curie temperature（TC）, maximum value of magnetic entropy change（（？SM）（max））,and chemical composition, all of which yield high accuracy in the prediction of TC and（？SM）（max）. The performance metric coefficient scores of determination（R^2） for the three models are 0.96, 0.87, and 0.91. These results suggest that all of the models are well-developed predictive models on the challenging issue of generalization ability for untrained data, which can not only provide us with suggestions for real experiments but also help us gain physical insights to find proper composition for further magnetic refrigeration applications.