A Deep Learning Ensemble Method for Forecasting Daily Crude Oil Price Based on Snapshot Ensemble of Transformer Model

The oil industries are an important part of a country’s economy.The crude oil’s price is influenced by a wide range of variables.Therefore,how accurately can countries predict its behavior and what predictors to employ are two main questions.In this view,we propose utilizing deep learning and ensemble learning techniques to boost crude oil’s price forecasting performance.The suggested method is based on a deep learning snapshot ensemble method of the Transformer model.To examine the superiority of the proposed model,this paper compares the proposed deep learning ensemble model against different machine learning and statistical models for daily Organization of the Petroleum Exporting Countries(OPEC)oil price forecasting.Experimental results demonstrated the outperformance of the proposed method over statistical and machine learning methods.More precisely,the proposed snapshot ensemble of Transformer method achieved relative improvement in the forecasting performance compared to autoregressive integrated moving average ARIMA(1,1,1),ARIMA(0,1,1),autoregressive moving average(ARMA)(0,1),vector autoregression(VAR),random walk(RW),support vector machine(SVM),and random forests(RF)models by 99.94%,99.62%,99.87%,99.65%,7.55%,98.38%,and 99.35%,respectively,according to mean square error metric.

Chaotic Flower Pollination with Deep Learning Based COVID-19 Classification Model

The Coronavirus Disease(COVID-19)pandemic has exposed the vulnerabilities of medical services across the globe,especially in underdeveloped nations.In the aftermath of the COVID-19 outbreak,a strong demand exists for developing novel computer-assisted diagnostic tools to execute rapid and cost-effective screenings in locations where many screenings cannot be executed using conventional methods.Medical imaging has become a crucial component in the disease diagnosis process,whereas X-rays and Computed Tomography(CT)scan imaging are employed in a deep network to diagnose the diseases.In general,four steps are followed in image-based diagnostics and disease classification processes by making use of the neural networks,such as network training,feature extraction,model performance testing and optimal feature selection.The current research article devises a Chaotic Flower Pollination Algorithm with a Deep Learning-Driven Fusion(CFPADLDF)approach for detecting and classifying COVID-19.The presented CFPA-DLDF model is developed by integrating two DL models to recognize COVID-19 in medical images.Initially,the proposed CFPA-DLDF technique employs the Gabor Filtering(GF)approach to pre-process the input images.In addition,a weighted voting-based ensemble model is employed for feature extraction,in which both VGG-19 and the MixNet models are included.Finally,the CFPA with Recurrent Neural Network(RNN)model is utilized for classification,showing the work’s novelty.A comparative analysis was conducted to demonstrate the enhanced performance of the proposed CFPADLDF model,and the results established the supremacy of the proposed CFPA-DLDF model over recent approaches.

Ensemble Deep Learning Framework for Situational Aspects-Based Annotation and Classification of International Student’s Tweets during COVID-19

As the COVID-19 pandemic swept the globe,social media plat-forms became an essential source of information and communication for many.International students,particularly,turned to Twitter to express their struggles and hardships during this difficult time.To better understand the sentiments and experiences of these international students,we developed the Situational Aspect-Based Annotation and Classification(SABAC)text mining framework.This framework uses a three-layer approach,combining baseline Deep Learning(DL)models with Machine Learning(ML)models as meta-classifiers to accurately predict the sentiments and aspects expressed in tweets from our collected Student-COVID-19 dataset.Using the pro-posed aspect2class annotation algorithm,we labeled bulk unlabeled tweets according to their contained aspect terms.However,we also recognized the challenges of reducing data’s high dimensionality and sparsity to improve performance and annotation on unlabeled datasets.To address this issue,we proposed the Volatile Stopwords Filtering(VSF)technique to reduce sparsity and enhance classifier performance.The resulting Student-COVID Twitter dataset achieved a sophisticated accuracy of 93.21%when using the random forest as a meta-classifier.Through testing on three benchmark datasets,we found that the SABAC ensemble framework performed exceptionally well.Our findings showed that international students during the pandemic faced various issues,including stress,uncertainty,health concerns,financial stress,and difficulties with online classes and returning to school.By analyzing and summarizing these annotated tweets,decision-makers can better understand and address the real-time problems international students face during the ongoing pandemic.

MDEV Model:A Novel Ensemble-Based Transfer Learning Approach for Pneumonia Classification Using CXR Images

Pneumonia is a dangerous respiratory disease due to which breathing becomes incredibly difficult and painful;thus,catching it early is crucial.Medical physicians’time is limited in outdoor situations due to many patients;therefore,automated systems can be a rescue.The input images from the X-ray equipment are also highly unpredictable due to variances in radiologists’experience.Therefore,radiologists require an automated system that can swiftly and accurately detect pneumonic lungs from chest x-rays.In medical classifications,deep convolution neural networks are commonly used.This research aims to use deep pretrained transfer learning models to accurately categorize CXR images into binary classes,i.e.,Normal and Pneumonia.The MDEV is a proposed novel ensemble approach that concatenates four heterogeneous transfer learning models:Mobile-Net,DenseNet-201,EfficientNet-B0,and VGG-16,which have been finetuned and trained on 5,856 CXR images.The evaluation matrices used in this research to contrast different deep transfer learning architectures include precision,accuracy,recall,AUC-roc,and f1-score.The model effectively decreases training loss while increasing accuracy.The findings conclude that the proposed MDEV model outperformed cutting-edge deep transfer learning models and obtains an overall precision of 92.26%,an accuracy of 92.15%,a recall of 90.90%,an auc-roc score of 90.9%,and f-score of 91.49%with minimal data pre-processing,data augmentation,finetuning and hyperparameter adjustment in classifying Normal and Pneumonia chests.

User Purchase Intention Prediction Based on Improved Deep Forest

Widely used deep neural networks currently face limitations in achieving optimal performance for purchase intention prediction due to constraints on data volume and hyperparameter selection.To address this issue,based on the deep forest algorithm and further integrating evolutionary ensemble learning methods,this paper proposes a novel Deep Adaptive Evolutionary Ensemble(DAEE)model.This model introduces model diversity into the cascade layer,allowing it to adaptively adjust its structure to accommodate complex and evolving purchasing behavior patterns.Moreover,this paper optimizes the methods of obtaining feature vectors,enhancement vectors,and prediction results within the deep forest algorithm to enhance the model’s predictive accuracy.Results demonstrate that the improved deep forest model not only possesses higher robustness but also shows an increase of 5.02%in AUC value compared to the baseline model.Furthermore,its training runtime speed is 6 times faster than that of deep models,and compared to other improved models,its accuracy has been enhanced by 0.9%.

An Intelligent Hazardous Waste Detection and Classification Model Using Ensemble Learning Techniques

Proper waste management models using recent technologies like computer vision,machine learning(ML),and deep learning(DL)are needed to effectively handle the massive quantity of increasing waste.Therefore,waste classification becomes a crucial topic which helps to categorize waste into hazardous or non-hazardous ones and thereby assist in the decision making of the waste management process.This study concentrates on the design of hazardous waste detection and classification using ensemble learning(HWDC-EL)technique to reduce toxicity and improve human health.The goal of the HWDC-EL technique is to detect the multiple classes of wastes,particularly hazardous and non-hazardous wastes.The HWDC-EL technique involves the ensemble of three feature extractors using Model Averaging technique namely discrete local binary patterns(DLBP),EfficientNet,and DenseNet121.In addition,the flower pollination algorithm(FPA)based hyperparameter optimizers are used to optimally adjust the parameters involved in the EfficientNet and DenseNet121 models.Moreover,a weighted voting-based ensemble classifier is derived using three machine learning algorithms namely support vector machine(SVM),extreme learning machine(ELM),and gradient boosting tree(GBT).The performance of the HWDC-EL technique is tested using a benchmark Garbage dataset and it obtains a maximum accuracy of 98.85%.

Covid-19 Diagnosis Using a Deep Learning Ensemble Model with Chest X-Ray Images

Covid-19 is a deadly virus that is rapidly spread around the world towards the end of the 2020.The consequences of this virus are quite frightening,especially when accompanied by an underlying disease.The novelty of the virus,the constant emergence of different variants and its rapid spread have a negative impact on the control and treatment process.Although the new test kits provide almost certain results,chest X-rays are extremely important to detect the progression and degree of the disease.In addition to the Covid-19 virus,pneumonia and harmless opacity of the lungs also complicate the diagnosis.Considering the negative results caused by the virus and the treatment costs,the importance of fast and accurate diagnosis is clearly seen.In this context,deep learning methods appear as an extremely popular approach.In this study,a hybrid model design with superior properties of convolutional neural networks is presented to correctly classify the Covid-19 disease.In addition,in order to contribute to the literature,a suitable dataset with balanced case numbers that can be used in all artificial intelligence classification studies is presented.With this ensemble model design,quite remarkable results are obtained for the diagnosis of three and four-class Covid-19.The proposed model can classify normal,pneumonia,and Covid-19 with 92.6%accuracy and 82.6%for normal,pneumonia,Covid-19,and lung opacity.

Ensemble Based Learning with Accurate Motion Contrast Detection

Recent developments in computer vision applications have enabled detection of significant visual objects in video streams.Studies quoted in literature have detected objects from video streams using Spatiotemporal Particle Swarm Optimization(SPSOM)and Incremental Deep Convolution Neural Networks(IDCNN)for detecting multiple objects.However,the study considered opticalflows resulting in assessing motion contrasts.Existing methods have issue with accuracy and error rates in motion contrast detection.Hence,the overall object detection performance is reduced significantly.Thus,consideration of object motions in videos efficiently is a critical issue to be solved.To overcome the above mentioned problems,this research work proposes a method involving ensemble approaches to and detect objects efficiently from video streams.This work uses a system modeled on swarm optimization and ensemble learning called Spatiotemporal Glowworm Swarm Optimization Model(SGSOM)for detecting multiple significant objects.A steady quality in motion contrasts is maintained in this work by using Chebyshev distance matrix.The proposed system achieves global optimization in its multiple object detection by exploiting spatial/temporal cues and local constraints.Its experimental results show that the proposed system scores 4.8%in Mean Absolute Error(MAE)while achieving 86%in accuracy,81.5%in precision,85%in recall and 81.6%in F-measure and thus proving its utility in detecting multiple objects.

DCEL:classifier fusion model for Android malware detection

The rapid growth of mobile applications,the popularity of the Android system and its openness have attracted many hackers and even criminals,who are creating lots of Android malware.However,the current methods of Android malware detection need a lot of time in the feature engineering phase.Furthermore,these models have the defects of low detection rate,high complexity,and poor practicability,etc.We analyze the Android malware samples,and the distribution of malware and benign software in application programming interface(API)calls,permissions,and other attributes.We classify the software’s threat levels based on the correlation of features.Then,we propose deep neural networks and convolutional neural networks with ensemble learning(DCEL),a new classifier fusion model for Android malware detection.First,DCEL preprocesses the malware data to remove redundant data,and converts the one-dimensional data into a two-dimensional gray image.Then,the ensemble learning approach is used to combine the deep neural network with the convolutional neural network,and the final classification results are obtained by voting on the prediction of each single classifier.Experiments based on the Drebin and Malgenome datasets show that compared with current state-of-art models,the proposed DCEL has a higher detection rate,higher recall rate,and lower computational cost.

Attenuate Class Imbalance Problem for Pneumonia Diagnosis Using Ensemble Parallel Stacked Pre-Trained Models

Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.

Novel Ensemble Modeling Method for Enhancing Subset Diversity Using Clustering Indicator Vector Based on Stacked Autoencoder

A single model cannot satisfy the high-precision prediction requirements given the high nonlinearity between variables.By contrast,ensemble models can effectively solve this problem.Three key factors for improving the accuracy of ensemble models are namely the high accuracy of a submodel,the diversity between subsample sets and the optimal ensemble method.This study presents an improved ensemble modeling method to improve the prediction precision and generalization capability of the model.Our proposed method first uses a bagging algorithm to generate multiple subsample sets.Second,an indicator vector is defined to describe these subsample sets.Third,subsample sets are selected on the basis of the results of agglomerative nesting clustering on indicator vectors to maximize the diversity between subsets.Subsequently,these subsample sets are placed in a stacked autoencoder for training.Finally,XGBoost algorithm,rather than the traditional simple average ensemble method,is imported to ensemble the model during modeling.Three machine learning public datasets and atmospheric column dry point dataset from a practical industrial process show that our proposed method demonstrates high precision and improved prediction ability.

Masked Face Recognition Using MobileNet V2 with Transfer Learning

Corona virus(COVID-19)is once in a life time calamity that has resulted in thousands of deaths and security concerns.People are using face masks on a regular basis to protect themselves and to help reduce corona virus transmission.During the on-going coronavirus outbreak,one of the major priorities for researchers is to discover effective solution.As important parts of the face are obscured,face identification and verification becomes exceedingly difficult.The suggested method is a transfer learning using MobileNet V2 based technology that uses deep feature such as feature extraction and deep learning model,to identify the problem of face masked identification.In the first stage,we are applying face mask detector to identify the face mask.Then,the proposed approach is applying to the datasets from Canadian Institute for Advanced Research10(CIFAR10),Modified National Institute of Standards and Technology Database(MNIST),Real World Masked Face Recognition Database(RMFRD),and Stimulated Masked Face Recognition Database(SMFRD).The proposed model is achieving recognition accuracy 99.82%with proposed dataset.This article employs the four pre-programmed models VGG16,VGG19,ResNet50 and ResNet101.To extract the deep features of faces with VGG16 is achieving 99.30%accuracy,VGG19 is achieving 99.54%accuracy,ResNet50 is achieving 78.70%accuracy and ResNet101 is achieving 98.64%accuracy with own dataset.The comparative analysis shows,that our proposed model performs better result in all four previous existing models.The fundamental contribution of this study is to monitor with face mask and without face mask to decreases the pace of corona virus and to detect persons using wearing face masks.

A Hybrid Deep Learning Approach to Classify the Plant Leaf Species

Many plant species have a startling degree of morphological similarity,making it difficult to split and categorize them reliably.Unknown plant species can be challenging to classify and segment using deep learning.While using deep learning architectures has helped improve classification accuracy,the resulting models often need to be more flexible and require a large dataset to train.For the sake of taxonomy,this research proposes a hybrid method for categorizing guava,potato,and java plumleaves.Two new approaches are used to formthe hybridmodel suggested here.The guava,potato,and java plum plant species have been successfully segmented using the first model built on the MobileNetV2-UNET architecture.As a second model,we use a Plant Species Detection Stacking Ensemble Deep Learning Model(PSD-SE-DLM)to identify potatoes,java plums,and guava.The proposed models were trained using data collected in Punjab,Pakistan,consisting of images of healthy and sick leaves from guava,java plum,and potatoes.These datasets are known as PLSD and PLSSD.Accuracy levels of 99.84%and 96.38%were achieved for the suggested PSD-SE-DLM and MobileNetV2-UNET models,respectively.

A spatiotemporal 3D convolutional neural network model for ENSO predictions: A test case for the 2020/21 La Niña conditions

2020–22年间热带太平洋经历了持续性多年的拉尼娜事件,多数耦合模式都难以准确预测其演变过程,这为厄尔尼诺-南方涛动(ENSO)的实时预测带来了很大的挑战.同时,目前学术界对此次持续性双拉尼娜事件的发展仍缺乏合理的物理解释,其所涉及的物理过程和机制有待于进一步分析.本研究利用再分析数据产品分析了热带东南太平洋东南风异常及其引起的次表层海温异常在此次热带太平洋海表温度(SST)异常演变中的作用,并构建了一个时空分离(Time-Space)的三维(3D)卷积神经网络模型(TS-3DCNN)对此次双拉尼娜事件进行实时预测和过程分析.通过将TS-3DCNN与中国科学院海洋研究所(IOCAS)中等复杂程度海气耦合模式(IOCAS ICM)的预测结果对比,表明TS-3DCNN模型对2020–22年双重拉尼娜现象的预测能力与IOCAS ICM相当,二者均能够从2021年初的初始场开始较好地预测2021年末El Niño3.4区SST的演变.此外,基于TS-3DCNN和IOCAS ICM的敏感性试验也验证了赤道外风场异常和次表层海温异常在2021年末赤道中东太平洋海表二次变冷过程中的关键作用.未来将神经网络与动力模式模式间的有效结合,进一步发展神经网络与物理过程相结合的混合建模是进一步提高ENSO事件预测能力的有效途径.

Locally Linear Back-propagation Based Contribution for Nonlinear Process Fault Diagnosis

This paper proposes a novel locally linear backpropagation based contribution(LLBBC) for nonlinear process fault diagnosis. As a method based on the deep learning model of auto-encoder(AE), LLBBC can deal with the fault diagnosis problem through extracting nonlinear features. When the on-line fault diagnosis task is in progress, a locally linear model is firstly built at the current fault sample. According to the basic idea of reconstruction based contribution(RBC), the propagation of fault information is described by using back-propagation(BP) algorithm. Then, a contribution index is established to measure the correlation between the variable and the fault, and the final diagnosis result is obtained by searching variables with large contributions. The smearing effect, which is an important factor affecting the performance of fault diagnosis, can be suppressed as well,and the theoretical analysis reveals that the correct diagnosis can be guaranteed by LLBBC. Finally, the feasibility and effectiveness of the proposed method are verified through a nonlinear numerical example and the Tennessee Eastman benchmark process.

生成扩散模型研究综述

扩散模型在生成模型领域具有高质量的样本生成能力,一经推出就不断地刷新图像生成评价指标FID分数的记录,成为了该领域的研究热点,而此类相关综述在国内还鲜有介绍。因此,文中对相关扩散生成模型的研究进行汇总与分析。首先,对去噪扩散概率模型、基于分数的扩散生成模型和随机微分方程的扩散生成模型这3类通用模型的特点和原理进行了论述,就每一类基本扩散模型中以优化模型内部算法、高效采样为改进目标的相关衍生模型进行分析。其次,对当下扩散模型在计算机视觉、自然语言处理、时间序列、多模态和跨学科领域等方面的应用进行总结。最后,基于上述论述,分别就目前扩散生成模型存在的采样步骤多、采样时间长等局限性提出了相关建议,并结合前述研究对未来扩散生成模型的发展方向进行了研判。

一种多模型的调度优化对抗攻击算法

对抗样本可通过单模型和集成模型这2种方式生成,其中集成模型生成的对抗样本往往具有更强的攻击成功率.目前集成模型的相关研究较少,现有的集成模型方式大多是在迭代中同时使用所有模型,没有合理考虑不同模型的差异问题,导致集成模型生成的对抗样本攻击成功率较低.为了进一步提高集成模型的攻击成功率,提出一种多模型的调度优化对抗攻击算法.首先通过计算各个模型的损失梯度差异进行模型的调度选择,在每轮迭代选择最优模型组合进行集成攻击得到最优梯度.其次使用前一阶段的动量项更新当前数据点,在更新后的数据点上使用当前阶段模型组合计算得到优化梯度.利用优化梯度结合变换梯度来调整得到最终梯度方向.在ImageNet数据集进行大量实验,结果表明:所提的集成算法以更少扰动得到更高的黑盒攻击成功率.与主流的全模型集成方法对比,黑盒攻击正常训练模型和经过对抗训练模型的平均成功率分别提高了3.4%和12%,且生成的对抗样本有更好的视觉效果.

基于时空特征组合模型的转辙机故障诊断

为解决转辙机故障诊断领域中存在的单一特征信息提取不足、单一诊断方法难以避免因方法局限性造成的分类误差,同时其存在一定程度的过拟合,以及泛化能力、鲁棒性不足的问题,提出了一种基于时空特征组合模型的故障诊断方法。首先,在ZYJ7电液转辙机的8种故障模式和正常模式所对应的油压曲线上提取时频域小波系数作为原始数据集,采用核主成分分析(kernel principal component analysis,KPCA)和长短期记忆网络(long short-term memory network,LSTM)提取其空间、时间特征,之后基于ADD(addition)思想构建时空特征集。其次,对卷积神经网络(convolutional neural network,CNN)、LSTM两分类器关键参数寻优后分别进行故障诊断,得到各个故障类型的概率值和误差系数。最后,利用误差倒数法对两分类器各个故障类型的概率值赋予权重,得到最终输出结果。仿真结果表明:CNN-LSTM组合模型诊断准确率达98.14%,较单一多层感知机(multi-layer perceptron,MLP)、CNN、LSTM模型准确率分别提升7.40%、5.55%、1.85%。可见此方法有效提高了转辙机诊断准确率,为集成学习模型在转辙机故障诊断领域的应用提供了一种思路。

基于目标搜寻和细节增强的水下单像素成像方法

针对当前水下单像素成像方法侧重于从整体角度重构目标图像,难以理想地恢复目标细节的问题,提出了一种基于目标搜寻和细节增强的水下单像素成像方法。目标搜寻旨在从图像中判断出目标部分和背景部分,从而增强目标信号,降低背景噪声;细节增强旨在学习采集信息的细粒度特征、增强重构图像的细节。首先用传统单像素成像方法快速重构目标图像;其次通过判断各行、各列最大像素点的差值来区别目标和背景环境;最后用基于分块模型的神经网络学习目标的细粒度特征,提高目标图像的细节部分。为了验证提出方法的可靠性,重构了空间环境和水下环境中的目标图像,实验结果表明,在两种实验环境下,该方法都可以较好地保存目标的细节信息,获得高质量的目标图像。

深度学习视域下的SPOC混合式教学模式研究--以交互设计课程为例

深度学习视域下的SPOC混合式教学模式强调培养学生的专业核心技能、自学能力及社会适应能力,注重线上线下教学内容的深化与重构,是新媒体时代教育发展的趋势。交互设计课程的混合式教学模式,以培养复合型设计人才为教学目标、以项目流程导向为教学形式、以“全场景”为教学空间、以“两境三合”为教学手段,构建以学生为主体的教学体系。输送优质的课程资源、构建多元化的教师队伍、打造个性化的学生团队、注重“多元考核”的教学评价等措施,有利于推动混合式教学模式的实施,培养新时代复合型人才,促进当代教育数字化转型。