A Deep Learning Ensemble Method for Forecasting Daily Crude Oil Price Based on Snapshot Ensemble of Transformer Model

The oil industries are an important part of a country’s economy.The crude oil’s price is influenced by a wide range of variables.Therefore,how accurately can countries predict its behavior and what predictors to employ are two main questions.In this view,we propose utilizing deep learning and ensemble learning techniques to boost crude oil’s price forecasting performance.The suggested method is based on a deep learning snapshot ensemble method of the Transformer model.To examine the superiority of the proposed model,this paper compares the proposed deep learning ensemble model against different machine learning and statistical models for daily Organization of the Petroleum Exporting Countries(OPEC)oil price forecasting.Experimental results demonstrated the outperformance of the proposed method over statistical and machine learning methods.More precisely,the proposed snapshot ensemble of Transformer method achieved relative improvement in the forecasting performance compared to autoregressive integrated moving average ARIMA(1,1,1),ARIMA(0,1,1),autoregressive moving average(ARMA)(0,1),vector autoregression(VAR),random walk(RW),support vector machine(SVM),and random forests(RF)models by 99.94%,99.62%,99.87%,99.65%,7.55%,98.38%,and 99.35%,respectively,according to mean square error metric.

Chaotic Flower Pollination with Deep Learning Based COVID-19 Classification Model

The Coronavirus Disease(COVID-19)pandemic has exposed the vulnerabilities of medical services across the globe,especially in underdeveloped nations.In the aftermath of the COVID-19 outbreak,a strong demand exists for developing novel computer-assisted diagnostic tools to execute rapid and cost-effective screenings in locations where many screenings cannot be executed using conventional methods.Medical imaging has become a crucial component in the disease diagnosis process,whereas X-rays and Computed Tomography(CT)scan imaging are employed in a deep network to diagnose the diseases.In general,four steps are followed in image-based diagnostics and disease classification processes by making use of the neural networks,such as network training,feature extraction,model performance testing and optimal feature selection.The current research article devises a Chaotic Flower Pollination Algorithm with a Deep Learning-Driven Fusion(CFPADLDF)approach for detecting and classifying COVID-19.The presented CFPA-DLDF model is developed by integrating two DL models to recognize COVID-19 in medical images.Initially,the proposed CFPA-DLDF technique employs the Gabor Filtering(GF)approach to pre-process the input images.In addition,a weighted voting-based ensemble model is employed for feature extraction,in which both VGG-19 and the MixNet models are included.Finally,the CFPA with Recurrent Neural Network(RNN)model is utilized for classification,showing the work’s novelty.A comparative analysis was conducted to demonstrate the enhanced performance of the proposed CFPADLDF model,and the results established the supremacy of the proposed CFPA-DLDF model over recent approaches.

Ensemble Deep Learning Framework for Situational Aspects-Based Annotation and Classification of International Student’s Tweets during COVID-19

As the COVID-19 pandemic swept the globe,social media plat-forms became an essential source of information and communication for many.International students,particularly,turned to Twitter to express their struggles and hardships during this difficult time.To better understand the sentiments and experiences of these international students,we developed the Situational Aspect-Based Annotation and Classification(SABAC)text mining framework.This framework uses a three-layer approach,combining baseline Deep Learning(DL)models with Machine Learning(ML)models as meta-classifiers to accurately predict the sentiments and aspects expressed in tweets from our collected Student-COVID-19 dataset.Using the pro-posed aspect2class annotation algorithm,we labeled bulk unlabeled tweets according to their contained aspect terms.However,we also recognized the challenges of reducing data’s high dimensionality and sparsity to improve performance and annotation on unlabeled datasets.To address this issue,we proposed the Volatile Stopwords Filtering(VSF)technique to reduce sparsity and enhance classifier performance.The resulting Student-COVID Twitter dataset achieved a sophisticated accuracy of 93.21%when using the random forest as a meta-classifier.Through testing on three benchmark datasets,we found that the SABAC ensemble framework performed exceptionally well.Our findings showed that international students during the pandemic faced various issues,including stress,uncertainty,health concerns,financial stress,and difficulties with online classes and returning to school.By analyzing and summarizing these annotated tweets,decision-makers can better understand and address the real-time problems international students face during the ongoing pandemic.

MDEV Model:A Novel Ensemble-Based Transfer Learning Approach for Pneumonia Classification Using CXR Images

Pneumonia is a dangerous respiratory disease due to which breathing becomes incredibly difficult and painful;thus,catching it early is crucial.Medical physicians’time is limited in outdoor situations due to many patients;therefore,automated systems can be a rescue.The input images from the X-ray equipment are also highly unpredictable due to variances in radiologists’experience.Therefore,radiologists require an automated system that can swiftly and accurately detect pneumonic lungs from chest x-rays.In medical classifications,deep convolution neural networks are commonly used.This research aims to use deep pretrained transfer learning models to accurately categorize CXR images into binary classes,i.e.,Normal and Pneumonia.The MDEV is a proposed novel ensemble approach that concatenates four heterogeneous transfer learning models:Mobile-Net,DenseNet-201,EfficientNet-B0,and VGG-16,which have been finetuned and trained on 5,856 CXR images.The evaluation matrices used in this research to contrast different deep transfer learning architectures include precision,accuracy,recall,AUC-roc,and f1-score.The model effectively decreases training loss while increasing accuracy.The findings conclude that the proposed MDEV model outperformed cutting-edge deep transfer learning models and obtains an overall precision of 92.26%,an accuracy of 92.15%,a recall of 90.90%,an auc-roc score of 90.9%,and f-score of 91.49%with minimal data pre-processing,data augmentation,finetuning and hyperparameter adjustment in classifying Normal and Pneumonia chests.

User Purchase Intention Prediction Based on Improved Deep Forest

Widely used deep neural networks currently face limitations in achieving optimal performance for purchase intention prediction due to constraints on data volume and hyperparameter selection.To address this issue,based on the deep forest algorithm and further integrating evolutionary ensemble learning methods,this paper proposes a novel Deep Adaptive Evolutionary Ensemble(DAEE)model.This model introduces model diversity into the cascade layer,allowing it to adaptively adjust its structure to accommodate complex and evolving purchasing behavior patterns.Moreover,this paper optimizes the methods of obtaining feature vectors,enhancement vectors,and prediction results within the deep forest algorithm to enhance the model’s predictive accuracy.Results demonstrate that the improved deep forest model not only possesses higher robustness but also shows an increase of 5.02%in AUC value compared to the baseline model.Furthermore,its training runtime speed is 6 times faster than that of deep models,and compared to other improved models,its accuracy has been enhanced by 0.9%.

Novel Ensemble Modeling Method for Enhancing Subset Diversity Using Clustering Indicator Vector Based on Stacked Autoencoder

A single model cannot satisfy the high-precision prediction requirements given the high nonlinearity between variables.By contrast,ensemble models can effectively solve this problem.Three key factors for improving the accuracy of ensemble models are namely the high accuracy of a submodel,the diversity between subsample sets and the optimal ensemble method.This study presents an improved ensemble modeling method to improve the prediction precision and generalization capability of the model.Our proposed method first uses a bagging algorithm to generate multiple subsample sets.Second,an indicator vector is defined to describe these subsample sets.Third,subsample sets are selected on the basis of the results of agglomerative nesting clustering on indicator vectors to maximize the diversity between subsets.Subsequently,these subsample sets are placed in a stacked autoencoder for training.Finally,XGBoost algorithm,rather than the traditional simple average ensemble method,is imported to ensemble the model during modeling.Three machine learning public datasets and atmospheric column dry point dataset from a practical industrial process show that our proposed method demonstrates high precision and improved prediction ability.

An Intelligent Hazardous Waste Detection and Classification Model Using Ensemble Learning Techniques

Proper waste management models using recent technologies like computer vision,machine learning(ML),and deep learning(DL)are needed to effectively handle the massive quantity of increasing waste.Therefore,waste classification becomes a crucial topic which helps to categorize waste into hazardous or non-hazardous ones and thereby assist in the decision making of the waste management process.This study concentrates on the design of hazardous waste detection and classification using ensemble learning(HWDC-EL)technique to reduce toxicity and improve human health.The goal of the HWDC-EL technique is to detect the multiple classes of wastes,particularly hazardous and non-hazardous wastes.The HWDC-EL technique involves the ensemble of three feature extractors using Model Averaging technique namely discrete local binary patterns(DLBP),EfficientNet,and DenseNet121.In addition,the flower pollination algorithm(FPA)based hyperparameter optimizers are used to optimally adjust the parameters involved in the EfficientNet and DenseNet121 models.Moreover,a weighted voting-based ensemble classifier is derived using three machine learning algorithms namely support vector machine(SVM),extreme learning machine(ELM),and gradient boosting tree(GBT).The performance of the HWDC-EL technique is tested using a benchmark Garbage dataset and it obtains a maximum accuracy of 98.85%.

Covid-19 Diagnosis Using a Deep Learning Ensemble Model with Chest X-Ray Images

Covid-19 is a deadly virus that is rapidly spread around the world towards the end of the 2020.The consequences of this virus are quite frightening,especially when accompanied by an underlying disease.The novelty of the virus,the constant emergence of different variants and its rapid spread have a negative impact on the control and treatment process.Although the new test kits provide almost certain results,chest X-rays are extremely important to detect the progression and degree of the disease.In addition to the Covid-19 virus,pneumonia and harmless opacity of the lungs also complicate the diagnosis.Considering the negative results caused by the virus and the treatment costs,the importance of fast and accurate diagnosis is clearly seen.In this context,deep learning methods appear as an extremely popular approach.In this study,a hybrid model design with superior properties of convolutional neural networks is presented to correctly classify the Covid-19 disease.In addition,in order to contribute to the literature,a suitable dataset with balanced case numbers that can be used in all artificial intelligence classification studies is presented.With this ensemble model design,quite remarkable results are obtained for the diagnosis of three and four-class Covid-19.The proposed model can classify normal,pneumonia,and Covid-19 with 92.6%accuracy and 82.6%for normal,pneumonia,Covid-19,and lung opacity.

Ensemble Based Learning with Accurate Motion Contrast Detection

Recent developments in computer vision applications have enabled detection of significant visual objects in video streams.Studies quoted in literature have detected objects from video streams using Spatiotemporal Particle Swarm Optimization(SPSOM)and Incremental Deep Convolution Neural Networks(IDCNN)for detecting multiple objects.However,the study considered opticalflows resulting in assessing motion contrasts.Existing methods have issue with accuracy and error rates in motion contrast detection.Hence,the overall object detection performance is reduced significantly.Thus,consideration of object motions in videos efficiently is a critical issue to be solved.To overcome the above mentioned problems,this research work proposes a method involving ensemble approaches to and detect objects efficiently from video streams.This work uses a system modeled on swarm optimization and ensemble learning called Spatiotemporal Glowworm Swarm Optimization Model(SGSOM)for detecting multiple significant objects.A steady quality in motion contrasts is maintained in this work by using Chebyshev distance matrix.The proposed system achieves global optimization in its multiple object detection by exploiting spatial/temporal cues and local constraints.Its experimental results show that the proposed system scores 4.8%in Mean Absolute Error(MAE)while achieving 86%in accuracy,81.5%in precision,85%in recall and 81.6%in F-measure and thus proving its utility in detecting multiple objects.

DCEL:classifier fusion model for Android malware detection

The rapid growth of mobile applications,the popularity of the Android system and its openness have attracted many hackers and even criminals,who are creating lots of Android malware.However,the current methods of Android malware detection need a lot of time in the feature engineering phase.Furthermore,these models have the defects of low detection rate,high complexity,and poor practicability,etc.We analyze the Android malware samples,and the distribution of malware and benign software in application programming interface(API)calls,permissions,and other attributes.We classify the software’s threat levels based on the correlation of features.Then,we propose deep neural networks and convolutional neural networks with ensemble learning(DCEL),a new classifier fusion model for Android malware detection.First,DCEL preprocesses the malware data to remove redundant data,and converts the one-dimensional data into a two-dimensional gray image.Then,the ensemble learning approach is used to combine the deep neural network with the convolutional neural network,and the final classification results are obtained by voting on the prediction of each single classifier.Experiments based on the Drebin and Malgenome datasets show that compared with current state-of-art models,the proposed DCEL has a higher detection rate,higher recall rate,and lower computational cost.

Attenuate Class Imbalance Problem for Pneumonia Diagnosis Using Ensemble Parallel Stacked Pre-Trained Models

Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.

DC2Net:An Asian Soybean Rust Detection Model Based on Hyperspectral Imaging and Deep Learning

Asian soybean rust(ASR)is one of the major diseases that causes serious yield loss worldwide,even up to 80%.Early and accurate detection of ASR is critical to reduce economic losses.Hyperspectral imaging,combined with deep learning,has already been proved as a powerful tool to detect crop diseases.However,current deep learning models are limited to extract both spatial and spectral features in hyperspectral images due to the use of fixed geometric structure of the convolutional kernels,leading to the fact that the detection accuracy of current models remains further improvement.

Locally Linear Back-propagation Based Contribution for Nonlinear Process Fault Diagnosis

This paper proposes a novel locally linear backpropagation based contribution(LLBBC) for nonlinear process fault diagnosis. As a method based on the deep learning model of auto-encoder(AE), LLBBC can deal with the fault diagnosis problem through extracting nonlinear features. When the on-line fault diagnosis task is in progress, a locally linear model is firstly built at the current fault sample. According to the basic idea of reconstruction based contribution(RBC), the propagation of fault information is described by using back-propagation(BP) algorithm. Then, a contribution index is established to measure the correlation between the variable and the fault, and the final diagnosis result is obtained by searching variables with large contributions. The smearing effect, which is an important factor affecting the performance of fault diagnosis, can be suppressed as well,and the theoretical analysis reveals that the correct diagnosis can be guaranteed by LLBBC. Finally, the feasibility and effectiveness of the proposed method are verified through a nonlinear numerical example and the Tennessee Eastman benchmark process.

Masked Face Recognition Using MobileNet V2 with Transfer Learning

Corona virus(COVID-19)is once in a life time calamity that has resulted in thousands of deaths and security concerns.People are using face masks on a regular basis to protect themselves and to help reduce corona virus transmission.During the on-going coronavirus outbreak,one of the major priorities for researchers is to discover effective solution.As important parts of the face are obscured,face identification and verification becomes exceedingly difficult.The suggested method is a transfer learning using MobileNet V2 based technology that uses deep feature such as feature extraction and deep learning model,to identify the problem of face masked identification.In the first stage,we are applying face mask detector to identify the face mask.Then,the proposed approach is applying to the datasets from Canadian Institute for Advanced Research10(CIFAR10),Modified National Institute of Standards and Technology Database(MNIST),Real World Masked Face Recognition Database(RMFRD),and Stimulated Masked Face Recognition Database(SMFRD).The proposed model is achieving recognition accuracy 99.82%with proposed dataset.This article employs the four pre-programmed models VGG16,VGG19,ResNet50 and ResNet101.To extract the deep features of faces with VGG16 is achieving 99.30%accuracy,VGG19 is achieving 99.54%accuracy,ResNet50 is achieving 78.70%accuracy and ResNet101 is achieving 98.64%accuracy with own dataset.The comparative analysis shows,that our proposed model performs better result in all four previous existing models.The fundamental contribution of this study is to monitor with face mask and without face mask to decreases the pace of corona virus and to detect persons using wearing face masks.

A Hybrid Deep Learning Approach to Classify the Plant Leaf Species

Many plant species have a startling degree of morphological similarity,making it difficult to split and categorize them reliably.Unknown plant species can be challenging to classify and segment using deep learning.While using deep learning architectures has helped improve classification accuracy,the resulting models often need to be more flexible and require a large dataset to train.For the sake of taxonomy,this research proposes a hybrid method for categorizing guava,potato,and java plumleaves.Two new approaches are used to formthe hybridmodel suggested here.The guava,potato,and java plum plant species have been successfully segmented using the first model built on the MobileNetV2-UNET architecture.As a second model,we use a Plant Species Detection Stacking Ensemble Deep Learning Model(PSD-SE-DLM)to identify potatoes,java plums,and guava.The proposed models were trained using data collected in Punjab,Pakistan,consisting of images of healthy and sick leaves from guava,java plum,and potatoes.These datasets are known as PLSD and PLSSD.Accuracy levels of 99.84%and 96.38%were achieved for the suggested PSD-SE-DLM and MobileNetV2-UNET models,respectively.

A spatiotemporal 3D convolutional neural network model for ENSO predictions: A test case for the 2020/21 La Niña conditions

2020–22年间热带太平洋经历了持续性多年的拉尼娜事件,多数耦合模式都难以准确预测其演变过程,这为厄尔尼诺-南方涛动(ENSO)的实时预测带来了很大的挑战.同时,目前学术界对此次持续性双拉尼娜事件的发展仍缺乏合理的物理解释,其所涉及的物理过程和机制有待于进一步分析.本研究利用再分析数据产品分析了热带东南太平洋东南风异常及其引起的次表层海温异常在此次热带太平洋海表温度(SST)异常演变中的作用,并构建了一个时空分离(Time-Space)的三维(3D)卷积神经网络模型(TS-3DCNN)对此次双拉尼娜事件进行实时预测和过程分析.通过将TS-3DCNN与中国科学院海洋研究所(IOCAS)中等复杂程度海气耦合模式(IOCAS ICM)的预测结果对比,表明TS-3DCNN模型对2020–22年双重拉尼娜现象的预测能力与IOCAS ICM相当,二者均能够从2021年初的初始场开始较好地预测2021年末El Niño3.4区SST的演变.此外,基于TS-3DCNN和IOCAS ICM的敏感性试验也验证了赤道外风场异常和次表层海温异常在2021年末赤道中东太平洋海表二次变冷过程中的关键作用.未来将神经网络与动力模式模式间的有效结合,进一步发展神经网络与物理过程相结合的混合建模是进一步提高ENSO事件预测能力的有效途径.

Multi-Model Ensemble Deep Learning Method to Diagnose COVID-19 Using Chest Computed Tomography Images

Deep learning based analyses of computed tomography(CT)images contribute to automated diagnosis of COVID-19,and ensemble learning may commonly provide a better solution.Here,we proposed an ensemble learning method that integrates several component neural networks to jointly diagnose COVID-19.Two ensemble strategies are considered:the output scores of all component models that are combined with the weights adjusted adaptively by cost function back propagation;voting strategy.A database containing 8347 CT slices of COVID-19,common pneumonia and normal subjects was used as training and testing sets.Results show that the novel method can reach a high accuracy of 99.37%(recall:0.9981;precision:0.9893),with an increase of about 7% in comparison to single-component models.And the average test accuracy is 95.62%(recall:0.9587;precision:0.9559),with a corresponding increase of 5.2%.Compared with several latest deep learning models on the identical test set,our method made an accuracy improvement up to 10.88%.The proposed method may be a promising solution for the diagnosis of COVID-19.

An Observation Data Driven Simulation and Analysis Framework for Early Stage <i>C. elegans</i>Embryogenesis

Recent developments in cutting-edge live microscopy and image analysis provide a unique opportunity to systematically investigate individual cell’s dynamics as well as simulation-based hypothesis testing. After a summary of data generation and analysis in the observation and modeling efforts related to C. elegans embryogenesis, we develop a systematic approach to model the basic behaviors of individual cells. Next, we present our ideas to model cell fate, division, and movement using 3D time-lapse images within an agent-based modeling framework. Then, we summarize preliminary result and discuss efforts in cell fate, division, and movement modeling. Finally, we discuss the ongoing efforts and future directions for C. elegans embryo modeling, including an inferred developmental landscape for cell fate, a quasi-equilibrium model for cell division, and multi-agent, deep reinforcement learning for cell movement.

采用集成深度森林模型实现退役电池容量估算

针对退役锂离子电池容量估算面临历史数据缺失,传统机器学习算法存在过拟合和单个模型估算不稳定的问题,提出一种基于集成深度森林的容量估算模型。首先,从退役电池一次满充数据中提取恒流充电时间和充电电流面积特征;然后,利用提取的特征和容量训练多个深度森林建立集成深度森林模型,并设计一种可信状态决策剔除集成模型中波动较大的估算值,取剩余估算值平均值作为最终估算结果。采用自测和公开数据集对所提方法进行验证,结果表明,该方法能实现退役电池剩余容量的准确稳定估算,最大误差仅为0.08 Ah,与传统机器学习算法相比,该方法能获得更高的容量估算精度。

基于集成学习与深度学习的洪水径流预报研究

深度学习模型凭借其对水文因素间复杂作用的优秀处理能力,在水文预报领域得到了一定的应用,然而,针对集成学习与深度学习耦合模型的研究仍有所缺失。通过融合集成学习AdaBoost算法与深度学习Informer模型,提出了一种组合模型,称为AdaBoost-Informer模型,以提高洪水径流预报的精度。该模型以历史雨量和径流数据作为数据输入,将具备长时序依赖捕获能力的Informer作为集成学习的弱预测器,使用网格搜索法进行超参数调优,使用AdaBoost集成学习算法对弱预测器进行加权组合得到强预测器。在浙江省椒江流域的应用分析表明:对比Random Forest、AdaBoost、Transformer、Informer等模型,AdaBoost-Informer模型表现最佳,RMSE为62.08 m^(3)/s,MAE为23.83 m^(3)/s,NSE为0.980,预报合格率为100%。所提模型可有效提高洪水预报精度,为防汛抢险和防洪系统调度提供决策依据。