A Deep Learning Ensemble Method for Forecasting Daily Crude Oil Price Based on Snapshot Ensemble of Transformer Model

The oil industries are an important part of a country’s economy.The crude oil’s price is influenced by a wide range of variables.Therefore,how accurately can countries predict its behavior and what predictors to employ are two main questions.In this view,we propose utilizing deep learning and ensemble learning techniques to boost crude oil’s price forecasting performance.The suggested method is based on a deep learning snapshot ensemble method of the Transformer model.To examine the superiority of the proposed model,this paper compares the proposed deep learning ensemble model against different machine learning and statistical models for daily Organization of the Petroleum Exporting Countries(OPEC)oil price forecasting.Experimental results demonstrated the outperformance of the proposed method over statistical and machine learning methods.More precisely,the proposed snapshot ensemble of Transformer method achieved relative improvement in the forecasting performance compared to autoregressive integrated moving average ARIMA(1,1,1),ARIMA(0,1,1),autoregressive moving average(ARMA)(0,1),vector autoregression(VAR),random walk(RW),support vector machine(SVM),and random forests(RF)models by 99.94%,99.62%,99.87%,99.65%,7.55%,98.38%,and 99.35%,respectively,according to mean square error metric.

Micro-expression recognition algorithm based on graph convolutional network and Transformer model

Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Vehicle Density Prediction in Low Quality Videos with Transformer Timeseries Prediction Model(TTPM)

Recent advancement in low-cost cameras has facilitated surveillance in various developing towns in India.The video obtained from such surveillance are of low quality.Still counting vehicles from such videos are necessity to avoid traf-fic congestion and allows drivers to plan their routes more precisely.On the other hand,detecting vehicles from such low quality videos are highly challenging with vision based methodologies.In this research a meticulous attempt is made to access low-quality videos to describe traffic in Salem town in India,which is mostly an un-attempted entity by most available sources.In this work profound Detection Transformer(DETR)model is used for object(vehicle)detection.Here vehicles are anticipated in a rush-hour traffic video using a set of loss functions that carry out bipartite coordinating among estimated and information acquired on real attributes.Every frame in the traffic footage has its date and time which is detected and retrieved using Tesseract Optical Character Recognition.The date and time extricated and perceived from the input image are incorporated with the length of the recognized objects acquired from the DETR model.This furnishes the vehicles report with timestamp.Transformer Timeseries Prediction Model(TTPM)is proposed to predict the density of the vehicle for future prediction,here the regular NLP layers have been removed and the encoding temporal layer has been modified.The proposed TTPM error rate outperforms the existing models with RMSE of 4.313 and MAE of 3.812.

CT-NET: A Novel Convolutional Transformer-Based Network for Short-Term Solar Energy Forecasting Using Climatic Information

Photovoltaic(PV)systems are environmentally friendly,generate green energy,and receive support from policies and organizations.However,weather fluctuations make large-scale PV power integration and management challenging despite the economic benefits.Existing PV forecasting techniques(sequential and convolutional neural networks(CNN))are sensitive to environmental conditions,reducing energy distribution system performance.To handle these issues,this article proposes an efficient,weather-resilient convolutional-transformer-based network(CT-NET)for accurate and efficient PV power forecasting.The network consists of three main modules.First,the acquired PV generation data are forwarded to the pre-processing module for data refinement.Next,to carry out data encoding,a CNNbased multi-head attention(MHA)module is developed in which a single MHA is used to decode the encoded data.The encoder module is mainly composed of 1D convolutional and MHA layers,which extract local as well as contextual features,while the decoder part includes MHA and feedforward layers to generate the final prediction.Finally,the performance of the proposed network is evaluated using standard error metrics,including the mean squared error(MSE),root mean squared error(RMSE),and mean absolute percentage error(MAPE).An ablation study and comparative analysis with several competitive state-of-the-art approaches revealed a lower error rate in terms of MSE(0.0471),RMSE(0.2167),and MAPE(0.6135)over publicly available benchmark data.In addition,it is demonstrated that our proposed model is less complex,with the lowest number of parameters(0.0135 M),size(0.106 MB),and inference time(2 ms/step),suggesting that it is easy to integrate into the smart grid.

Model Transformer Evaluation of High-Permeability Grain-Oriented Electrical Steels

The dependence of transformer performance on the material properties was investigated using two laboratory-processed 0.23 mm thick grain-oriented electrical steels domain-refined with elec-trolytically etched grooves having different magnetic properties. The iron loss at 1.7 T, 50 Hz and the flux density at 800 A/m of material A were 0.73 W/kg and 1.89 T, respectively; and those of material B, 0.83 W/kg and 1.88 T. Model stacked and wound transformer core experiments using the tested materials exhibited performance well reflecting the material characteristics. In a three-phase stacked core with step-lap joints excited to 1.7 T, 50 Hz, the core loss, the exciting current and the noise level were 0.86 W/kg, 0.74 A and 52 dB, respectively, with material A; and 0.97 W/kg, 1.0 A and 54 dB with material B. The building factors for the core losses of the two materials were almost the same in both core configurations. The effect of higher harmonics on transformer performance was also investigated.

A Study on the Impact of Voice-to-Text Technology on Academic Achievement of the Hearing-Impaired

Hearing loss is a significant barrier to academic achievement,with hearing-impaired(HI)individuals often facing challenges in speech recognition,language development,and social interactions.Lip-reading,a crucial skill for HI individuals,is essential for effective communication and learning.However,the COVID-19 pandemic has exacerbated the challenges faced by HI individuals,with the face masks hindering lip-reading.This literature review explores the relationship between hearing loss and academic achievement,highlighting the importance of lip-reading and the potential of artificial intelligence(AI)techniques in mitigating these challenges.The introduction of Voice-to-Text(VtT)technology,which provides real-time text captions,can significantly improve speech recognition and academic performance for HI students.AI models,such as Hidden Markov models and Transformer models,can enhance the accuracy and robustness of VtT technology in diverse educational settings.Furthermore,VtT technology can facilitate better teacher-student interactions,provide transcripts of lectures and classroom discussions,and bridge the gap in standardized testing performance between HI and hearing students.While challenges and limitations exist,the successful implementation of VtT technology can promote inclusive education and enhance academic achievement.Future research directions include popularizing VtT technology,addressing technological barriers,and customizing VtT systems to cater to individual needs.

Real-time predictions of the 2023–2024 climate conditions in the tropical Pacific using a purely data-driven Transformer model

Following triple La Ni?a events during 2020–2022,the future evolution of climate conditions over the tropical Pacific has been a focused interest in ENSO-related communities.Observations and modeling studies indicate that an El Ni?o event is occurring in 2023;however,large uncertainties remain in terms of its detailed evolution,and the factors affecting its resultant amplitude remain to be understood.Here,a novel deep learning-based Transformer model is adopted to make real-time predictions for the 2023–2024 climate conditions in the tropical Pacific.Several key fields vital to the El Ni?o and Southern Oscillation(ENSO)in the tropical Pacific are collectively and simultaneously utilized in model training and in making predictions;therefore,this purely data-driven model is configured in both training and predicting procedures such that the coupled ocean-atmosphere interactions are adequately represented.Also similar to dynamic models,the prediction procedure is executed in a rolling manner to allow ocean-atmosphere anomaly exchanges month by month;the related key fields during multi-month time intervals(TIs)prior to prediction target months are taken as input predictors,serving as initial conditions to precondition the future evolution more effectively.Real-time predictions indicate that the climate conditions in the tropical Pacific are surely to develop into an El Ni?o state in late 2023.Furthermore,sensitivity experiments are conducted to examine how prediction skills are affected by the input predictor specifications,including TIs during which information on initial conditions is retained for making predictions.A comparison with other dynamic coupled models is also made to demonstrate the prediction performance for the 2023–2024 El Ni?o event.

Integrated Modelling of Microstructure Evolution and Mechanical Properties Prediction for Q&P Hot Stamping Process of Ultra‑High Strength Steel

High strength steel products with good ductility can be produced via Q&P hot stamping process,while the phase transformation of the process is more complicated than common hot stamping since two-step quenching and one-step carbon partitioning processes are involved.In this study,an integrated model of microstructure evolution relating to Q&P hot stamping was presented with a persuasively predicted results of mechanical properties.The transformation of diffusional phase and non-diffusional phase,including original austenite grain size individually,were considered,as well as the carbon partitioning process which affects the secondary martensite transformation temperature and the subsequent phase transformations.Afterwards,the mechanical properties including hardness,strength,and elongation were calculated through a series of theoretical and empirical models in accordance with phase contents.Especially,a modified elongation prediction model was generated ultimately with higher accuracy than the existed Mileiko’s model.In the end,the unified model was applied to simulate the Q&P hot stamping process of a U-cup part based on the finite element software LS-DYNA,where the calculated outputs were coincident with the measured consequences.

Formal Verification of TASM Models by Translating into UPPAAL

Timed abstract state machine(TASM) is a formal specification language used to specify and simulate the behavior of real-time systems. Formal verification of TASM model can be fulfilled through model checking activities by translating into UPPAAL. Firstly, the translational semantics from TASM to UPPAAL is presented through atlas transformation language(ATL). Secondly, the implementation of the proposed model transformation tool TASM2UPPAAL is provided. Finally, a case study is given to illustrate the automatic transformation from TASM model to UPPAAL model.

HOSVD-based LPV modeling and mixed robust H_2/H_∞ control design for air-breathing hypersonic vehicle

This paper focuses on synthesizing a mixed robust H_2/H_∞ linear parameter varying（LPV） controller for the longitudinal motion of an air-breathing hypersonic vehicle via a high order singular value decomposition（HOSVD） approach.The design of hypersonic flight control systems is highly challenging due to the enormous complexity of the vehicle dynamics and the presence of significant uncertainties.Motivated by recent results on both LPV control and tensor-product（TP） model transformation approach,the velocity and altitude tracking control problems for the air-breathing hypersonic vehicle is reduced to that of a state feedback stabilizing controller design for a polytopic LPV system with guaranteed performances.The controller implementation is converted into a convex optimization problem with parameterdependent linear matrix inequalities（LMIs） constraints,which is intuitively tractable using LMI control toolbox.Finally,numerical simulation results demonstrate the effectiveness of the proposed approach.

Modeling waves and longshore transport potential in Half Moon Bay,Grays Harbor,Washington,USA

A local-scale phase-resolving wave transformation model with CGWAVE is established in connection with a regional-scale coupled STWAVE-ADCIRC wave-current model for its application in the Half Moon Bay, Grays Harbor.Wave transformation from offshore to the harbor entrance is simulated by the STWAVE model which includes wave-current interaction.The STWAVE results provide incident wave conditions for the local-scale CGWAVE model at its outer boundary. A simple method is developed to take into account the lateral variation of wave height in constructing the model’s wave boundary conditions.The model was validated for three wave condition cases which yielded good agreement with field data.The validated model was applied to predicting nearshore waves in the Half Moon Bay and longshore transport parameters along the wave breaking line for the existing condition and three engineering alternatives. A comparative analysis indicated that storm waves that have a combination of long period and large height are the most destructive to the crenulate shoreline in the Half Moon Bay; both 152 m jetty extension (Alt. 2) and diffraction mound enlargement (Alt. 3) would significantly reduce breaking wave height and longshore transport potential in the southwest corner of Half Moon Bay.

NC Machining of Spiral Bevel Gear and Hypoid Gear Based on Unity Transformation Model

A unity transformation model (UTM) was presented for flexible NC machining of spiral bevel gears and hypoid gears. The model can support various machining methods for Gleason spiral bevel gears and hypoid gears, including generation machining and formation machining for wheel or pinion on a universal five-axis machining center, and then directly produce NC codes for the selected machining method. Wheel machining and pinion machining under UTM were simulated in Vericut 6.0 and tested on a five-axis machining center TDNC-W2000 with NC unit TDNC-H8. The results from simulation and real-cut verify the feasibility of gear machining under UTM as well as the correctness of NC codes.

MoTransFrame:Model Transfer Framework for CNNs on Low-Resource Edge Computing Node

Deep learning technology has been widely used in computer vision,speech recognition,natural language processing,and other related fields.The deep learning algorithm has high precision and high reliability.However,the lack of resources in the edge terminal equipment makes it difficult to run deep learning algorithms that require more memory and computing power.In this paper,we propose MoTransFrame,a general model processing framework for deep learning models.Instead of designing a model compression algorithm with a high compression ratio,MoTransFrame can transplant popular convolutional neural networks models to resources-starved edge devices promptly and accurately.By the integration method,Deep learning models can be converted into portable projects for Arduino,a typical edge device with limited resources.Our experiments show that MoTransFrame has good adaptability in edge devices with limited memories.It is more flexible than other model transplantation methods.It can keep a small loss of model accuracy when the number of parameters is compressed by tens of times.At the same time,the computational resources needed in the reasoning process are less than what the edge node could handle.

A Model Transformation Approach for Detecting Distancing Violations in Weighted Graphs

This work presents the design of an Internet of Things(IoT)edge-based system based on model transformation and complete weighted graph to detect violations of social distancing measures in indoor public places.Awireless sensor network based on Bluetooth Low Energy is introduced as the infrastructure of the proposed design.A hybrid model transformation strategy for generating a graph database to represent groups of people is presented as a core middleware layer of the detecting system’s proposed architectural design.A Neo4j graph database is used as a target implementation generated from the proposed transformational system to store all captured real-time IoT data about the distances between individuals in an indoor area and answer user predefined queries,expressed using Neo4j Cypher,to provide insights from the stored data for decision support.As proof of concept,a discrete-time simulation model was adopted for the design of a COVID-19 physical distancing measures case study to evaluate the introduced system architecture.Twenty-one weighted graphs were generated randomly and the degrees of violation of distancing measures were inspected.The experimental results demonstrate the capability of the proposed system design to detect violations of COVID-19 physical distancing measures within an enclosed area.

AADL2TASM: a Verification and Analysis Tool for AADL Models

Architecture analysis and design language （AADL） is an architecture description language standard for embedded real-time systems and it is widely used in safety-critical applications. For facilitating verifcafion and analysis, model transformation is one of the methods. A synchronous subset of AADL and a general methodology for translating the AADL subset into timed abstract state machine （TASM） were studied. Based on the arias transformation language （ ATL ） framework, the associated translating tool AADL2TASM was implemented by defining the meta-model of both AADL and TASM, and the ATL transformation rules. A case study with property verification of the AADL model was also presented for validating the tool.

Asymptotic Efficiency of the Maximum Likelihood Estimator for the Box-Cox Transformation Model with Heteroscedastic Disturbances

This paper considers the asymptotic efficiency of the maximum likelihood estimator (MLE) for the Box-Cox transformation model with heteroscedastic disturbances. The MLE under the normality assumption (BC MLE) is a consistent and asymptotically efficient estimator if the “small ” condition is satisfied and the number of parameters is finite. However, the BC MLE cannot be asymptotically efficient and its rate of convergence is slower than ordinal order when the number of parameters goes to infinity. Anew consistent estimator of order is proposed. One important implication of this study is that estimation methods should be carefully chosen when the model contains many parameters in actual empirical studies.

Evaluation of deep learning algorithms for landslide susceptibility mapping in an alpine-gorge area:a case study in Jiuzhaigou County

With its high mountains,deep valleys,and complex geological formations,the Jiuzhaigou County has the typical characteristics of a disaster-prone mountainous region in southwestern China.On August 8,2017,a strong Ms 7.0 earthquake occurred in this region,causing some of the mountains in the area to become loose and cracked.Therefore,a survey and evaluation of landslides in this area can help to reveal hazards and take effective measures for subsequent disaster management.However,different evaluation models can yield different spatial distributions of landslide susceptibility,and thus,selecting the appropriate model and performing the optimal combination of parameters is the most effective way to improve susceptibility evaluation.In order to construct an evaluation indicator system suitable for Jiuzhaigou County,we extracted 12 factors affecting the occurrence of landslides,including slope,elevation and slope surface,and made samples.At the core of the transformer model is a self-attentive mechanism that enables any two of the features to be interlinked,after which feature extraction is performed via a forward propagation network(FFN).We exploited its coding structure to transform it into a deep learning model that is more suitable for landslide susceptibility evaluation.The results show that the transformer model has the highest accuracy(86.89%),followed by the random forest and support vector machine models(84.47%and 82.52%,respectively),and the logistic regression model achieves the lowest accuracy(79.61%).Accordingly,this deep learning model provides a new tool to achieve more accurate zonation of landslide susceptibility in Jiuzhaigou County.

Fake News Encoder Classifier (FNEC) for Online Published News Related to COVID-19 Vaccines

In the past few years,social media and online news platforms have played an essential role in distributing news content rapidly.Consequently.verification of the authenticity of news has become a major challenge.During the COVID-19 outbreak,misinformation and fake news were major sources of confusion and insecurity among the general public.In the first quarter of the year 2020,around 800 people died due to fake news relevant to COVID-19.The major goal of this research was to discover the best learning model for achieving high accuracy and performance.A novel case study of the Fake News Classification using ELECTRA model,which achieved 85.11%accuracy score,is thus reported in this manuscript.In addition to that,a new novel dataset called COVAX-Reality containing COVID-19 vaccine-related news has been contributed.Using the COVAX-Reality dataset,the performance of FNEC is compared to several traditional learning models i.e.,Support Vector Machine(SVM),Naive Bayes(NB),Passive Aggressive Classifier(PAC),Long Short-Term Memory(LSTM),Bi-directional LSTM(Bi-LSTM)and Bi-directional Encoder Representations from Transformers(BERT).For the evaluation of FNEC,standard metrics(Precision,Recall,Accuracy,and F1-Score)were utilized.

Multi-parametric Sensitivity Analysis of Improved Transformer Thermal Models Considering Nonlinear Effect of Oil Time Constant

This paper aims to develop an approach to investigating the effect of a particular parameter on the output accuracy of transformer thermal models,i.e.sensitivity analysis,which can not only reveal the most sensitive parameter of a thermal model but also improve model output accuracies.For the first time,the nonlinear time constant(NTC)of transformer oil is proposed to reshape three practical top-oil temperature models based on an expression of nonlinear thermal conductance:the modified IEEE clause 7 model,Swift’s model,and Susa’s model.Then,the multi-parametric sensitivity analysis(MPSA)is undertaken to reveal the effect of each parameter on the model output accuracy.Through onsite data validation,the results show that the accuracy performance of the proposed NTC thermal models are improved significantly by considering the nonlinear effect of oil time constant.Moreover,the derived sensitivity performances can clearly reveal the most dominant parameter of the model,so as to simplify the model parameter identification process by reducing the number of insensitive parameters.Finally,the heat-run test data is used as a reference to validate parameters optimized through a genetic algorithm(GA),which demonstrates that the proposed NTC IEEE model has not only one sensitive parameter but also superior accuracy performance.