CT-NET: A Novel Convolutional Transformer-Based Network for Short-Term Solar Energy Forecasting Using Climatic Information

Photovoltaic(PV)systems are environmentally friendly,generate green energy,and receive support from policies and organizations.However,weather fluctuations make large-scale PV power integration and management challenging despite the economic benefits.Existing PV forecasting techniques(sequential and convolutional neural networks(CNN))are sensitive to environmental conditions,reducing energy distribution system performance.To handle these issues,this article proposes an efficient,weather-resilient convolutional-transformer-based network(CT-NET)for accurate and efficient PV power forecasting.The network consists of three main modules.First,the acquired PV generation data are forwarded to the pre-processing module for data refinement.Next,to carry out data encoding,a CNNbased multi-head attention(MHA)module is developed in which a single MHA is used to decode the encoded data.The encoder module is mainly composed of 1D convolutional and MHA layers,which extract local as well as contextual features,while the decoder part includes MHA and feedforward layers to generate the final prediction.Finally,the performance of the proposed network is evaluated using standard error metrics,including the mean squared error(MSE),root mean squared error(RMSE),and mean absolute percentage error(MAPE).An ablation study and comparative analysis with several competitive state-of-the-art approaches revealed a lower error rate in terms of MSE(0.0471),RMSE(0.2167),and MAPE(0.6135)over publicly available benchmark data.In addition,it is demonstrated that our proposed model is less complex,with the lowest number of parameters(0.0135 M),size(0.106 MB),and inference time(2 ms/step),suggesting that it is easy to integrate into the smart grid.

A Deep Learning Ensemble Method for Forecasting Daily Crude Oil Price Based on Snapshot Ensemble of Transformer Model

The oil industries are an important part of a country’s economy.The crude oil’s price is influenced by a wide range of variables.Therefore,how accurately can countries predict its behavior and what predictors to employ are two main questions.In this view,we propose utilizing deep learning and ensemble learning techniques to boost crude oil’s price forecasting performance.The suggested method is based on a deep learning snapshot ensemble method of the Transformer model.To examine the superiority of the proposed model,this paper compares the proposed deep learning ensemble model against different machine learning and statistical models for daily Organization of the Petroleum Exporting Countries(OPEC)oil price forecasting.Experimental results demonstrated the outperformance of the proposed method over statistical and machine learning methods.More precisely,the proposed snapshot ensemble of Transformer method achieved relative improvement in the forecasting performance compared to autoregressive integrated moving average ARIMA(1,1,1),ARIMA(0,1,1),autoregressive moving average(ARMA)(0,1),vector autoregression(VAR),random walk(RW),support vector machine(SVM),and random forests(RF)models by 99.94%,99.62%,99.87%,99.65%,7.55%,98.38%,and 99.35%,respectively,according to mean square error metric.

Micro-expression recognition algorithm based on graph convolutional network and Transformer model

Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.

Vehicle Density Prediction in Low Quality Videos with Transformer Timeseries Prediction Model(TTPM)

Recent advancement in low-cost cameras has facilitated surveillance in various developing towns in India.The video obtained from such surveillance are of low quality.Still counting vehicles from such videos are necessity to avoid traf-fic congestion and allows drivers to plan their routes more precisely.On the other hand,detecting vehicles from such low quality videos are highly challenging with vision based methodologies.In this research a meticulous attempt is made to access low-quality videos to describe traffic in Salem town in India,which is mostly an un-attempted entity by most available sources.In this work profound Detection Transformer(DETR)model is used for object(vehicle)detection.Here vehicles are anticipated in a rush-hour traffic video using a set of loss functions that carry out bipartite coordinating among estimated and information acquired on real attributes.Every frame in the traffic footage has its date and time which is detected and retrieved using Tesseract Optical Character Recognition.The date and time extricated and perceived from the input image are incorporated with the length of the recognized objects acquired from the DETR model.This furnishes the vehicles report with timestamp.Transformer Timeseries Prediction Model(TTPM)is proposed to predict the density of the vehicle for future prediction,here the regular NLP layers have been removed and the encoding temporal layer has been modified.The proposed TTPM error rate outperforms the existing models with RMSE of 4.313 and MAE of 3.812.

Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

A Study on the Impact of Voice-to-Text Technology on Academic Achievement of the Hearing-Impaired

Hearing loss is a significant barrier to academic achievement,with hearing-impaired(HI)individuals often facing challenges in speech recognition,language development,and social interactions.Lip-reading,a crucial skill for HI individuals,is essential for effective communication and learning.However,the COVID-19 pandemic has exacerbated the challenges faced by HI individuals,with the face masks hindering lip-reading.This literature review explores the relationship between hearing loss and academic achievement,highlighting the importance of lip-reading and the potential of artificial intelligence(AI)techniques in mitigating these challenges.The introduction of Voice-to-Text(VtT)technology,which provides real-time text captions,can significantly improve speech recognition and academic performance for HI students.AI models,such as Hidden Markov models and Transformer models,can enhance the accuracy and robustness of VtT technology in diverse educational settings.Furthermore,VtT technology can facilitate better teacher-student interactions,provide transcripts of lectures and classroom discussions,and bridge the gap in standardized testing performance between HI and hearing students.While challenges and limitations exist,the successful implementation of VtT technology can promote inclusive education and enhance academic achievement.Future research directions include popularizing VtT technology,addressing technological barriers,and customizing VtT systems to cater to individual needs.

HOSVD-based LPV modeling and mixed robust H_2/H_∞ control design for air-breathing hypersonic vehicle

This paper focuses on synthesizing a mixed robust H_2/H_∞ linear parameter varying（LPV） controller for the longitudinal motion of an air-breathing hypersonic vehicle via a high order singular value decomposition（HOSVD） approach.The design of hypersonic flight control systems is highly challenging due to the enormous complexity of the vehicle dynamics and the presence of significant uncertainties.Motivated by recent results on both LPV control and tensor-product（TP） model transformation approach,the velocity and altitude tracking control problems for the air-breathing hypersonic vehicle is reduced to that of a state feedback stabilizing controller design for a polytopic LPV system with guaranteed performances.The controller implementation is converted into a convex optimization problem with parameterdependent linear matrix inequalities（LMIs） constraints,which is intuitively tractable using LMI control toolbox.Finally,numerical simulation results demonstrate the effectiveness of the proposed approach.

Model Transformer Evaluation of High-Permeability Grain-Oriented Electrical Steels

The dependence of transformer performance on the material properties was investigated using two laboratory-processed 0.23 mm thick grain-oriented electrical steels domain-refined with elec-trolytically etched grooves having different magnetic properties. The iron loss at 1.7 T, 50 Hz and the flux density at 800 A/m of material A were 0.73 W/kg and 1.89 T, respectively; and those of material B, 0.83 W/kg and 1.88 T. Model stacked and wound transformer core experiments using the tested materials exhibited performance well reflecting the material characteristics. In a three-phase stacked core with step-lap joints excited to 1.7 T, 50 Hz, the core loss, the exciting current and the noise level were 0.86 W/kg, 0.74 A and 52 dB, respectively, with material A; and 0.97 W/kg, 1.0 A and 54 dB with material B. The building factors for the core losses of the two materials were almost the same in both core configurations. The effect of higher harmonics on transformer performance was also investigated.

Modeling waves and longshore transport potential in Half Moon Bay,Grays Harbor,Washington,USA

A local-scale phase-resolving wave transformation model with CGWAVE is established in connection with a regional-scale coupled STWAVE-ADCIRC wave-current model for its application in the Half Moon Bay, Grays Harbor.Wave transformation from offshore to the harbor entrance is simulated by the STWAVE model which includes wave-current interaction.The STWAVE results provide incident wave conditions for the local-scale CGWAVE model at its outer boundary. A simple method is developed to take into account the lateral variation of wave height in constructing the model’s wave boundary conditions.The model was validated for three wave condition cases which yielded good agreement with field data.The validated model was applied to predicting nearshore waves in the Half Moon Bay and longshore transport parameters along the wave breaking line for the existing condition and three engineering alternatives. A comparative analysis indicated that storm waves that have a combination of long period and large height are the most destructive to the crenulate shoreline in the Half Moon Bay; both 152 m jetty extension (Alt. 2) and diffraction mound enlargement (Alt. 3) would significantly reduce breaking wave height and longshore transport potential in the southwest corner of Half Moon Bay.

Evaluation of deep learning algorithms for landslide susceptibility mapping in an alpine-gorge area:a case study in Jiuzhaigou County

With its high mountains,deep valleys,and complex geological formations,the Jiuzhaigou County has the typical characteristics of a disaster-prone mountainous region in southwestern China.On August 8,2017,a strong Ms 7.0 earthquake occurred in this region,causing some of the mountains in the area to become loose and cracked.Therefore,a survey and evaluation of landslides in this area can help to reveal hazards and take effective measures for subsequent disaster management.However,different evaluation models can yield different spatial distributions of landslide susceptibility,and thus,selecting the appropriate model and performing the optimal combination of parameters is the most effective way to improve susceptibility evaluation.In order to construct an evaluation indicator system suitable for Jiuzhaigou County,we extracted 12 factors affecting the occurrence of landslides,including slope,elevation and slope surface,and made samples.At the core of the transformer model is a self-attentive mechanism that enables any two of the features to be interlinked,after which feature extraction is performed via a forward propagation network(FFN).We exploited its coding structure to transform it into a deep learning model that is more suitable for landslide susceptibility evaluation.The results show that the transformer model has the highest accuracy(86.89%),followed by the random forest and support vector machine models(84.47%and 82.52%,respectively),and the logistic regression model achieves the lowest accuracy(79.61%).Accordingly,this deep learning model provides a new tool to achieve more accurate zonation of landslide susceptibility in Jiuzhaigou County.

Fake News Encoder Classifier (FNEC) for Online Published News Related to COVID-19 Vaccines

In the past few years,social media and online news platforms have played an essential role in distributing news content rapidly.Consequently.verification of the authenticity of news has become a major challenge.During the COVID-19 outbreak,misinformation and fake news were major sources of confusion and insecurity among the general public.In the first quarter of the year 2020,around 800 people died due to fake news relevant to COVID-19.The major goal of this research was to discover the best learning model for achieving high accuracy and performance.A novel case study of the Fake News Classification using ELECTRA model,which achieved 85.11%accuracy score,is thus reported in this manuscript.In addition to that,a new novel dataset called COVAX-Reality containing COVID-19 vaccine-related news has been contributed.Using the COVAX-Reality dataset,the performance of FNEC is compared to several traditional learning models i.e.,Support Vector Machine(SVM),Naive Bayes(NB),Passive Aggressive Classifier(PAC),Long Short-Term Memory(LSTM),Bi-directional LSTM(Bi-LSTM)and Bi-directional Encoder Representations from Transformers(BERT).For the evaluation of FNEC,standard metrics(Precision,Recall,Accuracy,and F1-Score)were utilized.

Implementation of Rapid Code Transformation Process Using Deep Learning Approaches

Our previous work has introduced the newly generated program using the code transformation model GPT-2,verifying the generated programming codes through simhash(SH)and longest common subsequence(LCS)algo-rithms.However,the entire code transformation process has encountered a time-consuming problem.Therefore,the objective of this study is to speed up the code transformation process signi􀀀cantly.This paper has proposed deep learning approaches for modifying SH using a variational simhash(VSH)algorithm and replacing LCS with a piecewise longest common subsequence(PLCS)algorithm to faster the veri􀀀cation process in the test phase.Besides the code transformation model GPT-2,this study has also introduced MicrosoMASS and Facebook BART for a comparative analysis of their performance.Meanwhile,the explainable AI technique using local interpretable model-agnostic explanations(LIME)can also interpret the decision-making ofAImodels.The experimental results show that VSH can reduce the number of quali􀀀ed programs by 22.11%,and PLCS can reduce the execution time of selected pocket programs by 32.39%.As a result,the proposed approaches can signi􀀀cantly speed up the entire code transformation process by 1.38 times on average compared with our previous work.

Generative pretrained transformer 4:an innovative approach to facilitate value-based healthcare

Objective Appropriate medical imaging is important for value-based care.We aim to evaluate the performance of generative pretrained transformer 4(GPT-4),an innovative natural language processing model,providing appropriate medical imaging automatically in different clinical scenarios.Methods Institutional Review Boards(IRB)approval was not required due to the use of nonidentifiable data.Instead,we used 112 questions from the American College of Radiology(ACR)Radiology-TEACHES Program as prompts,which is an open-sourced question and answer program to guide appropriate medical imaging.We included 69 free-text case vignettes and 43 simplified cases.For the performance evaluation of GPT-4 and GPT-3.5,we considered the recommendations of ACR guidelines as the gold standard,and then three radiologists analyzed the consistency of the responses from the GPT models with those of the ACR.We set a five-score criterion for the evaluation of the consistency.A paired t-test was applied to assess the statistical significance of the findings.Results For the performance of the GPT models in free-text case vignettes,the accuracy of GPT-4 was 92.9%,whereas the accuracy of GPT-3.5 was just 78.3%.GPT-4 can provide more appropriate suggestions to reduce the overutilization of medical imaging than GPT-3.5(t=3.429,P=0.001).For the performance of the GPT models in simplified scenarios,the accuracy of GPT-4 and GPT-3.5 was 66.5%and 60.0%,respectively.The differences were not statistically significant(t=1.858,P=0.070).GPT-4 was characterized by longer reaction times(27.1 s in average)and extensive responses(137.1 words on average)than GPT-3.5.Conclusion As an advanced tool for improving value-based healthcare in clinics,GPT-4 may guide appropriate medical imaging accurately and efficiently。

Integrated Modelling of Microstructure Evolution and Mechanical Properties Prediction for Q&P Hot Stamping Process of Ultra‑High Strength Steel

High strength steel products with good ductility can be produced via Q&P hot stamping process,while the phase transformation of the process is more complicated than common hot stamping since two-step quenching and one-step carbon partitioning processes are involved.In this study,an integrated model of microstructure evolution relating to Q&P hot stamping was presented with a persuasively predicted results of mechanical properties.The transformation of diffusional phase and non-diffusional phase,including original austenite grain size individually,were considered,as well as the carbon partitioning process which affects the secondary martensite transformation temperature and the subsequent phase transformations.Afterwards,the mechanical properties including hardness,strength,and elongation were calculated through a series of theoretical and empirical models in accordance with phase contents.Especially,a modified elongation prediction model was generated ultimately with higher accuracy than the existed Mileiko’s model.In the end,the unified model was applied to simulate the Q&P hot stamping process of a U-cup part based on the finite element software LS-DYNA,where the calculated outputs were coincident with the measured consequences.

Shear strength criteria for rock,rock joints,rockfill and rock masses:Problems and some solutions

Although many intact rock types can be very strong,a critical confining pressure can eventually be reached in triaxial testing,such that the Mohr shear strength envelope becomes horizontal.This critical state has recently been better defined,and correct curvature or correct deviation from linear Mohr-Coulomb（MC） has finally been found.Standard shear testing procedures for rock joints,using multiple testing of the same sample,in case of insufficient samples,can be shown to exaggerate apparent cohesion.Even rough joints do not have any cohesion,but instead have very high friction angles at low stress,due to strong dilation.Rock masses,implying problems of large-scale interaction with engineering structures,may have both cohesive and frictional strength components.However,it is not correct to add these,following linear M-C or nonlinear Hoek-Brown（H-B） standard routines.Cohesion is broken at small strain,while friction is mobilized at larger strain and remains to the end of the shear deformation.The criterion ＇c then σn tan φ＇ should replace ＇c plus σn tan φ＇ for improved fit to reality.Transformation of principal stresses to a shear plane seems to ignore mobilized dilation,and caused great experimental difficulties until understood.There seems to be plenty of room for continued research,so that errors of judgement of the last 50 years can be corrected.

Correlation of coordinate transformation parameters

Coordinate transformation parameters between two spatial Cartesian coordinate systems can be solved from the positions of non-colinear corresponding points. Based on the characteristics of translation, rotation and zoom components of the transformation, the complete solution is divided into three steps. Firstly, positional vectors are regulated with respect to the centroid of sets of points in order to separate the translation compo- nents. Secondly, the scale coefficient and rotation matrix are derived from the regulated positions independent- ly and correlations among transformation model parameters are analyzed. It is indicated that this method is applicable to other sets of non-position data to separate the respective attributions for transformation parameters.

A New Approach to Development of Composite Web Services

With the requirement for complex Web Services steadily increasing, there is a growing interest for reusing basic, existing Web Services to composite new ones. We present a composite Web Services development approach driven by model driven architecture （MDA） ,using UML（Unified Modeling Language） class diagram to model structure HM （Plateform Independent Platform） and UML activity diagram to model behavior PIM, then by model transformation converting the PIMs to specific Web Services specification platforms and execution platform to get the corresponding PSMs （platform specific models）. The main contributions of this paper are the all-around solution to Web Services composition development and the transformation rules for structure and behavior model of Web Services between PIMs and PSMs.

Formal Verification of TASM Models by Translating into UPPAAL

Timed abstract state machine(TASM) is a formal specification language used to specify and simulate the behavior of real-time systems. Formal verification of TASM model can be fulfilled through model checking activities by translating into UPPAAL. Firstly, the translational semantics from TASM to UPPAAL is presented through atlas transformation language(ATL). Secondly, the implementation of the proposed model transformation tool TASM2UPPAAL is provided. Finally, a case study is given to illustrate the automatic transformation from TASM model to UPPAAL model.

Research on transformation from UML statechart to interface automata

This paper studies the problem of deriving an interface automata model from UML statechart, in which, interface automata is a formaliged model for describing component behavior in an open system, but there is no universal criterion for deriving behavior from component to construct the model. UML is a widely used modeling standard, yet it is very difficult to apply it to system verification and testing directly for its imprecise semantics. After analyzing the expression ability of the two models, several transforma- tion rules are defined and each step of transformation is described in detail, after that, the approach is illustrated with an example. The paper provides a method for acquiring interface automata and lays the foundation for related research.

Theoretical investigation of interaction between a rectangular plate and fractional viscoelastic foundation

The interaction between plates and foundations is a typical problem encountered in geotechnical engineering. The long-term plate performance is highly dependent on the theological characteristics of ground soil. Compared with conventional linear theology, the fractional calculus-based theory is a more powerful mathematical tool that can address this issue. This paper proposes a fractional Merchant model （FMM） to investigate the time-dependent behavior of a simply supported rectangular plate on viscoelastic foundation. The correspondence principle involving Laplace transforms was employed to derive the closed-form solutions of plate response under uniformly distributed load. The plate deflection, bending moment, and foundation reaction calculated using the FMM were compared with the results obtained from the analogous elastic model （EM） and the standard Merchant model （SMM）. It is shown that the upper and lower bound solutions of the FMM can be determined using the EM. In addition, a parametric study was performed to examine the influences of the model parameters on the time- dependent behavior of the plate-foundation interaction problem. The results indicate that a small fractional differential order corresponds to a plate resting on a sandy soil foundation, while the fractional differential order value should be increased for a clayey soil foundation. The long-term performance of a foundation plate can be accurately simulated by varying the values of the fractional differential order and the viscosity coefficient. The observations from this study reveal that the proposed fractional model has the capability to capture the variation of plate deflection over many decades of time.