DPAL-BERT:A Faster and Lighter Question Answering Model

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.

Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction

The remaining useful life(RUL)of a system is generally predicted by utilising the data collected from the sensors that continuously monitor different indicators.Recently,different deep learning(DL)techniques have been used for RUL prediction and achieved great success.Because the data is often time-sequential,recurrent neural network(RNN)has attracted significant interests due to its efficiency in dealing with such data.This paper systematically reviews RNN and its variants for RUL prediction,with a specific focus on understanding how different components(e.g.,types of optimisers and activation functions)or parameters(e.g.,sequence length,neuron quantities)affect their performance.After that,a case study using the well-studied NASA’s C-MAPSS dataset is presented to quantitatively evaluate the influence of various state-of-the-art RNN structures on the RUL prediction performance.The result suggests that the variant methods usually perform better than the original RNN,and among which,Bi-directional Long Short-Term Memory generally has the best performance in terms of stability,precision and accuracy.Certain model structures may fail to produce valid RUL prediction result due to the gradient vanishing or gradient exploring problem if the parameters are not chosen appropriately.It is concluded that parameter tuning is a crucial step to achieve optimal prediction performance.

PRI modulation recognition and sequence search under small sample prerequisite

Pulse repetition interval(PRI)modulation recognition and pulse sequence search are significant for effective electronic support measures.In modern electromagnetic environments,different types of inter-pulse slide radars are highly confusing.There are few available training samples in practical situations,which leads to a low recognition accuracy and poor search effect of the pulse sequence.In this paper,an approach based on bi-directional long short-term memory(BiLSTM)networks and the temporal correlation algorithm for PRI modulation recognition and sequence search under the small sample prerequisite is proposed.The simulation results demonstrate that the proposed algorithm can recognize unilinear,bilinear,sawtooth,and sinusoidal PRI modulation types with 91.43% accuracy and complete the pulse sequence search with 30% missing pulses and 50% spurious pulses under the small sample prerequisite.

Detection and Defense Method Against False Data Injection Attacks for Distributed Load Frequency Control System in Microgrid

In the realm of microgrid(MG),the distributed load frequency control(LFC)system has proven to be highly susceptible to the negative effects of false data injection attacks(FDIAs).Considering the significant responsibility of the distributed LFC system for maintaining frequency stability within the MG,this paper proposes a detection and defense method against unobservable FDIAs in the distributed LFC system.Firstly,the method integrates a bi-directional long short-term memory(Bi LSTM)neural network and an improved whale optimization algorithm(IWOA)into the LFC controller to detect and counteract FDIAs.Secondly,to enable the Bi LSTM neural network to proficiently detect multiple types of FDIAs with utmost precision,the model employs a historical MG dataset comprising the frequency and power variances.Finally,the IWOA is utilized to optimize the proportional-integral-derivative(PID)controller parameters to counteract the negative impacts of FDIAs.The proposed detection and defense method is validated by building the distributed LFC system in Simulink.