A deep learning method based on prior knowledge with dual training for solving FPK equation

The evolution of the probability density function of a stochastic dynamical system over time can be described by a Fokker–Planck–Kolmogorov(FPK) equation, the solution of which determines the distribution of macroscopic variables in the stochastic dynamic system. Traditional methods for solving these equations often struggle with computational efficiency and scalability, particularly in high-dimensional contexts. To address these challenges, this paper proposes a novel deep learning method based on prior knowledge with dual training to solve the stationary FPK equations. Initially, the neural network is pre-trained through the prior knowledge obtained by Monte Carlo simulation(MCS). Subsequently, the second training phase incorporates the FPK differential operator into the loss function, while a supervisory term consisting of local maximum points is specifically included to mitigate the generation of zero solutions. This dual-training strategy not only expedites convergence but also enhances computational efficiency, making the method well-suited for high-dimensional systems. Numerical examples, including two different two-dimensional(2D), six-dimensional(6D), and eight-dimensional(8D) systems, are conducted to assess the efficacy of the proposed method. The results demonstrate robust performance in terms of both computational speed and accuracy for solving FPK equations in the first three systems. While the method is also applicable to high-dimensional systems, such as 8D, it should be noted that computational efficiency may be marginally compromised due to data volume constraints.

Cascade Human Activity Recognition Based on Simple Computations Incorporating Appropriate Prior Knowledge

The purpose of Human Activities Recognition(HAR)is to recognize human activities with sensors like accelerometers and gyroscopes.The normal research strategy is to obtain better HAR results by finding more efficient eigenvalues and classification algorithms.In this paper,we experimentally validate the HAR process and its various algorithms independently.On the base of which,it is further proposed that,in addition to the necessary eigenvalues and intelligent algorithms,correct prior knowledge is even more critical.The prior knowledge mentioned here mainly refers to the physical understanding of the analyzed object,the sampling process,the sampling data,the HAR algorithm,etc.Thus,a solution is presented under the guidance of right prior knowledge,using Back-Propagation neural networks(BP networks)and simple Convolutional Neural Networks(CNN).The results show that HAR can be achieved with 90%–100%accuracy.Further analysis shows that intelligent algorithms for pattern recognition and classification problems,typically represented by HAR,require correct prior knowledge to work effectively.

Dynamic Spectrum Access Based on Prior Knowledge Enabled Reinforcement Learning with Double Actions in Complex Electromagnetic Environment

The spectrum access problem of cognitive users in the fast-changing dynamic interference spectrum environment is addressed in this paper.The prior knowledge for the dynamic spectrum access is modeled and a reliability quantification scheme is presented to guide the use of the prior knowledge in the learning process.Furthermore,a spectrum access scheme based on the prior knowledge enabled RL(PKRL)is designed,which effectively improved the learning efficiency and provided a solution for users to better adapt to the fast-changing and high-density electromagnetic environment.Compared with the existing methods,the proposed algorithm can adjust the access channel online according to historical information and improve the efficiency of the algorithm to obtain the optimal access policy.Simulation results show that,the convergence speed of the learning is improved by about 66%with the invariant average throughput.

Reasoning Disaster Chains with Bayesian Network Estimated Under Expert Prior Knowledge

With the acceleration of global climate change and urbanization,disaster chains are always connected to artificial systems like critical infrastructure.The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers.The Bayesian network(BN)was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies.To capture the interacting relationships among different factors,a scenario representation model of disaster chains was developed,followed by the determination of the BN structure.In deriving the conditional probability tables of the BN model,we found that,due to the lack of data and the significant uncertainty of disaster chains,parameter learning methodologies based on data or expert knowledge alone are insufficient.By integrating both sample data and expert knowledge with the maximum entropy principle,we proposed a parameter estimation algorithm under expert prior knowledge(PEUK).Taking the rainstorm disaster chain as an example,we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior(MAP)algorithm and the direct expert opinion elicitation method.The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions.

A Structure Learning Algorithm for Bayesian Network Using Prior Knowledge

Learning structure from data is one of the most important fundamental tasks of Bayesian network research. Particularly, learning optional structure of Bayesian network is a non-deterministic polynomial-time （NP） hard problem. To solve this problem, many heuristic algorithms have been proposed, and some of them learn Bayesian network structure with the help of different types of prior knowledge. However, the existing algorithms have some restrictions on the prior knowledge, such as quality restriction and use restriction. This makes it di？cult to use the prior knowledge well in these algorithms. In this paper, we introduce the prior knowledge into the Markov chain Monte Carlo （MCMC） algorithm and propose an algorithm called Constrained MCMC （C-MCMC） algorithm to learn the structure of the Bayesian network. Three types of prior knowledge are defined： existence of parent node, absence of parent node, and distribution knowledge including the conditional probability distribution （CPD） of edges and the probability distribution （PD） of nodes. All of these types of prior knowledge are easily used in this algorithm. We conduct extensive experiments to demonstrate the feasibility and effectiveness of the proposed method C-MCMC.

Can prior knowledge help graph-based methods for keyword extraction？

Graph-based methods are one of the widely used unsupervised approaches for keyword extraction. In this approach, words are linked according to their co- occurrences within the document. Afterwards, graph-based ranking algorithms are used to rank words and those with the highest scores are selected as keywords. Although graph-based methods are effective for keyword extraction, they rank words merely based on word graph topology. In fact, we have various prior knowledge to identify how likely the words are keywords. The knowledge of words may be frequency-based, position-based, or semantic- based. In this paper, we propose to incorporate prior knowledge with graph-based methods for keyword extraction and investigate the contributions of the prior knowledge. Experiments reveal that prior knowledge can significantly improve the performance of graph-based keyword extraction. Moreover, by combining prior knowl- edge with neighborhood knowledge, in experiments we achieve the best results compared to previous graph-based methods.

Knowledge-based adaptive polarimetric detection in heterogeneous clutter

The detection performance and the constant false alarm rate behavior of the conventional adaptive detectors are severely degraded in heterogeneous clutter. This paper designs and analyses a knowledge-based （KB） adaptive polarimetric detector in het-erogeneous clutter. The proposed detection scheme is composed of a data selector using polarization knowledge and an adaptive polarization detector using training data. A polarization data selector based on the maximum likelihood estimation is proposed to remove outliers from the heterogeneous training data. This selector can remove outliers effectively, thus the training data is purified for estimating the clutter covariance matrix. Consequently, the performance of the adaptive detector is improved. We assess the performance of the KB adaptive polarimetric detector and the adaptive polarimetric detector without a data selector using simulated data and IPIX radar data. The results show that the KB adaptive polarization detector outperforms its non-KB counterparts.

Applications and potentials of machine learning in optoelectronic materials research:An overview and perspectives

Optoelectronic materials are essential for today's scientific and technological development,and machine learning provides new ideas and tools for their research.In this paper,we first summarize the development history of optoelectronic materials and how materials informatics drives the innovation and progress of optoelectronic materials and devices.Then,we introduce the development of machine learning and its general process in optoelectronic materials and describe the specific implementation methods.We focus on the cases of machine learning in several application scenarios of optoelectronic materials and devices,including the methods related to crystal structure,properties(defects,electronic structure)research,materials and devices optimization,material characterization,and process optimization.In summarizing the algorithms and feature representations used in different studies,it is noted that prior knowledge can improve optoelectronic materials design,research,and decision-making processes.Finally,the prospect of machine learning applications in optoelectronic materials is discussed,along with current challenges and future directions.This paper comprehensively describes the application value of machine learning in optoelectronic materials research and aims to provide reference and guidance for the continuous development of this field.

A Method of Eliminating Information Disclosure in View Publishing

Although it is convenient to exchange data by publishing view, but it may disclose sensitive information. The problem of how to eliminate information disclosure becomes a core problem in the view publishing process. In order to eliminate information disclosure, deciding view security algorithm and eliminating information disclosure algorithm are proposed, and the validity of the algorithms are proved by experiment. The experimental results showing, deciding view security algorithm can decide the safety of a set of views under prior knowledge, and eliminating information disclosure algorithm can eliminate disclosure efficiently.

Patch-based vehicle logo detection with patch intensity and weight matrix

A patch-based method for detecting vehicle logos using prior knowledge is proposed.By representing the coarse region of the logo with the weight matrix of patch intensity and position,the proposed method is robust to bad and complex environmental conditions.The bounding-box of the logo is extracted by a thershloding approach.Experimental results show that 93.58% location accuracy is achieved with 1100 images under various environmental conditions,indicating that the proposed method is effective and suitable for the location of vehicle logo in practical applications.

Bayesian Inference of Empirical Coefficient for Foundation Settlement

A new approach based on Bayesian theory is proposed to determine the empirical coefficient in soil settlement calculation. Prior distribution is assumed to he uniform in [ 0.2,1.4 ]. Posterior density function is developed in the condition of prior distribution combined with the information of observed samples at four locations on a passenger dedicated fine. The results show that the posterior distribution of the empirical coefficient obeys Gaussian distribution. The mean value of the empirical coefficient decreases gradually with the increasing of the load on ground, and variance variation shows no regularity.

Autonomous mobile robot global path planning: a prior information-based particle swarm optimization approach

The path planning of autonomous mobile robots(PPoAMR)is a very complex multi-constraint problem.The main goal is to find the shortest collision-free path from the starting point to the target point.By the fact that the PPoAMR problem has the prior knowledge that the straight path between the starting point and the target point is the optimum solution when obstacles are not considered.This paper proposes a new path planning algorithm based on the prior knowledge of PPoAMR,which includes the fitness value calculation method and the prior knowledge particle swarm optimization(PKPSO)algorithm.The new fitness calculation method can preserve the information carried by each individual as much as possible by adding an adaptive coefficient.The PKPSO algorithm modifies the particle velocity update method by adding a prior particle calculated from the prior knowledge of PPoAMR and also implemented an elite retention strategy,which improves the local optima evasion capability.In addition,the quintic polynomial trajectory optimization approach is devised to generate a smooth path.Finally,some experimental comparisons with those state-of-the-arts are carried out to demonstrate the effectiveness of the proposed path planning algorithm.

Adaptive and augmented active anomaly detection on dynamic network traffic streams

Active anomaly detection queries labels of sampled instances and uses them to incrementally update the detection model,and has been widely adopted in detecting network attacks.However,existing methods cannot achieve desirable performance on dynamic network traffic streams because(1)their query strategies cannot sample informative instances to make the detection model adapt to the evolving stream and(2)their model updating relies on limited query instances only and fails to leverage the enormous unlabeled instances on streams.To address these issues,we propose an active tree based model,adaptive and augmented active prior-knowledge forest(A3PF),for anomaly detection on network trafic streams.A prior-knowledge forest is constructed using prior knowledge of network attacks to find feature subspaces that better distinguish network anomalies from normal traffic.On one hand,to make the model adapt to the evolving stream,a novel adaptive query strategy is designed to sample informative instances from two aspects:the changes in dynamic data distribution and the uncertainty of anomalies.On the other hand,based on the similarity of instances in the neighborhood,we devise an augmented update method to generate pseudo labels for the unlabeled neighbors of query instances,which enables usage of the enormous unlabeled instances during model updating.Extensive experiments on two benchmarks,CIC-IDS2017 and UNSW-NB15,demonstrate that A3PF achieves significant improvements over previous active methods in terms of the area under the receiver operating characteristic curve(AUC-ROC)(20.9%and 21.5%)and the area under the precision-recall curve(AUC-PR)(44.6%and 64.1%).

Using knowledge inference to suppress the lamp disturbance for fire detection

Fire detection in buildings is crucial for people’s lives and property.Conventional temperature and smoke sen-sors have many disadvantages:the limited cover range;detection delays;the difficulty in distinguishing smoke and fire.Recently,research on convolutional neural networks(CNN)for fire image detection has become a hot topic.However,existing fire classification and object detection methods are often interfered with by flash-lights,red objects and the high-brightness background,resulting in a high false alarm rate.Besides,light and lamps often exist in buildings.To address this issue,this paper focuses on introducing scene prior knowledge and causal inference mechanisms to suppress the lamp disturbance.Firstly,we train the YoloV3 network to detect and recognize lamps.Secondly,to reduce the dataset bias,we mask the lamp regions with the pro-posed Local Grabcut segmentation method.Last,compared with direct fire classification methods,our proposed methods reduce about 34.6%false alarm rate based on InceptionV4 networks.The experimental results verify the effectiveness among different CNN architectures(Resnet101,Firenet,Densenet121).The code is online at https://github.com/kailaisun/fire-detection-without-lamp.

Entity and relation extraction with rule-guided dictionary as domain knowledge

Entity and relation extraction is an indispensable part of domain knowledge graph construction,which can serve relevant knowledge needs in a specific domain,such as providing support for product research,sales,risk control,and domain hotspot analysis.The existing entity and relation extraction methods that depend on pretrained models have shown promising performance on open datasets.However,the performance of these methods degrades when they face domain-specific datasets.Entity extraction models treat characters as basic semantic units while ignoring known character dependency in specific domains.Relation extraction is based on the hypothesis that the relations hidden in sentences are unified,thereby neglecting that relations may be diverse in different entity tuples.To address the problems above,this paper first introduced prior knowledge composed of domain dictionaries to enhance characters’dependence.Second,domain rules were built to eliminate noise in entity relations and promote potential entity relation extraction.Finally,experiments were designed to verify the effectiveness of our proposed methods.Experimental results on two domains,including laser industry and unmanned ship,showed the superiority of our methods.The F1 value on laser industry entity,unmanned ship entity,laser industry relation,and unmanned ship relation datasets is improved by+1%,+6%,+2%,and+1%,respectively.In addition,the extraction accuracy of entity relation triplet reaches 83%and 76%on laser industry entity pair and unmanned ship entity pair datasets,respectively.

Multitask Learning with Multiscale Residual Attention for Brain Tumor Segmentation and Classification

Automatic segmentation and classification of brain tumors are of great importance to clinical treatment.However,they are challenging due to the varied and small morphology of the tumors.In this paper,we propose a multitask multiscale residual attention network(MMRAN)to simultaneously solve the problem of accurately segmenting and classifying brain tumors.The proposed MMRAN is based on U-Net,and a parallel branch is added at the end of the encoder as the classification network.First,we propose a novel multiscale residual attention module(MRAM)that can aggregate contextual features and combine channel attention and spatial attention better and add it to the shared parameter layer of MMRAN.Second,we propose a method of dynamic weight training that can improve model performance while minimizing the need for multiple experiments to determine the optimal weights for each task.Finally,prior knowledge of brain tumors is added to the postprocessing of segmented images to further improve the segmentation accuracy.We evaluated MMRAN on a brain tumor data set containing meningioma,glioma,and pituitary tumors.In terms of segmentation performance,our method achieves Dice,Hausdorff distance(HD),mean intersection over union(MIoU),and mean pixel accuracy(MPA)values of 80.03%,6.649 mm,84.38%,and 89.41%,respectively.In terms of classification performance,our method achieves accuracy,recall,precision,and F1-score of 89.87%,90.44%,88.56%,and 89.49%,respectively.Compared with other networks,MMRAN performs better in segmentation and classification,which significantly aids medical professionals in brain tumor management.The code and data set are available at https://github.com/linkenfaqiu/MMRAN.

Leveraging Large Data with Weak Supervision for Joint Feature and Opinion Word Extraction

Product feature and opinion word extraction is very important for fine granular sentiment analysis. In this paper, we leverage large-scale unlabeled data for joint extraction of feature and opinion words under a knowledge poor setting, in which only a few feature-opinion pairs are utilized as weak supervision. Our major contributions are two- fold： first, we propose a data-driven approach to represent product features and opinion words as a list of corpus-level syntactic relations, which captures rich language structures; second, we build a simple yet robust unsupervised model with prior knowledge incorporated to extract new feature and opinion words, which obtains high performance robustly. The extraction process is based upon a bootstrapping framework which, to some extent, reduces error propagation under large data. Experimental results under various settings compared with state-of-the-art baselines demonstrate that our method is effective and promising.

Effects of In-video Questions and Feedback on Learning Performance

The eye-tracking technology was used in this study to investigate the effects of embedded questions and feedback in instructional videos on learning performance and attention allocation and whether an expertise reversal effect existed.The experiment involved 49 learners with high-level prior knowledge and 45 ones with low-level prior knowledge from a university.Meanwhile,they learned instructional videos with no embedded feedback,embedded questions without feedback and embedded questions with feedback.Findings from the experiment showed that the instructional videos with embedded questions but without feedback not only improved the participants’attention but also enhanced their learning performance.Furthermore,there was an expertise reversal effect on the learning performance whereby instructional videos with embedded questions but without feedback improved the learning performance of learners with low-level prior knowledge,but not those with high-level prior knowledge.

Chinese EFL Learners’Phonetics Learning Guided by Visuospatial Cues through the Medium of Mobile Phones

The relevant studies using a cross sectional view of speech organs supplemented with visuospatial cues and verbal text to explore EFL learners’learning effectiveness and behavior through mobile devices when learning English phonetics are scarce.This study was attempted to investigate whether the presence of visuospatial cues can benefit EFL learners with different levels of prior knowledge in learning English phonetics through mobile devices.The present study investigated the interaction between the experimental condition and the learners’prior knowledge on their task performances and cognitive load ratings.Fifty-six English as a foreign language(EFL)learners recruited from two sections of a linguistics course participated in the experiment.First,their background knowledge concerning English phonetics was evaluated to determine their prior knowledge level.Then,they were randomly assigned into two experimental conditions-picture-plus-text and picture-plus-text-plus-cueing.After the experimental treatment,the participants were administered retention and transfer tests as well as cognitive load measurement.Experimental treatment and prior knowledge were the independent variables,while retention test,transfer test,study time,and number of clicks were the dependent variables.The results of the present study emphasized the importance of visuospatial cues on inducing deep cognitive processing as indicated by the learners’test performance and study patterns.