Progress in Neural NLP:Modeling,Learning,and Reasoning

Natural language processing(NLP)is a subfield of artificial intelligence that focuses on enabling computers to understand and process human languages.In the last five years,we have witnessed the rapid development of NLP in tasks such as machine translation,question-answering,and machine reading comprehension based on deep learning and an enormous volume of annotated and unannotated data.In this paper,we will review the latest progress in the neural network-based NLP framework(neural NLP)from three perspectives:modeling,learning,and reasoning.In the modeling section,we will describe several fundamental neural network-based modeling paradigms,such as word embedding,sentence embedding,and sequence-to-sequence modeling,which are widely used in modern NLP engines.In the learning section,we will introduce widely used learning methods for NLP models,including supervised,semi-supervised,and unsupervised learning;multitask learning;transfer learning;and active learning.We view reasoning as a new and exciting direction for neural NLP,but it has yet to be well addressed.In the reasoning section,we will review reasoning mechanisms,including the knowledge,existing non-neural inference methods,and new neural inference methods.We emphasize the importance of reasoning in this paper because it is important for building interpretable and knowledgedriven neural NLP models to handle complex tasks.At the end of this paper,we will briefly outline our thoughts on the future directions of neural NLP.

A NON-PARAMETER BAYESIAN CLASSIFIER FOR FACE RECOGNITION

A non-parameter Bayesian classifier based on Kernel Density Estimation (KDE)is presented for face recognition, which can be regarded as a weighted Nearest Neighbor (NN)classifier in formation. The class conditional density is estimated by KDE and the bandwidthof the kernel function is estimated by Expectation Maximum (EM) algorithm. Two subspaceanalysis methods-linear Principal Component Analysis (PCA) and Kernel-based PCA (KPCA)are respectively used to extract features, and the proposed method is compared with ProbabilisticReasoning Models (PRM), Nearest Center (NC) and NN classifiers which are widely used in facerecognition systems. The experiments are performed on two benchmarks and the experimentalresults show that the KDE outperforms PRM, NC and NN classifiers.

A Knowledge-reuse Based Intelligent Reasoning Model for Worsted Process Optimization

The textile process planning is a knowledge reuse process in nature, which depends on the expert’s knowledge and experience. It seems to be very difficult to build up an integral mathematical model to optimize hundreds of the processing parameters. In fact, the existing process cases which were recorded to ensure the ability to trace production steps can also be used to optimize the process itself. This paper presents a novel knowledge-reuse based hybrid intelligent reasoning model (HIRM) for worsted process optimization. The model architecture and reasoning mechanism are respectively described. An applied case with HIRM is given to demonstrate that the best process decision can be made, and important processing parameters such as for raw material optimized.

Developing a geographic Case-Based Reasoning approach

Case-Based Reasoning （CBR） is an AI approach and been applied to many areas. However, one area - geography - has not been investigated systematically and thus has been identified as the focus for this study. This paper intends to further extend current CBR to a geographic CBR （Geo-CBR）. First, the concept of Geo-CBR is proposed. Second, a representation model for geographic cases has been established based on the Tesseral model and on a further extension in spatio-temporal dimensions for geographic cases. Third, a reasoning model for Geo-CBR is developed by considering the spatio-temporat characteristics and the uncertain and limited information of geographic cases. Finally, the Geo-CBR model is applied to forecasting the production of ocean fisheries to demonstrate the applicability of the developed Geo-CBR in solving problems in the real world. According to the experimental results, Geo-CBR is an effective and easy-to-implement approach for predicting geographic cases quantitatively.

Parameterized continuous models of fuzzy reasoning

In this paper,the ideas of universal logic is introduced into fuzzy systems.After giving the definitions of the softened fuzzy reasoning models based on Schweizer-Sklar t-norms and Schweizer-Sklar implications,i.e.,α-models andβ-models,we give the sufficient and necessary conditions for these models to be continuous,and discuss the continuity of some commonly used models.We also prove that when anα-model or aβ-model is used as a fuzzy controller,it has universal property with respect to function approximation.The results we obtained show thatα-models andβ-models are more flexible than the existing models in applications.

A Common Reasoning Model and Its Application in Knowledge-Based System

To use reasoning knowledge accurately and efficiently,many reasoning methods have been proposed. However,the differences in form among the methods may obstruct the systematical analysis and harmonious integration of them.In this paper,a common reasoning model JUM(Judgement Model)is introduced. According to JUM,a common knowledge representation form is abstracted from different reasoning methods and its limitation is reduced.We also propose an algorithm for transforming one type of JUMs into another.In some cases,the algorithm can be used to resolve the key problem of integrating different types of JUM in one system.It is possible that a new architecture of knowledge-based system can be realized under JUM.