基于加权值函数分解的多智能体分层强化学习技能发现方法

针对目前大多数多智能体强化学习算法在智能体数量增多以及环境动态不稳定的情况下导致的维度爆炸和奖励稀疏的问题,提出了一种基于加权值函数分解的多智能体分层强化学习技能发现算法。首先,该算法将集中训练分散执行的架构与分层强化学习相结合,在上层采用加权值函数分解的方法解决智能体在训练过程中容易忽略最优策略而选择次优策略的问题;其次,在下层采用独立Q学习算法使其能够在多智能体环境中分散式地处理高维复杂的任务;最后,在底层独立Q学习的基础上引入技能发现策略,使智能体之间相互学习互补的技能。分别在简易团队运动和星际争霸Ⅱ两个仿真实验平台上对该算法与多智能体强化学习算法和分层强化学习算法进行对比,实验表明,该算法在奖励回报以及双方对抗胜率等性能指标上都有所提高,提升了整个多智能体系统的决策能力和收敛速度,验证了算法的可行性。

连续空间中的随机技能发现算法

针对大规模、连续空间随着状态维度指数级增加造成的"维数灾"问题,提出基于Option分层强化学习基础框架的改进的随机技能发现算法。通过定义随机Option生成一棵随机技能树,构造一个随机技能树集合。将任务目标分成子目标,通过学习低阶Option策略,减少因智能体增大而引起学习参数的指数增大。以二维有障碍栅格连续空间内两点间最短路径规划为任务,进行仿真实验和分析,实验结果表明:由于Option被随机定义,因此算法在初始性能上具有间歇的不稳定性,但是随着随机技能树集合的增加,能较快地收敛到近似最优解,能有效克服因为维数灾引起的难以求取最优策略或收敛速度过慢的问题。

ACP-based social computing and parallel intelligence： Societies 5.0 and beyond

Social computing, as the technical foundation of future computational smart societies, has the potential to improve the effectiveness of opensource big data usage, systematically integrate a variety of elements including time, human, resources, scenarios, and organizations in the current cyber-physical-social world, and establish a novel social structure with fair information, equal rights, and a flat configuration. Meanwhile, considering the big modeling gap between the model world and the physical world, the concept of parallel intelligence is introduced. With the help of software-defined everything, parallel intelligence bridges the big modeling gap by means of constructing artificial systems where computational experiments can be implemented to verify social policies, economic strategies, and even military operations. Artificial systems play the role of ＂social laboratories＂ in which decisions are computed before they are executed in our physical society. Afterwards, decisions with the expected outputs are executed in parallel in both the artificial and physical systems to interactively sense, compute, evaluate and adjust system behaviors in real-time, leading system behaviors in the physical system converging to those proven to be optimal in the artificial ones. Thus, the smart guidance and management for our society can be achieved.

A survey on rough set theory and its applications

After probability theory, fuzzy set theory and evidence theory, rough set theory is a new mathematical tool for dealing with vague, imprecise, inconsistent and uncertain knowledge. In recent years, the research and applications on rough set theory have attracted more and more researchers＇ attention. And it is one of the hot issues in the artificial intelligence field. In this paper, the basic concepts, operations and characteristics on the rough set theory are introduced firstly, and then the extensions of rough set model, the situation of their applications, some application software and the key problems in applied research for the rough set theory are presented.

Video-based construction vehicles detection and its application in intelligent monitoring system

While vehicle detection on highways has been reported before, to the best of our knowledge, intelligent monitoring system that aims at detecting hydraulic excavators and dump trucks on state-owned land has not been explored thoroughly yet. In this paper, we present an automatic, video-based algorithm for detecting hydraulic excavators and dump trucks. Derived from lessons learned from video processing, we proposed methods for foreground detection based on an improved frame difference algorithm, and then detected hydraulic excavators and dump trucks in the respective region of interest. From our analysis, we proposed methods based on inverse valley feature of mechanical arm and spatial-temporal reasoning for hydraulic excavator detection. In addition, we explored dump truck detection strategies that combine structured component projection with the spatial relationship. Experiments on real-monitoring sites demonstrated the promising performance of our system.

Exploiting structural similarity of log files in fault diagnosis for Web service composition

With increasing deployment of Web services, the research on the dependability and availability of Web service composition becomes more and more active. Since unexpected faults of Web service composition may occur in different levels at runtime, log analysis as a typical data- driven approach for fault diagnosis is more applicable and scalable in various architectures. Considering the trend that more and more service logs are represented using XML or JSON format which has good flexibility and interoperability, fault classification problem of semi-structured logs is considered as a challenging issue in this area. However, most existing approaches focus on the log content analysis but ignore the structural information and lead to poor performance. To improve the accuracy of fault classification, we exploit structural similarity of log files and propose a similarity based Bayesian learning approach for semi-structured logs in this paper. Our solution estimates degrees of similarity among structural elements from heterogeneous log data, constructs combined Bayesian network （CBN）, uses similarity based learning algorithm to compute probabilities in CBN, and classifies test log data into most probable fault categories based on the generated CBN. Experimental results show that our approach outperforms other learning approaches on structural log datasets.

Salient pairwise spatio-temporal interest points for real-time activity recognition

Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation efficiency, cluttered backgrounds and intro-variability among same type of actions. Spatio-temporal interest point （STIP） based methods have shown promising results to tackle human action classification in complex scenes efficiently. However, the state-of-the-art works typically utilize bag-of-visual words （BoVW） model which only focuses on the word distribution of STIPs and ignore the distinctive character of word structure. In this paper, the distribution of STIPs is organized into a salient directed graph, which reflects salient motions and can be divided into a time salient directed graph and a space salient directed graph, aiming at adding spatio-temporal discriminant to BoVW. Generally speaking, both salient directed graphs are constructed by labeled STIPs in pairs. In detail, the ＂directional co-occurrence＂ property of different labeled pairwise STIPs in same frame is utilized to represent the time saliency, and the space saliency is reflected by the ＂geometric relationships＂ between same labeled pairwise STIPs across different frames. Then, new statistical features namely the Time Salient Pairwise feature （TSP） and the Space Salient Pairwise feature （SSP） are designed to describe two salient directed graphs, respectively. Experiments are carried out with a homogeneous kernel SVM classifier, on four challenging datasets KTH, ADL and UT-Interaction. Final results confirm the complementary of TSP and SSP, and our multi-cue representation TSP ＋ SSP ＋ BoVW can properly describe human actions with large intro-variability in real-time.

Fourier Locally Linear Soft Constrained MACE for facial landmark localization

This paper proposes a novel nonlinear correlation filter for facial landmark localization. Firstly, we prove that SVM as a classifier can also be used for localization. Then, soft constrained Minimum Average Correlation Energy filter （soft constrained MACE） is proposed, which is more resistent to overfittings to training set than other variants of correlation filter. In order to improve the performance for the multi-mode of the targets, locally linear framework is introduced to our model, which results in Fourier Locally Linear Soft Constraint MACE （FL^2 SC-MACE）. Furthermore, we formulate the fast implementation and show that the time consumption in test process is independent of the number of training samples. The merits of our method include accurate localization performance, desiring generalization capability to the variance of objects, fast testing speed and insensitivity to parameter settings. We conduct the cross-set eye localization experiments on challenging FRGC, FERET and BioID datasets. Our method surpasses the state-of-arts especially in pixelwise accuracy.

Scene-adaptive hierarchical data association and depth-invariant part-based appearance model for indoor multiple objects tracking

Indoor multi-tracking is more challenging compared with outdoor tasks due to frequent occlusion, view-truncation, severe scale change and pose variation, which may bring considerable unreliability and ambiguity to target representation and data association. So discriminative and reliable target representation is vital for accurate data association in multi-tracking. Pervious works always combine bunch of features to increase the discriminative power, but this is prone to error accumulation and unnecessary computational cost, which may increase ambiguity on the contrary. Moreover, reliability of a same feature in different scenes may vary a lot, especially for currently widespread network cameras, which are settled in various and complex indoor scenes, previous fixed feature selection schemes cannot meet general requirements. To properly handle these problems, first, we propose a scene-adaptive hierarchical data association scheme, which adaptively selects features with higher reliability on target representation in the applied scene, and gradually combines features to the minimum requirement of discriminating ambiguous targets; second, a novel depth-invariant part-based appearance model using RGB-D data is proposed which makes the appearance model robust to scale change, partial occlusion and view-truncation. The introduce of RGB-D data increases the diversity of features, which provides more types of features for feature selection in data association and enhances the final multi-tracking performance. We validate our method from several aspects including scene-adaptive feature selection scheme, hierarchical data association scheme and RGB-D based appearance modeling scheme in various indoor scenes, which demonstrates its effectiveness and efficiency on improving multi-tracking performances in various indoor scenes.

Predicting Resting-state Functional Connectivity With Efficient Structural Connectivity

The complex relationship between structural connectivity(SC) and functional connectivity(FC) of human brain networks is still a critical problem in neuroscience. In order to investigate the role of SC in shaping resting-state FC, numerous models have been proposed. Here, we use a simple dynamic model based on the susceptible-infected-susceptible(SIS) model along the shortest paths to predict FC from SC. Unlike the previous dynamic model based on SIS theory, we focus on the shortest paths as the principal routes to transmit signals rather than the empirical structural brain network. We first simplify the structurally connected network into an efficient propagation network according to the shortest paths and then combine SIS infection theory with the efficient network to simulate the dynamic process of human brain activity. Finally, we perform an extensive comparison study between the dynamic models embedded in the efficient network, the dynamic model embedded in the structurally connected network and dynamic mean field(DMF) model predicting FC from SC. Extensive experiments on two different resolution datasets indicate that i) the dynamic model simulated on the shortest paths can predict FC among both structurally connected and unconnected node pairs; ii) though there are fewer links in the efficient propagation network, the predictive power of FC derived from the efficient propagation network is better than the dynamic model simulated on a structural brain network; iii) in comparison with the DMF model,the dynamic model embedded in the shortest paths is found to perform better to predict FC.

Sequential Bag-of-Words model for human action classification

Recently, approaches utilizing spatial-temporal features to form Bag-of-Words （BoWs） models have achieved great success due to their simplicity and effectiveness. But they still have difficulties when distinguishing between actions with high inter-ambiguity. The main reason is that they describe actions by orderless bag of features, and ignore the spatial and temporal structure information of visual words. In order to improve classification performance, we present a novel approach called sequential Bag-of-Words. It captures temporal sequential structure by segmenting the entire action into sub-actions. Meanwhile, we pay more attention to the distinguishing parts of an action by classifying sub- actions separately, which is then employed to vote for the final result. Extensive experiments are conducted on challenging datasets and real scenes to evaluate our method. Concretely, we compare our results to some state-of-the-art classification approaches and confirm the advantages of our approach to distinguish similar actions. Results show that our approach is robust and outperforms most existing BoWs based classification approaches, especially on complex datasets with interactive activities, cluttered backgrounds and inter-class action ambiguities.

Density peaks clustering based integrate framework for multi-document summarization

We present a novel unsupervised integrated score framework to generate generic extractive multi- document summaries by ranking sentences based on dynamic programming （DP） strategy. Considering that cluster-based methods proposed by other researchers tend to ignore informativeness of words when they generate summaries, our proposed framework takes relevance, diversity, informativeness and length constraint of sentences into consideration comprehensively. We apply Density Peaks Clustering （DPC） to get relevance scores and diversity scores of sentences simultaneously. Our framework produces the best performance on DUC2004, 0.396 of ROUGE-1 score, 0.094 of ROUGE-2 score and 0.143 of ROUGE-SU4 which outperforms a series of popular baselines, such as DUC Best, FGB [7], and BSTM [10].

Investigation of Eddy Current Nondestructive Testing for Carbon Fiber-Reinforced Plastic （CFRP） Based on Electromagnetic Field Analysis

CFRP （carbon fiber reinforced plastic） is used extensively in aircraft and spacecraft structures, because of its excellent mechanical properties. Ultrasonic testing, which is used as a non-destructive testing technique for CFRP, requires a contact medium. In contrast, eddy current testing does not require a contact medium, and when used for CFRP testing it has advantages not available with other techniques. CFRP is a laminate, with each layer being anisotropically conductive, and the distribution of the induced eddy current is yet to be determined. Here, to determine the eddy current distribution in the detection of flaws in cross-ply CFRP （0°/90°） by using a cross-point probe, we performed an FEM （finite element method） analysis of electromagnetic fields. We investigated the nature of the flaw signals and the differences in eddy current distributions between materials with and without flaws.

Forecasting tourism demand by extracting fuzzy Takagi-Sugeno rules from trained SVMs

Tourism demand forecasting has attracted substantial interest because of the significant economic contributions of the fast-growing tourism industry. Although various quantitative forecasting techniques have been widely studied, highly accurate and understandable forecasting models have not been developed. The present paper proposes a novel tourism demand forecasting method that extracts fuzzy Takagi-Sugeno （T-S） rules from trained SVMs. Unlike previous approaches, this study uses fuzzy T-S models extracted from the outputs of trained SVMs on tourism data. Owing to the symbolic fuzzy rules and the generalization ability of SVMs, the extracted fuzzy T-S rules exhibit high forecasting accuracy and include understandable pre-condition parts for practitioners. Based on the tourism demand forecasting problem in Hong Kong SAR, China as a case study, empirical findings on tourist arrivals from nine overseas origins reveal that the proposed approach performs comparably with SVMs and can achieve better prediction accuracy than other forecasting techniques for most origins. The findings demonstrated that decision makers can easily interpret fuzzy T-S rules extracted from SVMs. Thus, the approach is highly beneficial to tourism market management. This finding demonstrates the excellent scientific and practical values of the proposed approach in tourism demand forecasting.

Tenant-centric Sub-Tenancy Architecture in Software-as-a-Service

Multi-tenancy architecture （MTA） is often used in Software-as-a-Service （SaaS） and the central idea is that multiple tenant applications can be developed using components stored in the SaaS infrastructure. Recently, MTA has been extended to allow a tenant application to have its own sub-tenants, where the tenant application acts like a SaaS infrastructure. In other words, MTA is extended to STA （Sub-Tenancy Architecture）. In STA, each tenant application needs not only to develop its own functionalities, but also to prepare an infrastructure to allow its sub-tenants to develop customized applications. This paper applies Crowdsourcing as the core to STA component in the development life cycle. In addition, to discovering adequate fit tenant developers or components to help build and compose new components, dynamic and static ranking models are proposed. Furthermore, rank computation architecture is presented to deal with the case when the number of tenants and components becomes huge. Finally, experiments are performed to demonstrate that the ranking models and the rank computation architecture work as design.

Research progress of artificial psychology and artificial emotion in China

Since the concept of artificial psychology and artificial emotion was first presented, it has become a topic of interest in academic circles and enterprises. In this article, we first briefly introduce the basic concepts and principles of artificial psychology and artificial emotion, analyzing the unified macro-model of the cross-disciplinary system architecture against the need-motivation-behavior framework. Second, we discuss the origin of artificial psychology and artificial emotion, its course of development, and its present situation in China. We also present a review of the published papers and research endeavors of Chinese universities and research institutions and the technical engineering applications of artificial psychology and artificial emotion. Finally, we summarize the challenges to the further development of artificial psychology and artificial emotion and our recommendations for improving the cognitive computing model of psychological states and developing reliable and accurate humanoid interaction and cooperation technology, robot platforms with emotions and humanoid interaction and cooperation capabilities, and humanoid robots for the elderly and the disabled in smart homes. We believe that, with intensive research, artificial psychology and artificial emotion may be developed further and may eventually reach maturity.

A multi-objective optimization framework for ill-posed inverse problems

Many image inverse problems are ill-posed for no unique solutions. Most of them have incommensurable or mixed-type objectives. In this study, a multi-objective optimization framework is introduced to model such ill-posed inverse problems. The conflicting objectives are designed according to the properties of ill-posedness and certain techniques. Multi-objective evolutionary algorithms have capability to optimize multiple objectives simultaneously and obtain a set of trade-off solutions. For that reason, we use multi-objective evolutionary algorithms to keep the trade-off between these objectives for image ill-posed problems. Two case studies of sparse reconstruction and change detection are imple- mented. In the case study of sparse reconstruction, the measurement error term and the sparsity term are optimized by multi-objective evolutionary algorithms, which aims at balancing the trade-off between enforcing sparsity and reducing measurement error. In the case study of image change detection, two conflicting objectives are constructed to keep the trade-off between robustness to noise and preserving the image details. Experimental results of the two case studies confirm the multi-objective optimization framework for ill-posed inverse problems in image processing is effective.

Frame interpolation with pixel-level motion vector field and mesh based hole filling

Most of the traditional methods are based on block motion compensation tending to involve heavy blocking artifacts in the interpolated frames. In this paper, a new frame interpolation method with pixel-level motion vector field （MVF） is proposed. Our method consists of the following four steps： （i） applying the pixel-level motion vectors （MVs） estimated by optical flow algorithm to eliminate blocking artifacts （ii） motion post-processing and super-sampling anti-aliasing to solve the problems caused by pixel-level MVs （iii） robust warping method to address collisions and holes caused by occlusions （iv） a new holes filling method using triangular mesh （HFTM） to reduce the artifacts caused by holes. Experimental results show that the proposed method can effectively alleviate the holes and blocking artifacts in interpolated frames, and outperforms existing methods both in terms of objective and subjective performances, especially for sequences with complex motions.

Acoustic Valve Integration in Exhaust Line Systems

In recent years the demand for the acoustic performance of exhaust systems has increased and will further increase in the future. The main drivers are new pass-by-noise regulation and new powertrain technologies paired with exhaust muffler volume, weight and costs constraints. In the following paper several application examples for Adaptive ValveTM （self-actuated in-pipe valve）, in-muffler valve and electric valve are shown and the related benefits on the system performance are assessed. It is shown that implementing a valve into an exhaust system has a significant influence on the NVH performance. The resulting backpressure penalties can be minimized using the right implementation strategy of the valves in the exhaust system. Hence the exhaust system has to be specifically designed for the integration of a valve. All three valve types have additional benefits to their standard application for overall noise reduction and muffler volume reduction, which are analyzed. The Adaptive ValveTM, for example, is often used on cars with long pipe routing and has the additional benefit of reducing pipe resonance in the system. Another example, the electric valve, can be coupled with vehicle communication networks and hence the flexibility in application is significantly increased.

A syncretic representation for image classification and face recognition

For representation based image classification methods, it is very important to well represent the target image. As pixels at same positions of training samples and test samples of an object usually have different intensities, it brings difficulty in correctly classifying the object. In this paper, we proposed a novel method to reduce the effects of this issue for image classification. Our method first produces a new representation （i.e. virtual image） of original image, which can enhance the importance of moderate pixel intensities and reduce the effects of larger or smaller pixel intensities. Then virtual images and corresponding original images are respectively used to represent a test sample and obtain two rep- resentation results. Finally, this method fuses these two results to classify the test sample. The integration of original image and its virtual image is able to improve the accuracy of image classification. The experiments of image classification show that the proposed method can obtain a higher accuracy than the conventional classification methods.