Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

Temporally Preserving Latent Variable Models:Offline and Online Training for Reconstruction and Interpretation of Fault Data for Gearbox Condition Monitoring

Latent variable models can effectively determine the condition of essential rotating machinery without needing labeled data.These models analyze vibration data via an unsupervised learning strategy.Temporal preservation is necessary to obtain an informative latent manifold for the fault diagnosis task.In a temporalpreserving context,two approaches exist to develop a condition-monitoring methodology:offline and online.For latent variable models,the available training modes are not different.While many traditional methods use offline training,online training can dynamically adjust the latent manifold,possibly leading to better fault signature extraction from the vibration data.This study explores online training using temporal-preserving latent variable models.Within online training,there are two main methods:one focuses on reconstructing data and the other on interpreting the data components.Both are considered to evaluate how they diagnose faults over time.Using two experimental datasets,the study confirms that models from both training modes can detect changes in machinery health and identify faults even under varying conditions.Importantly,the complementarity of offline and online models is emphasized,reassuring their versatility in fault diagnostics.Understanding the implications of the training approach and the available model formulations is crucial for further research in latent variable modelbased fault diagnostics.

A linear varying coefficient ARCH-M model with a latent variable

Motivated by the psychological factor of time-varying risk-return relationship, this paper studies a linear varying coefficient ARCH-M model with a latent variable. Due to the unobservable property of the latent variable, a corrected likelihood method is employed for parametric estimation. Estimators are proved to be consistent and asymptotically normal under certain regularity conditions. A simple test statistic is also proposed for testing latent variable effect. Simulation results confirm that the proposed estimators and test perform well.The model is further applied to examine whether the risk-return relationship depends on investor's sentiment in American Market and some explainable results are obtained.

Bayesian Empirical Likelihood Estimation of Quantile Structural Equation Models

Structural equation model(SEM) is a multivariate analysis tool that has been widely applied to many fields such as biomedical and social sciences. In the traditional SEM, it is often assumed that random errors and explanatory latent variables follow the normal distribution, and the effect of explanatory latent variables on outcomes can be formulated by a mean regression-type structural equation. But this SEM may be inappropriate in some cases where random errors or latent variables are highly nonnormal. The authors develop a new SEM, called as quantile SEM(QSEM), by allowing for a quantile regression-type structural equation and without distribution assumption of random errors and latent variables. A Bayesian empirical likelihood(BEL) method is developed to simultaneously estimate parameters and latent variables based on the estimating equation method. A hybrid algorithm combining the Gibbs sampler and Metropolis-Hastings algorithm is presented to sample observations required for statistical inference. Latent variables are imputed by the estimated density function and the linear interpolation method. A simulation study and an example are presented to investigate the performance of the proposed methodologies.

Dirichlet process and its developments: a survey

The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable prior therefore is especially critical in the nonparametric Bayesian fitting.As the distribution of distribution,Dirichlet process(DP)is the most appreciated nonparametric prior due to its nice theoretical proprieties,modeling flexibility and computational feasibility.In this paper,we review and summarize some developments of DP during the past decades.Our focus is mainly concentrated upon its theoretical properties,various extensions,statistical modeling and applications to the latent variable models.