Convergence of Markov decision processes with constraints and state-action dependent discount factors

This paper is concerned with the convergence of a sequence of discrete-time Markov decision processes(DTMDPs)with constraints,state-action dependent discount factors,and possibly unbounded costs.Using the convex analytic approach under mild conditions,we prove that the optimal values and optimal policies of the original DTMDPs converge to those of the"limit"one.Furthermore,we show that any countablestate DTMDP can be approximated by a sequence of finite-state DTMDPs,which are constructed using the truncation technique.Finally,we illustrate the approximation by solving a controlled queueing system numerically,and give the corresponding error bound of the approximation.

First passage Markov decision processes with constraints and varying discount factors

This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.

Fault Detection Based on FIR Filters with Variable Discount Factors for Industrial Robot Drive Systems

To enhance dynamic tracking performance and anti-disturbance capacity of finite impulse response(FIR) filters, variable discount factors are introduced to the recursive least squares(RLS) algorithm. By employing improved FIR filters to conduct modelling of industrial robot drive systems, dynamic characteristics of the target systems are identified. Then the fault detection for a target system can be utilized by analyzing the coefficients of the FIR filter. Finally, an application of the fault detection scheme to a kind of brushless DC motor drive system is described. Compared with reference methods, the proposed scheme achieves effective fault detection and performs better in dynamic tracking and robustness according to the final simulation results.

A closed-form pricing formula for European options in an illiquid asset market

This article addresses the problem of pricing European options when the underlying asset is not perfectly liquid.A liquidity discounting factor as a function of market-wide liquidity governed by a mean-reverting stochastic process and the sensitivity of the underlying price to market-wide liquidity is firstly introduced,so that the impact of liquidity on the underlying asset can be captured by the option pricing model.The characteristic function is analytically worked out using the Feynman–Kac theorem and a closed-form pricing formula for European options is successfully derived thereafter.Through numerical experiments,the accuracy of the newly derived formula is verified,and the significance of incorporating liquidity risk into option pricing is demonstrated.

Quasi-variable Discount Rates and Market Volatility: An Empirical Study

This paper proposes an assumption of quasi-variable discount rates to explain the excess volatility puzzle of stock market. Under the assumption, the ARMAX model is derived based on the CCAPM model and CRRA utility function to describe the linear relationship between the discount rate and the consumption growth rate. We conducted empirical research on this model using historical data of the US stock market. The results confirm a significantly negative relationship between consumption growth rate and discount rate. Subsequently, the results of Monte Carlo simulation show that given the risk preference coefficient and dividend sequence, the rational expectations price fluctuation obtained under the assumption of quasivariable discount rate is the largest.

A Closed Form Characterization of the Stationary Outcome in Multilateral Bargaining

In this paper we consider infinite horizon multilateral bargaining with al- ternate offers. We prove that there exists only one stationary subgame perfect equilib- rium outcome and it corresponds to the unique invariant measure of a column stochas- tic matrix. We characterize this stationary subgame perfect equilibrium outcome in a closed form, and also extend the approach to the multilateral bargaining with random moves.

An average-value-at-risk criterion for Markov decision processes with unbounded costs

We study the Markov decision processes under the average-value-at-risk criterion.The state space and the action space are Borel spaces,the costs are admitted to be unbounded from above,and the discount factors are state-action dependent.Under suitable conditions,we establish the existence of optimal deterministic stationary policies.Furthermore,we apply our main results to a cash-balance model.