A minimization problem of the risk probability in first passage semi-Markov decision processes with loss rates

This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total loss incurred during a first passage time to some target set exceeds a loss level. First, we establish the optimality equation via a successive approximation technique, and show that the value function is the unique solution to the optimality equation. Second, we give suitable conditions, under which we prove the existence of optimal policies and develop an algorithm for computing ?-optimal policies. Finally, we apply our main results to a business system.

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.

Rationale for Decision-Making Processes in Enhancement of Community Participation for Sustainable Mangrove Management in Lamu, Kenya

Decision-making is the process of deciding between two or more options in order to take the most appropriate and successful course of action in order to achieve sustainable mangrove management. However, the distinctiveness of mangrove as an ecosystem, and thus the attendant socio-economic and governance ramifications, causes the idea of decision making to become relatively distinct from other decision making process As a result, the purpose of this research was to evaluate the impact that community engagement plays in the decision-making process as it relates to the establishment of governance norms for sustainable mangrove management in Lamu County. In this study, a correlational research design was applied, and the researchers employed a mixed techniques approach. The target population was 296 respondents. The research used questionnaires and interviews to collect data. A descriptive statistical technique was utilized to perform an inspection and analysis on the data that was gathered. The findings indicated that having awareness about governance standards is beneficial during the process of making decisions. In addition, the findings demonstrated that respondents had the impression that the decision-making process was not done properly. On the other hand, the participants pointed out the positive aspects of the decision-making process and agreed that the participation of both gender was essential for the sustainable management of mangroves. Based on these data, it appeared that full community engagement in decision-making is necessary for sustainable management of mangrove forests.

A Comparative Analysis of Visualization Methods in Architecture:Employing Virtual Reality to Support the Decision-Making Process in the Architecture,Engineering,and Construction Industry

The design process of the built environment relies on the collaborative effort of all parties involved in the project.During the design phase,owners,end users,and their representatives are expected to make the most critical design and budgetary decisions-shaping the essential traits of the project,hence emerge the need and necessity to create and integrate mechanisms to support the decision-making process.Design decisions should not be based on assumptions,past experiences,or imagination.An example of the numerous problems that are a result of uninformed design decisions is“change orders”,known as the deviation from the original scope of work,which leads to an increase of the overall cost,and changes to the construction schedule of the project.The long-term aim of this inquiry is to understand the user’s behavior,and establish evidence-based control measures,which are actions and processes that can be implemented in practice to decrease the volume and frequency of the occurrence of change orders.The current study developed a foundation for further examination by proposing potential control measures,and testing their efficiency,such as integrating Virtual Reality(VR).The specific aim was to examine the effect of different visualization methods(i.e.,VR vs.construction drawings)on,(1)how well the subjects understand the information presented about the future/planned environment;(2)the subjects’perceived confidence in what the future environment will look like;(3)the likelihood of changing the built environment;(4)design review time;and(5)accuracy in reviewing and understanding the design.

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Robust analysis of discounted Markov decision processes with uncertain transition probabilities

Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.

Discrete-Time Hybrid Decision Processes: The Discounted Case

This paper is a sequel to Kageyama et al. [1], in which a Markov-type hybrid process has been constructed and the corresponding discounted total reward has been characterized by the recursive equation. The objective of this paper is to formulate a hybrid decision process and to give the existence and characterization of optimal policies.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

Adaptive Strategies for Accelerating the Convergence of Average Cost Markov Decision Processes Using a Moving Average Digital Filter

This paper proposes a technique to accelerate the convergence of the value iteration algorithm applied to discrete average cost Markov decision processes. An adaptive partial information value iteration algorithm is proposed that updates an increasingly accurate approximate version of the original problem with a view to saving computations at the early iterations, when one is typically far from the optimal solution. The proposed algorithm is compared to classical value iteration for a broad set of adaptive parameters and the results suggest that significant computational savings can be obtained, while also ensuring a robust performance with respect to the parameters.

Conditional Value-at-Risk for Random Immediate Reward Variables in Markov Decision Processes

We consider risk minimization problems for Markov decision processes. From a standpoint of making the risk of random reward variable at each time as small as possible, a risk measure is introduced using conditional value-at-risk for random immediate reward variables in Markov decision processes, under whose risk measure criteria the risk-optimal policies are characterized by the optimality equations for the discounted or average case. As an application, the inventory models are considered.

Underwater multiple target tracking decision making based on an analytic network process

Underwater multi-target tracking logic and decision （UMTLD） has difficulty resolving multi-target tracking problems for underwater vehicles. Present methods assume factors in UMTLD are uncorrelated, when these are actually in a complex, interdependent relationship. To provide this, an index set of multi-target tracking decision characteristics and an analytic network process （ANP） model of the UMTLD method was -established. This method brings the index set of multi-target tracking decision into the ANP model, and the optimization multitarket tracking decision is achieved via computation of the resulting supermatrix. The rationality and robustness of decision results increase in simulations by 13% and 47% respectively with analytic hierarchy process （AHP）. These results indicate that the ANP method should be the preferred method when UMTLD factors are interdependent.

Nitrogen removal influence factors in A/O process and decision trees for nitrification/denitrification system

In order to improve nitrogen removal in anoxic/oxic(A/O) process effectively for treating domestic wastewaters, the influence factors, DO(dissolved oxygen), nitrate recirculation, sludge recycle, SRT(solids residence time), influent COD/TN and HRT(hydraulic retention time) were studied. Results indicated that it was possible to increase nitrogen removal by using corresponding control strategies, such as, adjusting the DO set point according to effluent ammonia concentration; manipulating nitrate recirculation flow according to nitrate concentration at the end of anoxic zone. Based on the experiments results, a knowledge-based approach for supervision of the nitrogen removal problems was considered, and decision trees for diagnosing nitrification and denitrification problems were built and successfully applied to A/O process.

Application of Decision Tree Algorithm in Stamping Process

Various process parameters exert different effects in stamping process. In order to study the relationships among the process parameters of box stamping process, including the blank holder force, friction coefficient, depth of drawbead, offset and length of drawbead, the decision tree algorithm C4.5 was performed to generate the decision tree using the result data of the box stamping simulation. The design and improvement methods of the decision tree were presented. Potential and valuable rules were generated by traversing the decision tree, which plays an instructive role on the practical design. The rules show that the correct combination of blank holder force and setting of drawbead are the dominant contribution for controlling the cracking and wrinkling in box stamping process. In order to validate the rules, the stamping process for box was also performed. The experiment results show good agreement with the generated rules.

Study on Multi-enterprise Collaborative Resource Planning Based Process-oriented Decision Model

Aiming to share the information,knowledge and optimizing resource via collaborating with multiple external partners across their supply chains,the concept model and system framework of multi-enterprises collaborative resource planning (MECORP) are put forward.While there is Considerable pressure to improve the operation of MECORP system,their inherent complexity can make modelling a MECORP system a difficult task.Yet there could be considerable benefits in designing MECORP taking into account the operation of the system.In order to address the central research issue of developing of a methodology that can assist a manager in making decisions by modelling the operation of MECORP system.The methodology called process-oriented deci- sion model (PODM) is presented in the paper.This uses an abstracted network to model MECORP system.The MECORP system supported by PODM,can effective optimize the manifold resource,coordinate the relationship of multiple partners and assist deci- sion.Finally,an industry excample of MECORP system is described to illustrate the application of PODM.

An Investment Decision Support System for Process Industries

Most studies on investment evaluation mainly focus on enterprise economic benefits only, without process operability and sustainability considered. In this paper, we suggest that investment evaluation in process industries should be executed under three strategic objectives--enterprise benefits, social benefits and customer benefits. A systematic investment evaluation and decision-making method with a four-step procedure based on the analytic hierarchy process (AHP) is proposed to evaluate various qualitative and quantitative elements with various criteria. At the first step, the decision hierarchy is constructed under the three strategic objectives. Second, pair-wise comparison is utilized to evaluate the weights of elements and criteria. Third, qualitative elements are quantified by pair-wise comparison and quantitative elements are re-scaled by a uniform criterion. At the last, the best choice is made through synthesizing values upward in the hierarchy. An investment decision support system (DSS) is developed based on Microsoft Excel, and applied to a retrofit investment of united fluid catalytic cracking(FCC) and liquefied gas separation process in a refinery plant.

COLLISION AVOIDANCE DECISION-MAKING MODEL OF MULTI-AGENTS IN VIRTUAL DRIVING ENVIRONMENT WITH ANALYTIC HIERARCHY PROCESS

Collision avoidance decision-making models of multiple agents in virtual driving environment are studied. Based on the behavioral characteristics and hierarchical structure of the collision avoidance decision-making in real life driving, delphi approach and mathematical statistics method are introduced to construct pair-wise comparison judgment matrix of collision avoidance decision choices to each collision situation. Analytic hierarchy process （AHP） is adopted to establish the agents＇ collision avoidance decision-making model. To simulate drivers＇ characteristics, driver factors are added to categorize driving modes into impatient mode, normal mode, and the cautious mode. The results show that this model can simulate human＇s thinking process, and the agents in the virtual environment can deal with collision situations and make decisions to avoid collisions without intervention. The model can also reflect diversity and uncertainly of real life driving behaviors, and solves the multi-objective, multi-choice ranking priority problem in multi-vehicle collision scenarios. This collision avoidance model of multi-agents model is feasible and effective, and can provide richer and closer-to-life virtual scene for driving simulator, reflecting real-life traffic environment more truly, this model can also promote the practicality of driving simulator.

Description and analysis of non-stationary signals by nested semi-Markov processes

The possibility of describing the time-dependent processes of scattering by underlying surfaces and the clear sky, as well as the seasonal behaviour of the refractive index of troposphere by using nested semi-Markov processes has been consid- ered. Local Gaussian models can be used to describe the process inside each phase state. The possibility of describing the sta- tistics of reflections from the sea and the refractive index by using Kravchenko finite functions has been shown for the first time.

Some Considerations on Earthquake Prediction Decision Process

A theory on information prediction process proposed by Weng Wenpo(1991)is applied to the earthquake prediction decision process.Four cycles represent the theory(conception),earthquake prediction decision result,anomalies,and earthquake assemblage,respectively.The interception and overlapping of the four cycles indicate different combinations,resulting in formation of 13 regions.In the case of decision conclusion on earthquake to occur,seven decision results of different characters are distinguished.The six other results were obtained in the case of absence of decision.Results of four characters show correct decision on earthquake to occur and those of three characters show the erroneous decision on earthquake to occur.Until now,theories of earthquake prediction have been incomplete,and the coincidence ratio of decision on earthquake to occur is also considerably low.Systematic analysis of the decision process is beneficial to understanding the causes for missing,virtual,pseudo,false,and correct