A minimization problem of the risk probability in first passage semi-Markov decision processes with loss rates

This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total loss incurred during a first passage time to some target set exceeds a loss level. First, we establish the optimality equation via a successive approximation technique, and show that the value function is the unique solution to the optimality equation. Second, we give suitable conditions, under which we prove the existence of optimal policies and develop an algorithm for computing ?-optimal policies. Finally, we apply our main results to a business system.

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.

Rationale for Decision-Making Processes in Enhancement of Community Participation for Sustainable Mangrove Management in Lamu, Kenya

Decision-making is the process of deciding between two or more options in order to take the most appropriate and successful course of action in order to achieve sustainable mangrove management. However, the distinctiveness of mangrove as an ecosystem, and thus the attendant socio-economic and governance ramifications, causes the idea of decision making to become relatively distinct from other decision making process As a result, the purpose of this research was to evaluate the impact that community engagement plays in the decision-making process as it relates to the establishment of governance norms for sustainable mangrove management in Lamu County. In this study, a correlational research design was applied, and the researchers employed a mixed techniques approach. The target population was 296 respondents. The research used questionnaires and interviews to collect data. A descriptive statistical technique was utilized to perform an inspection and analysis on the data that was gathered. The findings indicated that having awareness about governance standards is beneficial during the process of making decisions. In addition, the findings demonstrated that respondents had the impression that the decision-making process was not done properly. On the other hand, the participants pointed out the positive aspects of the decision-making process and agreed that the participation of both gender was essential for the sustainable management of mangroves. Based on these data, it appeared that full community engagement in decision-making is necessary for sustainable management of mangrove forests.

A Comparative Analysis of Visualization Methods in Architecture:Employing Virtual Reality to Support the Decision-Making Process in the Architecture,Engineering,and Construction Industry

The design process of the built environment relies on the collaborative effort of all parties involved in the project.During the design phase,owners,end users,and their representatives are expected to make the most critical design and budgetary decisions-shaping the essential traits of the project,hence emerge the need and necessity to create and integrate mechanisms to support the decision-making process.Design decisions should not be based on assumptions,past experiences,or imagination.An example of the numerous problems that are a result of uninformed design decisions is“change orders”,known as the deviation from the original scope of work,which leads to an increase of the overall cost,and changes to the construction schedule of the project.The long-term aim of this inquiry is to understand the user’s behavior,and establish evidence-based control measures,which are actions and processes that can be implemented in practice to decrease the volume and frequency of the occurrence of change orders.The current study developed a foundation for further examination by proposing potential control measures,and testing their efficiency,such as integrating Virtual Reality(VR).The specific aim was to examine the effect of different visualization methods(i.e.,VR vs.construction drawings)on,(1)how well the subjects understand the information presented about the future/planned environment;(2)the subjects’perceived confidence in what the future environment will look like;(3)the likelihood of changing the built environment;(4)design review time;and(5)accuracy in reviewing and understanding the design.

Remaining useful life prediction for engineering systems under dynamic operational conditions: A semi-Markov decision process-based approach

For critical engineering systems such as aircraft and aerospace vehicles, accurate Remaining Useful Life(RUL) prediction not only means cost saving, but more importantly, is of great significance in ensuring system reliability and preventing disaster. RUL is affected not only by a system's intrinsic deterioration, but also by the operational conditions under which the system is operating. This paper proposes an RUL prediction approach to estimate the mean RUL of a continuously degrading system under dynamic operational conditions and subjected to condition monitoring at short equi-distant intervals. The dynamic nature of the operational conditions is described by a discrete-time Markov chain, and their influences on the degradation signal are quantified by degradation rates and signal jumps in the degradation model. The uniqueness of our proposed approach is formulating the RUL prediction problem in a semi-Markov decision process framework, by which the system mean RUL can be obtained through the solution to a limited number of equations. To extend the use of our proposed approach in real applications, different failure standards according to different operational conditions are also considered. The application and effectiveness of this approach are illustrated by a turbofan engine dataset and a comparison with existing results for the same dataset.

Underwater multiple target tracking decision making based on an analytic network process

Underwater multi-target tracking logic and decision （UMTLD） has difficulty resolving multi-target tracking problems for underwater vehicles. Present methods assume factors in UMTLD are uncorrelated, when these are actually in a complex, interdependent relationship. To provide this, an index set of multi-target tracking decision characteristics and an analytic network process （ANP） model of the UMTLD method was -established. This method brings the index set of multi-target tracking decision into the ANP model, and the optimization multitarket tracking decision is achieved via computation of the resulting supermatrix. The rationality and robustness of decision results increase in simulations by 13% and 47% respectively with analytic hierarchy process （AHP）. These results indicate that the ANP method should be the preferred method when UMTLD factors are interdependent.

Nitrogen removal influence factors in A/O process and decision trees for nitrification/denitrification system

In order to improve nitrogen removal in anoxic/oxic(A/O) process effectively for treating domestic wastewaters, the influence factors, DO(dissolved oxygen), nitrate recirculation, sludge recycle, SRT(solids residence time), influent COD/TN and HRT(hydraulic retention time) were studied. Results indicated that it was possible to increase nitrogen removal by using corresponding control strategies, such as, adjusting the DO set point according to effluent ammonia concentration; manipulating nitrate recirculation flow according to nitrate concentration at the end of anoxic zone. Based on the experiments results, a knowledge-based approach for supervision of the nitrogen removal problems was considered, and decision trees for diagnosing nitrification and denitrification problems were built and successfully applied to A/O process.

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Robust analysis of discounted Markov decision processes with uncertain transition probabilities

Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.

Discrete-Time Hybrid Decision Processes: The Discounted Case

This paper is a sequel to Kageyama et al. [1], in which a Markov-type hybrid process has been constructed and the corresponding discounted total reward has been characterized by the recursive equation. The objective of this paper is to formulate a hybrid decision process and to give the existence and characterization of optimal policies.

Application of Decision Tree Algorithm in Stamping Process

Various process parameters exert different effects in stamping process. In order to study the relationships among the process parameters of box stamping process, including the blank holder force, friction coefficient, depth of drawbead, offset and length of drawbead, the decision tree algorithm C4.5 was performed to generate the decision tree using the result data of the box stamping simulation. The design and improvement methods of the decision tree were presented. Potential and valuable rules were generated by traversing the decision tree, which plays an instructive role on the practical design. The rules show that the correct combination of blank holder force and setting of drawbead are the dominant contribution for controlling the cracking and wrinkling in box stamping process. In order to validate the rules, the stamping process for box was also performed. The experiment results show good agreement with the generated rules.

Study on Multi-enterprise Collaborative Resource Planning Based Process-oriented Decision Model

Aiming to share the information,knowledge and optimizing resource via collaborating with multiple external partners across their supply chains,the concept model and system framework of multi-enterprises collaborative resource planning (MECORP) are put forward.While there is Considerable pressure to improve the operation of MECORP system,their inherent complexity can make modelling a MECORP system a difficult task.Yet there could be considerable benefits in designing MECORP taking into account the operation of the system.In order to address the central research issue of developing of a methodology that can assist a manager in making decisions by modelling the operation of MECORP system.The methodology called process-oriented deci- sion model (PODM) is presented in the paper.This uses an abstracted network to model MECORP system.The MECORP system supported by PODM,can effective optimize the manifold resource,coordinate the relationship of multiple partners and assist deci- sion.Finally,an industry excample of MECORP system is described to illustrate the application of PODM.

An Investment Decision Support System for Process Industries

Most studies on investment evaluation mainly focus on enterprise economic benefits only, without process operability and sustainability considered. In this paper, we suggest that investment evaluation in process industries should be executed under three strategic objectives--enterprise benefits, social benefits and customer benefits. A systematic investment evaluation and decision-making method with a four-step procedure based on the analytic hierarchy process (AHP) is proposed to evaluate various qualitative and quantitative elements with various criteria. At the first step, the decision hierarchy is constructed under the three strategic objectives. Second, pair-wise comparison is utilized to evaluate the weights of elements and criteria. Third, qualitative elements are quantified by pair-wise comparison and quantitative elements are re-scaled by a uniform criterion. At the last, the best choice is made through synthesizing values upward in the hierarchy. An investment decision support system (DSS) is developed based on Microsoft Excel, and applied to a retrofit investment of united fluid catalytic cracking(FCC) and liquefied gas separation process in a refinery plant.

COLLISION AVOIDANCE DECISION-MAKING MODEL OF MULTI-AGENTS IN VIRTUAL DRIVING ENVIRONMENT WITH ANALYTIC HIERARCHY PROCESS

Collision avoidance decision-making models of multiple agents in virtual driving environment are studied. Based on the behavioral characteristics and hierarchical structure of the collision avoidance decision-making in real life driving, delphi approach and mathematical statistics method are introduced to construct pair-wise comparison judgment matrix of collision avoidance decision choices to each collision situation. Analytic hierarchy process （AHP） is adopted to establish the agents＇ collision avoidance decision-making model. To simulate drivers＇ characteristics, driver factors are added to categorize driving modes into impatient mode, normal mode, and the cautious mode. The results show that this model can simulate human＇s thinking process, and the agents in the virtual environment can deal with collision situations and make decisions to avoid collisions without intervention. The model can also reflect diversity and uncertainly of real life driving behaviors, and solves the multi-objective, multi-choice ranking priority problem in multi-vehicle collision scenarios. This collision avoidance model of multi-agents model is feasible and effective, and can provide richer and closer-to-life virtual scene for driving simulator, reflecting real-life traffic environment more truly, this model can also promote the practicality of driving simulator.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

Some Considerations on Earthquake Prediction Decision Process

A theory on information prediction process proposed by Weng Wenpo(1991)is applied to the earthquake prediction decision process.Four cycles represent the theory(conception),earthquake prediction decision result,anomalies,and earthquake assemblage,respectively.The interception and overlapping of the four cycles indicate different combinations,resulting in formation of 13 regions.In the case of decision conclusion on earthquake to occur,seven decision results of different characters are distinguished.The six other results were obtained in the case of absence of decision.Results of four characters show correct decision on earthquake to occur and those of three characters show the erroneous decision on earthquake to occur.Until now,theories of earthquake prediction have been incomplete,and the coincidence ratio of decision on earthquake to occur is also considerably low.Systematic analysis of the decision process is beneficial to understanding the causes for missing,virtual,pseudo,false,and correct

Adaptive Strategies for Accelerating the Convergence of Average Cost Markov Decision Processes Using a Moving Average Digital Filter

This paper proposes a technique to accelerate the convergence of the value iteration algorithm applied to discrete average cost Markov decision processes. An adaptive partial information value iteration algorithm is proposed that updates an increasingly accurate approximate version of the original problem with a view to saving computations at the early iterations, when one is typically far from the optimal solution. The proposed algorithm is compared to classical value iteration for a broad set of adaptive parameters and the results suggest that significant computational savings can be obtained, while also ensuring a robust performance with respect to the parameters.

Development of a Web-Based Decision Support System for Cell Formation Problems Considering Alternative Process Routings and Machine Sequences

In this study, we use the respective advantages of the tabu search (TS) and the Web-based technologies to develop a Web-based decision support system (DSS) for cell formation (CF) problems considering alternative process routings and machine sequences simultaneously. With the assistance of our developed Web-based system, the CF practitioners in the production departments can interact with the systems without knowing the details of algorithms and can get the best machine cells and part families with minimize the total intercellular movement wherever and whenever they may need it. To further verify the feasibility and effectiveness of the system developed, an example taken from the literature is ado- pted for illustrational purpose. Moreover, a set of test problems with various sizes drawn from the literature is used to test the performance of the proposed system. Corresponding results are compared to several well-known algorithms previously published. The results indicate that the proposed system improves the best results found in the literature for 67% of the test problems. These show that the proposed system should thus be useful to both practitioners and researchers.