Robust analysis of discounted Markov decision processes with uncertain transition probabilities

Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.

Variable structure control for descriptor Markovian jump systems subject to partially unknown transition probabilities

The descriptor Markovian jump systems( DMJSs)with partially unknown transition probabilities( PUTPs) are studied by means of variable structure control. First,by virtue of the strictly linear matrix inequality( LMI) technique,a sufficient condition is presented, under which the DMJSs subject to PUTPs are stochastically admissible. Secondly,a novel sliding surface function based on the system state and input is constructed for DMJSs subject to PUTPs; and a dynamic sliding mode controller is synthesized, which guarantees that state trajectories will reach the pre-specified sliding surface in finite time despite uncertainties and disturbances. The results indicate that by checking the feasibility of a series of LMIs,the stochastic admissibility of the overall closed loop system is determined. Finally,the validity of the theoretical results is illustrated with the example of the direct-current motor. Furthermore,compared with the existing literature,the state convergence rate,buffeting reduction and overshoot reduction are obviously optimized.

Transition probabilities for NII 2p4f-2p3d and 2s2p^23d-2s2p^23p obtained by a semiclassical method

Based on NII spectra, some transition probabilities for 2p4f-2p3d and 2s2p23d-2s2p23p are obtained by a semi- classical method. The results are in good agreement with other measurements and the data reported by the National Institute of Standards and Technology. The transition probability for a line of 424.18nm is reported for the first time. Meanwhile, a feasible method of calculating transition parameters related to special excited configurations or highly excited states is provided.

Theoretical Calculations of Transition Probabilities and Oscillator Strengths for Sc（Ⅲ） and Y（Ⅲ）

The Weakest Bound Electron Potential Model theory is used to calculate transition probability-values and oscillator strength-values for individual lines of Sc（Ⅲ） and Y（Ⅲ）. In this method, by solving the SchrSdinger equation of the weakest bound electron, the expressions of energy eigenvalue and the radial function can be obtained. And a coupled equation is used to determine the parameters which are needed in the calculations. The ob- tained results of Sc（III） from this work agree very well with the accepted values taken from the National Institute of Standards and Technoligy （NIST） data base, most deviations are within the accepted level. For Y（Ⅲ） there are no accepted values reported by the NIST data base. So we compared our results of Y（Ⅲ） with other theoretical results, good agreement is also obtained.

Electric-dipole allowed(E1) and forbidden(E2,M1 and M2) transition probabilities of 4f for N^+

By applying systematically enlarged multi-configuration Dirac-Fock wavefunction, the transitions for electricdipole allowed （E1） and forbidden （E2, M1 and M2） lines are studied among 4f pair coupling and low-lying configurations for singly ionized nitrogen. Most important effects of relativity, electron correlation, the rearrangement of electron density, Breit interaction, and quantum electrodynamic effects are included in the computation. Then, allowed （E1） and forbidden （E2, M1 and M2） transition probabilities of 4f for N＋ are obtained and compared with experimental results. Good agreement with available experimental results is found and most of the data of 4f are presented for the first time.

Transition probabilities and lifetimes of the low-lying levels of Fe XIV

The multi-configuration Dirac-Fock method is employed to calculate the transition energies, probabilities, and oscillator strengths for electric dipole allowed （El） and forbidden （M1, E2, M2） lines for the 3s^23p, 3s3p^2, 3s^23d, 3p^3, and 3s3p3d configurations of Fe XIV. The lifetimes of all 40 levels of these low-lying configurations are also derived. The valence valence and core-valence correlation effects are accounted for in a systematic way. Breit interactions and quantum electrodynamics （QED） effects are estimated in subsequent relativistic configuration interaction （CI） calculations. The present results are in good agreement with other available theoretical and experimental values, and therefore can be used for the further astrophysical investigations.

Upward Transition Probabilities B(E2)↑ Properties Study of Even-Even ^104-114Ru Nuclei

In this research work, the upward transition probabilities for the transition levels, 0+ → 2+, 2+ → 4+, 4+ → 6+ and 6+ → 8+ levels of even-even neutron rich 104-114Ru isotopes have been calculated by using the Global Best Fit (GBF) method. In addition, the associated parameters such as, Quadrupole moment and Deformation parameter of even-even 104-114Ru have been calculated. The dependency of these nuclear parameters shows the nuclear magic number tendency.

Confidence intervals for Markov chain transition probabilities based on next generation sequencing reads data

Background:Markov chains(MC)have been widely used to model molecular sequences.The estimations of MC transition matrix and confidence intervals of the transition probabilities from long sequence data have been intensively studied in the past decades.In next generation sequencing(NGS),a large amount of short reads are generated.These short reads can overlap and some regions of the genome may not be sequenced resulting in a new type of data.Based on NGS data,the transition probabilities of MC can be estimated by moment estimators.However,the classical asymptotic distribution theory for MC transition probability estimators based on long sequences is no longer valid.Methods:In this study,we present the asymptotic distributions of several statistics related to MC based on NGS data.We show that,after scaling by the effective coverage d defined in a previous study by the authors,these statistics based on NGS data approximate to the same distributions as the corresponding statistics for long sequences.Results:We apply the asymptotic properties of these statistics for finding the theoretical confidence regions for MC transition probabilities based on NGS short reads data.We validate our theoretical confidence intervals using both simulated data and real data sets,and compare the results with those by the parametric bootstrap method.Conclusions:We find that the asymptotic distributions of these statistics and the theoretical confidence intervals of transition probabilities based on NGS data given in this study are highly accurate,providing a powerful tool for NGS data analysis.

Reachable set estimation for discrete-time Markovian jump neural networks with unified uncertain transition probability

This paper focuses on the reachable set estimation for Markovian jump neural networks with time delay.By allowing uncertainty in the transition probabilities,a framework unifies and enhances the generality and realism of these systems.To fully exploit the unified uncertain transition probabilities,an equivalent transformation technique is introduced as an alternative to traditional estimation methods,effectively utilizing the information of transition probabilities.Furthermore,a vector Wirtinger-based summation inequality is proposed,which captures more system information compared to existing ones.Building upon these components,a novel condition that guarantees a reachable set estimation is presented for Markovian jump neural networks with unified uncertain transition probabilities.A numerical example is illustrated to demonstrate the superiority of the approaches.

Reliability Analysis of Multi-State Repairable System with Dynamic Transition Probabilities

Inadequate maintenance decisions lead to incremental overall costs. In order to minimize costs in maintenance of the multi-state repairable system, we model a preventive maintenance(PM) scheme of the multistate repairable system using non-Markov process. The periodically decreasing reliability model of the non-Markov dynamic system with dynamic transition probabilities is established to satisfy the probability change. The diesel engine system is taken as an example to illustrate the model. The reliability of the diesel engine is analyzed and its PM scheme is worked out. RENO software is used to simulate the diesel engine system. The maintenance cost of components and the optimal PM interval data of the system are obtained by using the minimal average cost as the objective function. The adaptability of PM is judged, and the optimal PM scheme is presented.

Determination of the 5d6p 3F4-5d2 3F transition probabilities of Ba I

Whether the transitions between 6s5d 3D and 5d6p 3F can be used for laser cooling of barium heavily depends upon the transition probabilities of 5d6p 3F-5d2 3F. Taking the transition 6s5d 3D3-5d6p 3F4 as a scale, the leakage rate of 5d6p 3Fa-5d2 3F was used to evaluate the transition probabilities. 706 nm laser pulses with different durations were applied to a barium atomic beam for 6s5d 3D3-5d2 3F4 optical pumping, and the remaining percentages in 6s5d 3D3 were measured. After exponential fitting, the transition probability of 5d6p 3F4-5d2 3F3,4 was determined to be 2.1（4） ~ 104 s-1, which is in agreement with theoretical calculations using the scaled Thomas-Fermi-Dirac method.

A Game-Theoretic Approach to Safe Crowd Evacuation in Emergencies

Obstacle removal in crowd evacuation is critical to safety and the evacuation system efficiency. Recently, manyresearchers proposed game theoreticmodels to avoid and remove obstacles for crowd evacuation. Game theoreticalmodels aim to study and analyze the strategic behaviors of individuals within a crowd and their interactionsduring the evacuation. Game theoretical models have some limitations in the context of crowd evacuation. Thesemodels consider a group of individuals as homogeneous objects with the same goals, involve complex mathematicalformulation, and cannot model real-world scenarios such as panic, environmental information, crowds that movedynamically, etc. The proposed work presents a game theoretic model integrating an agent-based model to removethe obstacles from exits. The proposed model considered the parameters named: (1) obstacle size, length, andwidth, (2) removal time, (3) evacuation time, (4) crowd density, (5) obstacle identification, and (6) route selection.The proposed work conducts various experiments considering different conditions, such as obstacle types, obstacleremoval, and several obstacles. Evaluation results show the proposed model’s effectiveness compared with existingliterature in reducing the overall evacuation time, cell selection, and obstacle removal. The study is potentially usefulfor public safety situations such as emergency evacuations during disasters and calamities.

Robustness of the octupole collectivity in 144Ba within the cranking covariant density functional theory in 3D lattice

The octupole deformation and collectivity in octupole double-magic nucleus 144Ba are investigated using the Cranking covariant density functional theory in a three-dimensional lattice space.The reduced B(E3)transition probability is implemented for the first time in semiclassical approximation based on the microscopically calculated electric octupole moments.The available data,including the I-ωrelation and electric transitional probabilities B(E2)and B(E3)are well reproduced.Furthermore,it is shown that the ground state of 144Ba exhibits axial octupole and quadrupole deformations that persist up to high spins(I≈24h).

Emission Intensity in the Hydrogen Atom Calculated from a Non-Probabilistic Approach to the Electron Transitions

Quantum aspects of the Joule-Lenz law for the transmission of energy allowed us to calculate the time rate of energy transitions between the quantum states of the hydrogen atom in a fully non-probabilistic way. The calculation has been extended to all transitions between p and s states having main quantum numbers not exceeding 6. An evident similarity between the intensity pattern obtained from the Joule-Lenz law and the corresponding quantum-mechanical transition pro-babilities has been shown.

Forbidden transition properties of fine-structure 2p^3 ~4S_(3/2)–2p^3 ~2D_(3/2,5/2) for nitrogen-like ions

Based on relativistic wave functions from multiconfiguration Dirac-Hartree-Fock and configuration interaction cal- culations, E2 and M1 transition probabilities of 2p3 4S3/2-2p3 2D3/2,5/2 are investigated in the nitrogen-like sequence with 7 〈 Z 〈 16. The contributions of the electron correlations, Breit interaction, and the quantum electrodynamic （QED） effects on the transition properties are analyzed. The present results can be used for diagnosing plasma. In addition, several N-like ions can also be recommended as a promising candidate for a highly charged ion （HCI） clock with a quality factor （Q） of trnnsition as hitch as 1020.

Transition parameters of Li-like ions(Z=7-11)in dense plasmas

The energy levels,transition energies,transition probabilities,weighted oscillator strengths,and line strengths of Li-like ions(Z=7-11)in dense plasmas are investigated in this work.The relativistic effects and electron correlation effects are described by the MCDHF method.The ion sphere model is applied to include the dense plasma screening effect.The ground configuration 1s^(2)2s and the excited 1s^(2)2p,1s^(2)3l(l=0-2)are considered.The configuration sets are enlarged until n=7 where the calculated energy levels have converged.The critical free electron densities of 1s^(2)3d states are estimated.Except for 1s^(2)3s-1s^(2)3p transitions,the transition energies for△n=0 increase,and for△n≠0 decrease with increasing free electron densities.For 1s^(2)3s-1s^(2)3p transitions,the spectra show blue-shift at lower free electron densities and red-shift at higher free electron densities,and the energy level crossing phenomens are observed at higher free electron densities.

Theoretical investigation on forbidden transition properties of fine-structure splitting in ~2D state for K-like ions with 26≤Z≤36

Excitation energies, magnetic dipole, and electric quadrupole transition probabilities of the 3d ^2D3/2-3d ^2D5/2 transition in the potassium-like （K-like） sequence with 26 ≤ Z ≤ 36 are investigated by using the multi-configuration Dirac-Hartree-Fock （MCDHF） method. The contributions of the electron correlations, Breit interaction, and the leading-order quantum electrodynamic （QED） effects on the transition properties are analyzed. The present results are interested in the laboratory tokamak and the astronomical observations. Furthermore, the feasibility of these ions for the highly charged ion （HCI） clocks is discussed. Considering the wavelength of lasers and manipulation process of the atomic clocks, Cu^10＋ and Zn^11＋ are recommended as promising candidates with achievable quality factors at the 10^15 level.

Development of diabetic complications and influencing factors among 32653 type 2 diabetes patients:retrospective cohort study using a multi-state Markov

Background:Patients with type 2 diabetes are at high risk for developing multiple chronic complications.However,there is a lack of studies of the cumulative number of diabetic complications in China.Methods:A retrospective cohort study was performed from 2009 to 2021.Type 2 diabetes patients who were first diagnosed after the age of 35 years between January 1,2009,and December 31,2017,were included.Five states were defined according to the number of chronic complications:no(S0),one(S1),two(S2),three(S3),and four or more complications(S4).A multi-state Markov model was constructed to estimate transition probability,transition intensity,mean sojourn time,and the possible factors for each state.Results:The study included 32653 type 2 diabetes patients(mean age,59.59 years;15929(48.8%)male),and mean follow-up time of 7.75 years.In all,4375 transitions were observed.The 12-year transition probability of from state S0 to S1 was the lowest at 16.4%,while that from S2 to S3 was the highest,at 45.6%.Higher fasting blood glucose,lower high-density lipoprotein cholesterol,higher total cholesterol,and an unhealthy diet were associated with higher risk of progression from S0 to S1.Being female,less than 60 years old,weekly physical activity,and vegetarian diet decreased this risk.Being female and less than 60 years old reduced the likelihood of transition from S1 to S2,whereas lower high-density lipoprotein cholesterol increased this likelihood.Conclusions:Following the occurrence of two complications in type 2 diabetes patients,the risk for accumulating a third complication within a short time is significantly increased.It is important to take advantage of the stable window period when patients have fewer than two complications,strengthen the monitoring of blood glucose and blood lipids,and encourage patients to maintain good living habits to prevent further deterioration.

Necessity of Exact Calculation for Transition Probability

This paper shows that exact calculation for transition probability can make some systems deviate fromFermi golden rule seriously. This paper also shows that the corresponding exact calculation of hopping rate inducedby phonons for deuteron in Pd-D system with the many-body electron screening, proposed by Ichimaru, can explainthe experimental fact observed in Pd-D system, and predicts that perfection and low-dimension of Pd lattice are veryimportant for the phonon-induced hopping rate enhancement in Pd-D system.

Extreme ultraviolet and x-ray transition wavelengths in Rb XXIV

Energy levels, radiative rates, oscillator strengths and line strengths are reported for transitions among the lowest 97 levels of the（1s22s22p6） 3s23p2, 3s23p3 d, 3s3p3, 3p4, 3s3p23 d, and 3s23d2 configurations of Rb XXIV. A multiconfiguration Dirac-Fock（MCDF） method is adopted for the calculations. Radiative rates, oscillator strengths, and line strengths are provided for all electric dipole（E1）, magnetic dipole（M1）, electric quadrupole（E2）, and magnetic quadrupole（M2）transitions from the ground level to all 97 levels, although calculations are performed for a much larger number of levels.To achieve the accuracy of the data, comparisons are provided with similar data obtained from the Flexible Atomic Code（FAC） and also with the available theoretical and experimental results. Our energy levels are found to be accurate to better than 1.2%. Wavelengths calculated are found to lie in EUV（extreme ultraviolet） and x-ray regions. Additionally, lifetimes for all 97 levels are obtained for the first time.