Choice of discount rate in reinforcement learning with long-delay rewards

In the world, most of the successes are results of longterm efforts. The reward of success is extremely high, but before that, a long-term investment process is required. People who are “myopic” only value short-term rewards and are unwilling to make early-stage investments, so they hardly get the ultimate success and the corresponding high rewards. Similarly, for a reinforcement learning(RL) model with long-delay rewards, the discount rate determines the strength of agent’s “farsightedness”.In order to enable the trained agent to make a chain of correct choices and succeed finally, the feasible region of the discount rate is obtained through mathematical derivation in this paper firstly. It satisfies the “farsightedness” requirement of agent. Afterwards, in order to avoid the complicated problem of solving implicit equations in the process of choosing feasible solutions,a simple method is explored and verified by theoreti cal demonstration and mathematical experiments. Then, a series of RL experiments are designed and implemented to verify the validity of theory. Finally, the model is extended from the finite process to the infinite process. The validity of the extended model is verified by theories and experiments. The whole research not only reveals the significance of the discount rate, but also provides a theoretical basis as well as a practical method for the choice of discount rate in future researches.

Discount Rate of China’s New Energy Power Industry

Under the dual pressures of energy crisis and environmental pollution,China’s new energy power industry has become a focal point for environmental management and requires greater investment.In this context,as a significant input of investment projects,discount rate requires a well-calibrated evaluation because new energy power investment projects are highly capital intensive.The main objective of this paper is to evaluate the discount rate of China’s new energy power industry.First,we use Moving Average to correct the parameters of capital asset pricing model(CAPM)and weighted average cost of capital,which extends the literature on the avoidance of CAPM noise information problem.Second,we study the industry-level annual discount rates of mainly China’s new energy power industries,including hydropower,nuclear power,wind power,and photovoltaic power industries for the period of 2014-2019.The results show that discount rates in China’s new energy power industries evolved differently between the years of 2014-2019 with average annual discount rates being 7.56%,5.83%,5.60%,and 8.64%,for the hydropower,nuclear power,wind power,and photovoltaic power industries,respectively.In 2019,the four annual discount rates were highest for the photovoltaic power industry(8.66%),followed by hydropower(7.17%),wind power(5.72%),and nuclear power industry(5.26%).Forecasting to 2020 from the 2019 evaluation base period,the discount rates are 6.37%,5.00%,6.57%,and 9.05%for the photovoltaic power,hydropower,wind power,and nuclear power industries,respectively.Under the different capital structures,their forecasts for the photovoltaic power,hydropower,wind power,and nuclear power industries in 2020 are,respectively,within[4.35%,9.24%],[3.92%,7.10%],[4.58%,10.40%],[5.46%,14.81%].We also discussed more details on capital structure and forecast period of discount rates for China’s new energy power industries.Our analysis shows that it is necessary to establish a new energy power industry database and steadily promote the implementation of policies.

Research on Sales Strategy Based on Supermarket Pipeline Data

“Low profit and high sales” is a strategy to increase sales volume by reducing the profit of unit goods, so that businesses can gain more profits. For flexible goods, price reduction can increase the total revenue, but when the goods are lack of flexibility, price reduction will reduce the total revenue. In this paper, according to the sales data provided by a supermarket, we preprocess the data, establish appropriate indicators to measure the daily discount strength of the mall, and establish a mathematical model between the discount strength, sales and profit margin. Through these models, we found that meager profits do bring up sales, but too low discounts can also hurt total profits. In addition, when shopping malls implement discount promotions, they will also bring some negative effects, and we give some suggestions for this.

Exchange rate,discount rate,interest rate and depreciation rate

exchange rate:汇率,外汇汇率,指一个国家的货币和另一个国家的货币的比价。中国人民银行每天公布人民币市场汇价。如:market Exchange Rates in RMB yuan set by the People’s Bank ofChina(September 4,1998)Currency Unit RateUS dollar 100 828.00Japanese yen 100 6.0291German mark 100 478.

Estimation of lung cancer burden in Australia,the Philippines,and Singapore:an evaluation of disability adjusted life years

Objective: Lung cancer is one of the leading cancers and major causes of cancer mortality worldwide. The economic burden associated with the high mortality of lung cancer is high, which accounts for nearly $180 billion on a global scale in 2008. This paper aims to understand the economic burden of lung cancer in terms of disability adjusted life years(DALY) in Australia, the Philippines, and Singapore.Methods: The years of life lost(YLL) and years lost due to disability(YLD) were calculated using the formula developed by Murray and Lopez in 1996 as part of a comprehensive assessment of mortality and disability for diseases, injuries and risk factors in 1990 and projected to 2020. The same formula is represented in the Global Burden of Disease template provided by the World Health Organization. Appropriate assumptions were made when data were unavailable and projections were performed using regression analysis to obtain data for 2015.Results: The total DALYs due to lung cancer in Australia, the Philippines, and Singapore were 91,695, 38,584, and 12,435,respectively, and the corresponding DALY rates per a population of 1,000 were 4.0, 0.4, and 2.2, respectively, with a discount rate of 3%. When researchers calculated DALYs without the discount rate, the burden of disease increased substantially; the DALYs were 117,438 in Australia, 50,977 in the Philippines, and 16,379 in Singapore. Overall, YLL or premature death accounted for more than 95% of DALYs in these countries.Conclusions: Strategies for prevention, early diagnosis, and prompt treatment must be devised for diseases where the major burden is due to mortality.

Study of evaluation method for the overseas oil and gas investment based on risk compensation

The overseas oil and gas investment evaluation is one of the core tasks in overseas investment of oil and gas companies,among which risk evaluation and benefit evaluation are the most important.This paper sets forth transmission paths of risk factors to the investment benefit by identifying 14 overseas oil and gas investment risks in four categories.On the basis of the concept of risk compensation,different compensation mechanisms specific to each risk are designed.The risk and benefit are integrated objectively to develop a comprehensive evaluation model by correcting the recoverable reserve,adjusting benefit evaluation parameters such as investments on exploration and development,and compensating for the changes in risk factors with time through dynamic discount rate.Moreover,two cases studies,namely the evaluations of Project A in Sudan and comparison among Blocks A–G,are used to describe usage method and applicable scope of such evaluation model,respectively.According to the results,oil price is a key influencing factor for enterprise internal risk and industrial risk.Risk compensation reduces comprehensive benefit of overseas oil and gas investment and undermines the investment feasibility and priority of blocks.The research findings of this paper are free from the effects of some subject factors and avoid multi-objective decision making,and also avoid the undesired repeated calculation of risk factors.

Modified Ramsey Rule, Optimal Carbon Tax and Economic Growth

In contrast to the overlapping-generations model, it is allowable to discount the future utility in a dynasty model without the ethical difficulty related to intergenerational conflicts. Much precedent research uses Ramsey-type optimal growth theory in order to estimate the social discount rate. However, one must note that almost all the formulations neglect the existence of negative intertemporal externalities. This problem is vital when one analyzes the global warming problem mainly caused by the excess concentration of carbon dioxide (CO2). This is because an adjoining effect of capital accumulation exists besides the improvement of product capacity, which is reflected in the rate of interest (or equivalently, the marginal productivity of capital). That is, one cannot neglect a negative externality to the future productivity that originates from the excess emissions of CO2. Accordingly, following the optimal growth theory, the effective social discount rate should be heightened by a proportional carbon tax to suppress future excess consumption/ emissions than in the case of the existing analyses, which exclude such an intertemporal external diseconomy.

Modified Ramsey Rule and Optimal Carbon Tax

The Ramsey rule is regarded as a convenient vehicle for estimating the social discount rate in general. Carbon pricing is treated as another theory of environmental economics. This study clarifies the theoretical relationship between the Ramsey rule and optimal carbon price, which has been overlooked in the existing research. It succeeds in deriving the optimal carbon price from the modified Ramsey rule in stationary state. Since the Ramsey rule decides the dynamics of an economy and a stationary state is its destination, by using the optimization condition of individual who are assumed to live infinitesimally short life, we can solve the optimal carbon price at stationary state.

Gender difference in network relationship between inter-temporal decisions and prefrontal activation levels in internet gaming disorder

Background:Impulsivity and decision-making are key factors in addiction.However,little is known about how gender and time sensitivity affect impulsivity in internet gaming disorder(IGD).Objective:To investigate the gender difference of impulsive decision-making and relevant brain responses in IGD.Methods:We conducted a functional magnetic resonance imaging(fMRI)study with 123 participants,including 59 IGD individuals(26 females)and 64 matched recreational game users(RGUs,23 females).Participants performed a delay-discounting task during fMRI scanning.We examined gender-by-group effects on behavioral and neural measures to explore the preference for immediate over delayed rewards and the associated brain activity.We also investigated the network correlations between addiction severity and behavioral and neural measures,and analyzed the mediating role of brain activity in the link between delay discounting parameters and IGD severity.Results:We found significant gender-by-group interactions.The imaging results revealed gender-by-group interactions in the dor-solateral prefrontal cortex,medial frontal gyrus,and inferior frontal gyrus(IFG).Post hoc analysis indicated that,for females,RGUs showed higher activity than IGD individuals in these brain regions,while for males IGD individuals exhibited higher activity than RGUs.The activation in the left IFG mediated the relation between Internet Addiction Test score and discount rate in females.In males,the activation in the right dlPFC mediated the relation between IAT score and time sensitivity.Discussion:Our findings imply that male IGD participants demonstrate impaired intertemporal decisions associated with neural dysfunction.Influencing factors for impulsive decision-making in IGD diverge between males(time sensitivity)and females(discount rate).These findings augment our comprehension of the neural underpinnings of gender differences in IGD and bear significant implications for devising effective intervention strategies for treating people with IGD.