Improved Double Deep Q Network Algorithm Based on Average Q-Value Estimation and Reward Redistribution for Robot Path Planning

By integrating deep neural networks with reinforcement learning,the Double Deep Q Network(DDQN)algorithm overcomes the limitations of Q-learning in handling continuous spaces and is widely applied in the path planning of mobile robots.However,the traditional DDQN algorithm suffers from sparse rewards and inefficient utilization of high-quality data.Targeting those problems,an improved DDQN algorithm based on average Q-value estimation and reward redistribution was proposed.First,to enhance the precision of the target Q-value,the average of multiple previously learned Q-values from the target Q network is used to replace the single Q-value from the current target Q network.Next,a reward redistribution mechanism is designed to overcome the sparse reward problem by adjusting the final reward of each action using the round reward from trajectory information.Additionally,a reward-prioritized experience selection method is introduced,which ranks experience samples according to reward values to ensure frequent utilization of high-quality data.Finally,simulation experiments are conducted to verify the effectiveness of the proposed algorithm in fixed-position scenario and random environments.The experimental results show that compared to the traditional DDQN algorithm,the proposed algorithm achieves shorter average running time,higher average return and fewer average steps.The performance of the proposed algorithm is improved by 11.43%in the fixed scenario and 8.33%in random environments.It not only plans economic and safe paths but also significantly improves efficiency and generalization in path planning,making it suitable for widespread application in autonomous navigation and industrial automation.

Evolutionary dynamics of tax-based strong altruistic reward andpunishment in a public goods game

In public goods games, punishments and rewards have been shown to be effective mechanisms for maintaining individualcooperation. However, punishments and rewards are costly to incentivize cooperation. Therefore, the generation ofcostly penalties and rewards has been a complex problem in promoting the development of cooperation. In real society,specialized institutions exist to punish evil people or reward good people by collecting taxes. We propose a strong altruisticpunishment or reward strategy in the public goods game through this phenomenon. Through theoretical analysis and numericalcalculation, we can get that tax-based strong altruistic punishment (reward) has more evolutionary advantages thantraditional strong altruistic punishment (reward) in maintaining cooperation and tax-based strong altruistic reward leads toa higher level of cooperation than tax-based strong altruistic punishment.

Evolutionary analysis of green credit and automobile enterprises under the mechanism of dynamic reward and punishment based on government regulation

To explore the green development of automobile enterprises and promote the achievement of the“dual carbon”target,based on the bounded rationality assumptions,this study constructed a tripartite evolutionary game model of gov-ernment,commercial banks,and automobile enterprises;introduced a dynamic reward and punishment mechanism;and analyzed the development process of the three parties’strategic behavior under the static and dynamic reward and punish-ment mechanism.Vensim PLE was used for numerical simulation analysis.Our results indicate that the system could not reach a stable state under the static reward and punishment mechanism.A dynamic reward and punishment mechanism can effectively improve the system stability and better fit real situations.Under the dynamic reward and punishment mechan-ism,an increase in the initial probabilities of the three parties can promote the system stability,and the government can im-plement effective supervision by adjusting the upper limit of the reward and punishment intensity.Finally,the implementa-tion of green credit by commercial banks plays a significant role in promoting the green development of automobile enter-prises.

Efficient Optimal Routing Algorithm Based on Reward and Penalty for Mobile Adhoc Networks

Mobile adhoc networks have grown in prominence in recent years,and they are now utilized in a broader range of applications.The main challenges are related to routing techniques that are generally employed in them.Mobile Adhoc system management,on the other hand,requires further testing and improvements in terms of security.Traditional routing protocols,such as Adhoc On-Demand Distance Vector(AODV)and Dynamic Source Routing(DSR),employ the hop count to calculate the distance between two nodes.The main aim of this research work is to determine the optimum method for sending packets while also extending life time of the network.It is achieved by changing the residual energy of each network node.Also,in this paper,various algorithms for optimal routing based on parameters like energy,distance,mobility,and the pheromone value are proposed.Moreover,an approach based on a reward and penalty system is given in this paper to evaluate the efficiency of the proposed algorithms under the impact of parameters.The simulation results unveil that the reward penalty-based approach is quite effective for the selection of an optimal path for routing when the algorithms are implemented under the parameters of interest,which helps in achieving less packet drop and energy consumption of the nodes along with enhancing the network efficiency.

Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning(RL)is an interesting extension of the traditional RL framework,where the dynamic environment and reward sparsity can cause conventional learning algorithms to fail.Reward shaping is a practical approach to improving sample efficiency by embedding human domain knowledge into the learning process.Existing reward shaping methods for goal-conditioned RL are typically built on distance metrics with a linear and isotropic distribution,which may fail to provide sufficient information about the ever-changing environment with high complexity.This paper proposes a novel magnetic field-based reward shaping(MFRS)method for goal-conditioned RL tasks with dynamic target and obstacles.Inspired by the physical properties of magnets,we consider the target and obstacles as permanent magnets and establish the reward function according to the intensity values of the magnetic field generated by these magnets.The nonlinear and anisotropic distribution of the magnetic field intensity can provide more accessible and conducive information about the optimization landscape,thus introducing a more sophisticated magnetic reward compared to the distance-based setting.Further,we transform our magnetic reward to the form of potential-based reward shaping by learning a secondary potential function concurrently to ensure the optimal policy invariance of our method.Experiments results in both simulated and real-world robotic manipulation tasks demonstrate that MFRS outperforms relevant existing methods and effectively improves the sample efficiency of RL algorithms in goal-conditioned tasks with various dynamics of the target and obstacles.

Effectiveness of Reward System on Assessment Outcomes in Mathematics

As assessment outcomes provide students with a sense of accomplishment that is boosted by the reward system,learning becomes more effective.This research aims to determine the effects of reward system prior to assessment in Mathematics.Quasi-experimental research design was used to examine whether there was a significant difference between the use of reward system and students’level of performance in Mathematics.Through purposive sampling,the respondents of the study involve 80 Grade 9 students belonging to two sections from Gaudencio B.Lontok Memorial Integrated School.Based on similar demographics and pre-test results,control and study group were involved as participants of the study.Data were treated and analyzed accordingly using statistical treatments such as mean and t-test for independent variables.There was a significant finding revealing the advantage of using the reward system compare to the non-reward system in increasing students’level of performance in Mathematics.It is concluded that the use of reward system is effective in improving the assessment outcomes in Mathematics.It is recommended to use reward system for persistent assessment outcomes prior to assessment,to be a reflection of the intended outcomes in Mathematics.

Generational Gap： Intrinsic （Non-monetary） Versus Extrinsic （Monetary） Rewards in the Workforce

Traditionally, organizations assume that compensation/pay and monetary benefits are what all employees need to work harder, be productive, or remain with the company. According to Abraham Maslow, within every person is a hierarchy of five needs： physiological needs, safety needs, social needs, esteem needs, and self-actualization needs Organizations must be able to identify what employees desire to secure optimum performance and to meet the needs of both employees and employers. This research focuses on the generational gap and the significance of intrinsic and extrinsic rewards in the workforce. The purpose and objective of this research are to test the significance of monetary versus non-monetary rewards among the different generations in the organization. A self-designed questionnaire distributed to a multi-generational group of employees of selected organizations was used to collect the analyzed data. Sixty-five （65%） responses were obtained. Secondary data were used to elucidate the needs in this area of study. Because the workforce is predicted to become more diverse in terms of age, organizations will be unlikely to implement one set of rewards for the multiple generations. This is due to the differing expectations and requirements among the generations. However, the results indicate no significant difference in monetary versus non-monetary rewards among the different generations in the workforce.

Effects of Grassland Eco-Protection Compensation and Reward System

Grassland ecological protection compensation and reward policy is the largest-scale investment concerning themost extensive areas since foundation of the PRC. It will be the long-term implementationpolicy for grassland ecological protection. In this study,based on macro-perspective, the policy effects ofgrasslandproductivity, ecological protection, animal husbandryoutput, pastoralists＇ income were ana- lyzed. The resultsshow that, afterimplementation of the policy, naturalgrass production and grasslandtheoretical stocking rateincreased. The averagenatural grasslandlivestockoverloading ratedecreased significantly, comprehensivenationalgrasslandvegetation coverageis increasing. Besides, adult cattleandbeef yield arefluctuated. Sheep head, adult sheep, sheep production, milk productionincreasedin varying degrees. The per capita netincomeof farmers and pastoralists, livestock income, the proportion oflivestockincomewere higher than those beforeimplementation of the policy.

Delta EEG Activity in Left Orbitofrontal Cortex in Rats Related to Food Reward and Craving

The orbitofrontal cortex （OFC） is particularly important for the neural representation of reward value. Previous studies indicated that electroencephalogram （EEG） activity in the OFC was involved in drug administration and withdrawal. The present study investigated EEG activity in the OFC in rats during the development of food reward and craving. Two environments were used separately for control and food-related EEG recordings. In the food-related environment rats were first trained to eat chocolate peanuts; then they either had no access to this food, but could see and smell it （craving trials）, or had free access to this food （reward trials）. The EEG in the left OFC was recorded during these trials. We showed that, in the food-related environment the EEG activity peaking in the delta band （2-4 Hz） was significantly correlated with the stimulus, increasing during food reward and decreasing during food craving when compared with that in the control environment. Our data suggests that EEG activity in the OFC can be altered by food reward; moreover, delta rhythm in this region could be used as an index monitoring changed signal underlying this reward.

A Functional Inhibitory Role of Habenular Glucagon-Like Peptide-1 (GLP-1) in Forebrain Reward Signaling

There is emerging evidence implicating glucagon-like peptide-1 (GLP-1) in reward, including palatable food reinforcement and alcohol-based reward circuitry. While recent findings suggest that mesolimbic structures, such as the ventral tegmental area (VTA) and the nucleus accumbens (NAc), are critical anatomical sites mediating the role of GLP-1’s inhibitory actions, the present study focused on the potential novel impact of GLP-1 within the habenula, a region of the forebrain expressing GLP-1 receptors. Given that the habenula has also been implicated in the neural control of reward and reinforcement, we hypothesized that this brain region, like the VTA and NAc, might mediate the anhedonic effects of GLP-1. Rats were stereotaxically implanted with guide cannula targeting the habenula and trained on a progressive ratio 3 (PR3) schedule of reinforcement. Separate rats were trained on an alcohol two-bottle choice paradigm with intermittent access. The GLP-1 agonist exendin-4 (Ex-4) was administered directly into the habenula to determine the effects on operant responding for palatable food as well as alcohol intake. Our results indicated that Ex-4 reliably suppressed PR3 responding and that this effect was dose-dependent. A similar suppressive effect on alcohol consumption was observed. These findings provide initial and compelling evidence that the habenula may mediate the inhibitory action of GLP-1 on reward, including operant and drug reward. Our findings further suggest that GLP-1 receptor mechanisms outside of the midbrain and ventral striatum are critically involved in brain reward neurotransmission.

On Principle of Rewards in English Learning

There is no question that learning a foreign language like English is different from learning other subjects, mainly because it is new to us Chinese and there is no enough environment. But that doesn’t mean we have no way to learn it and do it well .If asked to identify the most powerful influences on learning, motivation would probably be high on most teachers’ and learners’ lists. It seems only sensible to assume that English learning is most likely to occur when the learners want to learn. That is, when motivation such as interest, curiosity, or a desire achieves, the learners would be engaged in learning. However, how do we teachers motivate our students to like learning and learn well? Here, rewards both extrinsic and intrinsic are of great value and play a vital role in English learning.

Discussion on the Effectiveness of Educational Reward

The psychological mechanism of reward is to form operational conditioned reflex through positive reinforcement and negative reinforcement.The positive effect of reward is to strengthen external learning motivation,and reward can sometimes improve creativity.The negative effects are:weakening students'creativity,weakening the internal motivation of learning and hindering the development of autonomy.Teachers should apply educational rewards scientifically,take care of their age,consider the difficulty of tasks,pay attention to stimulating students'internal motivation,and give priority to spiritual rewards,supplemented by material rewards.

Co-effect of Demand-control-support Model and Effort-reward Imbalance Model on Depression Risk Estimation in Humans: Findings from Henan Province of China

Objective To investigate the co-effect of Demand-control-support （DCS） model and Effort-reward Imbalance （ERI） model on the risk estimation of depression in humans in comparison with the effects when they are used respectively. Methods A total of 3 632 males and 1 706 females from 13 factories and companies in Henan province were recruited in this cross-sectional study. Perceived job stress was evaluated with the Job Content Questionnaire and Effort-Reward Imbalance Questionnaire （Chinese version）. Depressive symptoms were assessed by using the Center for Epidemiological Studies Depression Scale （CES-D）. Results DC （demands/job control ratio） and ERI were shown to be independently associated with depressive symptoms. The outcome of low social support and overcommitment were similar. High DC and low social support （SS）, high ERI and high overcommitment, and high DC and high ERI posed greater risks of depressive symptoms than each of them did alone. ERI model and SS model seem to be effective in estimating the risk of depressive symptoms if they are used respectively. Conclusion The DC had better performance when it was used in combination with low SS. The effect on physical demands was better than on psychological demands. The combination of DCS and ERI models could improve the risk estimate of depressive symptoms in humans.

Brain areas activated by uncertain reward-based decision-making in healthy volunteers

Reward-based decision-making has been found to activate several brain areas, including the ven- trolateral prefronta~ lobe, orbitofrontal cortex, anterior cingulate cortex, ventral striatum, and mesolimbic dopaminergic system. In this study, we observed brain areas activated under three de- grees of uncertainty in a reward-based decision-making task （certain, risky, and ambiguous）. The tasks were presented using a brain function audiovisual stimulation system. We conducted brain scans of 15 healthy volunteers using a 3.0T magnetic resonance scanner. We used SPM8 to ana- lyze the location and intensity of activation during the reward-based decision-making task, with re- spect to the three conditions. We found that the orbitofrontal cortex was activated in the certain reward condition, while the prefrontal cortex, precentral gyrus, occipital visual cortex, inferior parietal lobe, cerebellar posterior lobe, middle temporal gyrus, inferior temporal gyrus, limbic lobe, and midbrain were activated during the ＇risk＇ condition. The prefrontal cortex, temporal pole, inferior temporal gyrus, occipital visual cortex, and cerebellar posterior lobe were activated during am- biguous decision-making. The ventrolateral prefrontal lobe, frontal pole of the prefrontal lobe, orbi- tofrontal cortex, precentral gyrus, inferior temporal gyrus, fusiform gyrus, supramarginal gyrus, infe- rior parietal Iobule, and cerebellar posterior lobe exhibited greater activation in the ＇risk＇ than in the ＇certain＇ condition （P 〈 0.05）. The frontal pole and dorsolateral region of the prefrontal lobe, as well as the cerebellar posterior lobe, showed significantly greater activation in the ＇ambiguous＇ condition compared to the ＇risk＇ condition （P 〈 0.05）. The prefrontal lobe, occipital lobe, parietal lobe, temporal lobe, limbic lobe, midbrain, and posterior lobe of the cerebellum were activated during deci- sion-making about uncertain rewards. Thus, we observed different levels and regions of activation for different types of reward processing during decision-making. Specifically, when the degree of reward uncertainty increased, the number of activated brain areas increased, including greater ac- tivation of brain areas associated with loss.

Impact of social relationship on firms' sharing reward program

In order to make strategic decision on firms’ sharing reward program( SRP), a nested Stackelberg game is developed. The sharing behavior among users and the rewarding strategy of firms are modeled. The optimal sharing bonus is worked out and the impact of social relationships among customers is discussed. The results show that the higher the bonus,the more efforts the inductor is willing to make to persuade the inductee into buying. In addition,the firms should take the social relationship into consideration when setting the optimal sharing bonus. If the social relationship is weak,there is no need to adopt the SRP. Otherwise,there are two ways to reward the inductors. Also,the stronger the social relationship,the fewer the sharing bonuses that should be offered to the inductors,and the higher the expected profits. As a result,it is reasonable for the firms to implement SRPs on the social media where users are familiar with each other.

Choice of discount rate in reinforcement learning with long-delay rewards

In the world, most of the successes are results of longterm efforts. The reward of success is extremely high, but before that, a long-term investment process is required. People who are “myopic” only value short-term rewards and are unwilling to make early-stage investments, so they hardly get the ultimate success and the corresponding high rewards. Similarly, for a reinforcement learning(RL) model with long-delay rewards, the discount rate determines the strength of agent’s “farsightedness”.In order to enable the trained agent to make a chain of correct choices and succeed finally, the feasible region of the discount rate is obtained through mathematical derivation in this paper firstly. It satisfies the “farsightedness” requirement of agent. Afterwards, in order to avoid the complicated problem of solving implicit equations in the process of choosing feasible solutions,a simple method is explored and verified by theoreti cal demonstration and mathematical experiments. Then, a series of RL experiments are designed and implemented to verify the validity of theory. Finally, the model is extended from the finite process to the infinite process. The validity of the extended model is verified by theories and experiments. The whole research not only reveals the significance of the discount rate, but also provides a theoretical basis as well as a practical method for the choice of discount rate in future researches.

Role of prefrontal cortex-nucleus accumbens glutamatergic projection in heroin reward and motivation

OBJECTIVE Glutamatergic projections from prefrontal cortex(PFc) to nucleus accumbens(NAc) regulate the dopamine(DA) release in NAc.However,it is not clear whether this circuit is effective for the reward and motivation of heroin addiction.Our study investigates the effects of metabotropic glutamate receptor 2/3(mGluR2/3) and the projections from ventromedial prefrontal cortex(vmPFc) to the NAc shell on the reward and motivation of heroin-addicted rats.METHODS First,rats were trained to selfadministration for 14 d.On the 15 thday,parts of rats were injected with mGluR 2/3 agonist LY379268(0.1,0.3 and 1.0 mg·kg-1,ip) systematically and another parts of rats were bilaterally microinjected with LY379268(0.3 and 1.0 g·L^(-1))at the volume of 0.5 μL into the ventral tegmental area(VTA),NAc core or NAc shell,respectively.All rats were followed by heroin self-administration testing under fixed ratio 1(FR1) schedule or progressed ratio(PR) schedule to observe the effect of LY379268 on the heroin reward or motivation.Second,rats were injected chemogenetic glutamatergic virus(pAAV-CaMKIIa-hM3 D(Gq)-mCherry or pAOV-CaMKIIa-hM4 D(Gi)-mCherry-3 Flag) or negative control virus in vmPFc,and trained to heroin self-administration for 14 d.On the 15 thday,rats were bilateral y microinjected with clozapine-N-oxide(CNO,1 mmol·L^(-1),0.5 μL) into NAc shell and tested the effect on the heroin reward or motivation.Finally,rats were injected optogenetical glutamatergic virus(AAV2/9-CaM KⅡ-hChR2-EYFP) or negative control virus in vmPFc,implanted 16 channel photoelectrode in ipsilateral NAc shell,and trained to heroin selfadministration for 14 d.On the 15 thday,rats were tested heroin reward under FR1 procedure with blue light stimulation in the wavelength of470 nm,frequency of 25 HZ and power of 5 mW.Each stimulation lasting for 1 h and interval for1 h.The spike changes before and after stimulation in NAc Shel neural nerve was recorded.RESULTS LY379268 cloud dose-dependent attenuated the heroin reward or motivation and the local effective site was mainly in the NAc shell.Chemogenetic results showed activation or inactivation the projection from vmPFc to NAc shell enhanced or attenuated the heroin reward and motivation,respectively.Optogenetical stimulation the same projection also enhanced the heroin reward,and a tonic neuronal firing at the nerve of NAc shell was observed during the light stimulation session.CONCLUSION mGluR2/3 activation in the NAc shell is involved in the inhibition of heroin reward and motivation.Activation the projection from PFc to NAc shell can enhance the effects on heroin reward and motivation.

Morphine attenuates frustration-like behavior induced by sucrose reward deprivation in rats

In this study, a T-maze-based frustration model in rats was established using sucrose-reward deprivation, The results revealed that rats maintained a 75% preference for the sucrose-reward arm in the reward phase. During the sucrose-deprivation frustration phase, both the preference for the sucrose-deprivation arm （62.5%） and time spent waiting in the sucrose-deprivation arm decreased. Acute injection of morphine increased the preference in a dose-dependent fashion, and prolonged the waiting duration in the sucrose-deprivation arm. These findings indicate that morphine specifically inhibited the frustration response induced by sucrose reward deprivation. To further elucidate the pharmacological mechanisms involved, the opioid receptor antagonist naloxone was given to model rats prior to the injection of morphine. The results revealed that naloxone administration markedly attenuated the anti-frustration-like effects of 3 mg/kg morphine treatment. These findings suggest that morphine attenuates the frustration-like response to reward deprivation in rats through the opioid receptor.

Time Series Analysis of Wheat Futures Reward in China

Different from the fact that the main researches are focused on single futures contract and lack of the comparison of different periods, this paper described the statistical characteristics of wheat futures reward time series of Zhengzhou Commodity Exchange in recent three years. Besides the basic statistic analysis, the paper used the GARCH and EGARCH model to describe the time series which had the ARCH effect and analyzed the persistence of volatility shocks and the leverage effect. The results showed that compared with that of normal one,wheat futures reward series were abnormality, leptokurtic and thick tail distribution. The study also found that two-part of the reward series had no autocorrelation. Among the six correlative series, three ones presented the ARCH effect. By using of the Auto-regressive Distributed Lag Model, GARCH model and EGARCH model, the paper demonstrates the persistence of volatility shocks and the leverage effect on the wheat futures reward time series. The results reveal that on the one hand, the statistical characteristics of the wheat futures reward are similar to the aboard mature futures market as a whole. But on the other hand, the results reflect some shortages such as the immatureness and the over-control by the government in the Chinese future market.

Hypothesizing “Reward” Gene Polymorphisms May Predict High Rates of Injury and Addiction in the Workforce: A Nutrient and Electrotherapeutic Based Solution

We hypothesize that individuals with genetic predisposition to Substance Use Disorder (SUD) may have greater likelihood of experiencing work related accidents. We further hypothesize that high risk populations will carry single or multiple polymorphisms associated with brain reward circuitry and/or brain reward cascade, including: Dopaminergic (i.e. DRD2 receptor genes);Serotonergic (i.e. 5-HTT2 receptor genes);Endorphinergic (i.e. pre-enkephalin genes);Gabergic (i.e. GABAA receptor genes);Neurotransmitter Metabolizing genes (i.e. MAO and COMT genes) among others (GARSRXTM). Analgesic addiction as well as “pseudoaddiction” must be treated to improve pain control and its management. We propose that non-pharmacological alternatives to pain relief, in high risk, addiction-prone individuals, are Electrotherapeutic Device(s) and Programs. We further propose patented KB220Z, a nutraceutical designed to release dopamine at the nucleus accumbens, will reduce craving behavior, in genetically programmed individuals. By utilizing both alternatives in DNA analyzed injured workers, a reduction in analgesic addiction (genuine or pseudo) leads to improved health and quicker return to work. We also hypothesize that this novel approach will impact costs related to injuries in the workforce. Effective management of chronic pain, especially in high addiction-prone workforce populations, is possible in spite of being particularly elusive. A series of factors encumber pain assessment and management, including analgesia addiction, pharmacogenomic response to pain medications, and genetically inherited factors involving gene polymorphisms. Additional research is required to test these stipulated hypotheses related to genetic proneness to addiction, but also proneness to accidents in the workplace and reduction of craving behavior. Our hypothesis that genotyping coupled with both KB220ZTM and the pharmaceutical-free Electrotherapy, will reduce iatrogenic induced analgesia addiction. This approach will achieve attainable effective pain management and quicker return to work. We propose outcomes such as the Reward Deficiency System SolutionTM may become an adjunct in the war against iatrogenic pain medication addiction.