Deep reinforcement learning using least-squares truncated temporal-difference

Policy evaluation(PE)is a critical sub-problem in reinforcement learning,which estimates the value function for a given policy and can be used for policy improvement.However,there still exist some limitations in current PE methods,such as low sample efficiency and local convergence,especially on complex tasks.In this study,a novel PE algorithm called Least-Squares Truncated Temporal-Difference learning(LST2D)is proposed.In LST2D,an adaptive truncation mechanism is designed,which effectively takes advantage of the fast convergence property of Least-Squares Temporal Difference learning and the asymptotic convergence property of Temporal Difference learning(TD).Then,two feature pre-training methods are utilised to improve the approximation ability of LST2D.Furthermore,an Actor-Critic algorithm based on LST2D and pre-trained feature representations(ACLPF)is proposed,where LST2D is integrated into the critic network to improve learning-prediction efficiency.Comprehensive simulation studies were conducted on four robotic tasks,and the corresponding results illustrate the effectiveness of LST2D.The proposed ACLPF algorithm outperformed DQN,ACER and PPO in terms of sample efficiency and stability,which demonstrated that LST2D can be applied to online learning control problems by incorporating it into the actor-critic architecture.

TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

Temporal and spatial differences in human activities performed in Urban Green Spaces of Vilnius(Lithuania)

Urban green spaces(UGS)are relevant to city well-being,as recognized by the United Nations’Sustainable Development Goals(SDGs).However,few studies have studied the temporal use of UGS.This work assessed the seasonal,weekly,and daily use of three urban green spaces(Vingis Park,Bernardino Garden,and Jomantas Park)in Vilnius(Lithuania).The study is based on an on-site observation-based survey,which recorded users’characteristics,activities,and weather conditions during summer and winter.The results showed that UGS’s seasonal,weekly,and daily use differed according to park and users’characteristics.Parks with a higher diversity of facilities had a high seasonal difference in the number of observed activities.User numbers were higher in the summer for activities with children,social activities,sports,and water activities than in the winter.Jomantas Park had the lowest variability in user characteristics.Weather variables were linked to changes in users’activities.Higher precipitation and lower temperature were associated with reducing the number of users and the diversity of registered activities.Most of the stationary activities were observed during summer.The diversity of the observed activities was associated with the available facilities rather than the park size.The distribution of stationary activities was spatially correlated with facility/equipment(benches,playgrounds,sports,and fitness equipment)and proximity to water features.The results of this study are relevant for UGS design,planning,and management.

Spatial and Temporal Differences of Urban Land Comprehensive Carrying Capacity in Guangxi

The comprehensive carrying capacity of urban land can reflect the re- source level, economic scale, social development and environmental pressure of ur- ban land carrying. The assessment indicator system of urban land comprehensive carrying capacity was constructed from the 4 aspects of resource, economy, society, environment, and principal component analysis and cluster analysis were used to evaluate the urban land comprehensive carrying capacity of Guangxi and the 14 cities in 2005-2014, and analyzed its spatial and temporal characteristics as well as the driving forces, with the aim to provide references for improving the urban land comprehensive carrying capacity. The results showed that, the overall urban land comprehensive carrying capacity in Guangxi increased in 2005-2014, and there were significant differences in the land comprehensive carrying capacities among the cities in Guangxi in 2005-2014, in which Liuzhou, Guilin, Nanning belonged to the regions with the highest carrying capacity, while Beihai, Yulin, Wutong belonged to the regions with high carrying capacity, and the carrying capacities of the other cities changed with the changes of time. The economic development degree was an important factor influencing urban land comprehensive carrying capacity, but could not directly represent the urban land comprehensive carrying capacity level.

Beyond statistical significance:Embracing minimal clinically important difference for better patient care

The minimal clinically important difference(MCID)represents a pivotal metric in bridging the gap between statistical significance and clinical relevance,addressing the direct impact of medical interventions from the patient's perspective.This comprehensive review analyzes the evolution,applications,and challenges of MCID across medical specialties,emphasizing its necessity in ensuring that clinical outcomes not only demonstrate statistical significance but also offer genuine clinical utility that aligns with patient expectations and needs.We discuss the evolution of MCID since its inception in the 1980s,its current applications across various medical specialties,and the methodologies used in its calculation,highlighting both anchor-based and distribution-based approaches.Furthermore,the paper delves into the challenges associated with the application of MCID,such as methodological variability and the interpretation difficulties that arise in clinical settings.Recommendations for the future include standardizing MCID calculation methods,enhancing patient involvement in setting MCID thresholds,and extending research to incorporate diverse global perspectives.These steps are critical to refining the role of MCID in patient-centered healthcare,addressing existing gaps in methodology and interpretation,and ensuring that medical interventions lead to significant,patient-perceived improvements.

Spatiotemporal variations of tenebrionid beetles(Coleoptera:Tenebrionidae)in the Gobi desert,Northwest China

Tenebrionid beetles represent a crucial arthropod taxon in the Gobi desert ecosystems owing to their species richness and high biomass,both of which are essential for maintaining ecosystem health and stability.However,the spatiotemporal variations of tenebrionid beetle assemblages in the Gobi desert remain poorly understood.In this study,the monthly dynamics of tenebrionid beetles in the central part of the Hexi Corridor,Northwest China,a representative area of the Gobi desert ecosystems,were monitored using pitfall trapping during 2015-2020.The following results were showed:(1)monthly activity of tenebrionid beetles was observed from March to October,with monthly activity peaking in spring and summer,and monthly activity periods and peak of tenebrionid beetle species exhibited interspecific differences that varied from year to year;(2)spatial distribution of tenebrionid beetle community was influenced by structural factors.Specifically,at a spatial scale of 24.00 m,tenebrionid beetle community was strongly and positively correlated with the dominant species,with distinct spatial distribution patterns observed for Blaps gobiensis and Microdera kraatzi alashanica;(3)abundance of tenebrionid beetles was positively correlated with monthly mean precipitation and monthly mean temperature,whereas monthly abundance of B.gobiensis and M.kraatzi alashanica was positively correlated with monthly mean precipitation;and(4)the cover of Reaumuria soongarica(Pall.)Maxim.and Nitraria sphaerocarpa Maxim.had a positive influence on the number of tenebrionid beetles captured.In conclusion,monthly variation in precipitation significantly influences the community dynamic of tenebrionid beetles,with precipitation and shrub cover jointly determining the spatial distribution pattern of these beetles in the Gobi desert ecosystems.

Novel genes involved in vascular dysfunction of the middle temporal gyrus in Alzheimer's disease:transcriptomics combined with machine learning analysis

Studies have shown that vascular dysfunction is closely related to the pathogenesis of Alzheimer's disease.The middle temporal gyrus region of the brain is susceptible to pronounced impairment in Alzheimer's disease.Identification of the molecules involved in vascular aberrance of the middle temporal gyrus would support elucidation of the mechanisms underlying Alzheimer's disease and discove ry of novel targets for intervention.We carried out single-cell transcriptomic analysis of the middle temporal gyrus in the brains of patients with Alzheimer's disease and healthy controls,revealing obvious changes in vascular function.CellChat analysis of intercellular communication in the middle temporal gyrus showed that the number of cell interactions in this region was decreased in Alzheimer's disease patients,with altered intercellular communication of endothelial cells and pericytes being the most prominent.Differentially expressed genes were also identified.Using the CellChat results,AUCell evaluation of the pathway activity of specific cells showed that the obvious changes in vascular function in the middle temporal gyrus in Alzheimer's disease were directly related to changes in the vascular endothelial growth factor(VEGF)A-VEGF receptor(VEGFR)2 pathway.AUCell analysis identified subtypes of endothelial cells and pericytes directly related to VEGFA-VEGFR2 pathway activity.Two subtypes of middle temporal gyrus cells showed significant alteration in AD:endothelial cells with high expression of Erb-B2 receptor tyrosine kinase 4(ERBB4^(high))and pericytes with high expression of angiopoietin-like 4(ANGPTL4^(high)).Finally,combining bulk RNA sequencing data and two machine learning algorithms(least absolute shrinkage and selection operator and random forest),four characteristic Alzheimer's disease feature genes were identified:somatostatin(SST),protein tyrosine phosphatase non-receptor type 3(PTPN3),glutinase(GL3),and tropomyosin 3(PTM3).These genes were downregulated in the middle temporal gyrus of patients with Alzheimer's disease and may be used to target the VEGF pathway.Alzheimer's disease mouse models demonstrated consistent altered expression of these genes in the middle temporal gyrus.In conclusion,this study detected changes in intercellular communication between endothelial cells and pericytes in the middle temporal gyrus and identified four novel feature genes related to middle temporal gyrus and vascular functioning in patients with Alzheimer's disease.These findings contribute to a deeper understanding of the molecular mechanisms underlying Alzheimer's disease and present novel treatment targets.

Efficient Temporal Difference Learning with Adaptive λ

Aim To find a more efficient learning method based on temporal difference learning for delayed reinforcement learning tasks. Methods A kind of Q learning algorithm based on truncated TD( λ ) with adaptive schemes of λ value selection addressed to absorbing Markov decision processes was presented and implemented on computers. Results and Conclusion Simulations on the shortest path searching problems show that using adaptive λ in the Q learning based on TTD( λ ) can speed up its convergence.

Water Vapor Motion Signal Extraction from FY-2E Longwave Infrared Window Images for Cloud-free Regions:The Temporal Difference Technique

The aim of this study is to calculate the low-level atmospheric motion vectors （AMVs） in clear areas with FY-2E IR2 window （11.59-12.79 μm） channel imagery,where the traditional cloud motion wind technique fails.A new tracer selection procedure,which we call the temporal difference technique,is demonstrated in this paper.This technique makes it possible to infer low-level wind by tracking features in the moisture pattern that appear as brightness temperature （TB） differences between consecutive sequences of 30-min-interval FY-2E IR2 images over cloud-free regions.The TB difference corresponding to a 10％ change in water vapor density is computed with the Moderate Resolution Atmospheric Transmission （MODTRAN4） radiative transfer model.The total contribution from each of the 10 layers is analyzed under four typical atmospheric conditions：tropical,midlatitude summer,U.S.standard,and midlatitude winter.The peak level of the water vapor weighting function for the four typical atmospheres is assigned as a specific height to the TB ＂wind＂.This technique is valid over cloudfree ocean areas.The proposed algorithm exhibits encouraging statistical results in terms of vector difference （VD）,speed bias （BIAS）,mean vector difference （MVD）,standard deviation （SD）,and root-mean-square error （RMSE）,when compared with the wind field of NCEP reanalysis data and rawinsonde observations.

Spatial-temporal differences in in-stream flow requirement based on GIS： A case study of Yan＇an region, northern Shaanxi

Although water has the central function of the bloodstream in the biosphere especially in arid or semi-arid regions such as Yah＇an region in northwestern China, yet the very limited attention is paid to the role of the water-related processes in ecosystem. In this research, based on continuous nearly 50-year data including runoff volume, sediment discharge as well as sediment accretion from hydrographic stations, and 10-year information of water quality from pollution monitoring stations, the method for measuring in-stream flow requirement has been put forward supported by experiential models and GIS spatial analysis. Additionally, the changes of in-stream flow requirement for environment and economic development have been addressed from spatial-temporal dimensions. The results show that： （1） According to the central streams in Yan＇an region, mean annual in-stream flow requirement reaches 1.0619 billion m^3, and the surface water for economic exploitation is 0.2445 billion m3 （2） Mean annual in-stream flow requirement for sediment transfers in flood period occupies over 80% of the integrated volume in a year. （3） From the 1950s to 1970s, in-stream flow requirement for sediment transfers is comparatively higher, while from the 1980s to 1990s, this requirement presents a decreasing tendency.

Temporal and spatial differences and imbalance of China's urbanization development during 1950-2006

A better understanding of the regional disparity and imbalance characteristics of China's urbanization development is the important premise for constituting correct policy and strategy and promoting the healthy and sustainable development of urbanization in the 21st century. The regional differences of China's urbanization level have close relations with natural conditions of landform and climate etc.,the urbanization level reduces with the eleva-tion of topography and decrease of precipitation. According to the statistical data set of ur-banization in 1950-2006,the temporal change course of inter-provincial disparity of Chinese urbanization level since the founding of New China in 1949 was studied,and then the inter-regional and intra-regional disparities of urbanization development were analyzed by the Theil index and its nested decomposition method,to grasp the dynamic change of spatial disparities of China's urbanization level on the whole. Using the imbalance index model,the imbalance status of urban population distribution relative to total population,grain output,total agricultural output value,gross output value of industry,tertiary industrial output value as well as gross regional product was discussed,to hold the balance characteristics of urbanization development relative to the regional development conditions from the macroscopic scales.

Temporal-spatial change and driving mechanism for regional difference of domestic tourism in China

By using the variation coefficient,Gini coefficient and Theil coefficient,this paper makes preliminary exploration of temporalspatial change features and driving mechanism for regional differences of domestic tourism in China from 1995 to 2009.According to the results,we drew the following conclusions:(1)The regional difference of domestic tourism in China tends to be narrowing generally,and is less than that of inbound tourism,playing an important role in narrowing the overall gap of regional tourism in China.(2)The regional internal difference features:inter-provincial difference in the eastern region and difference among the eastern,central and western regions are comparatively obvious and demonstrate a significantly shrinking tendency,the change tendency is consistent with the change tendency of overall difference and exercises a decisive role in overall difference,whereas the inter-provincial difference in the central and western regions is small and relatively stable,having less influence on the overall difference.(3)The temporal-spatial change in regional difference is featured by:low-level provinces occupy a majority and are concentrated in the central and western regions,showing a tendency of narrowing difference;high-level provinces are concentrated in the eastern region,showing a tendency of spreading to the central and western regions;spatial pattern of regional difference demonstrates 4 types,i.e."proliferation type","polarization type","quiescence type"and"collapse type."(4)The driving force for regional difference of domestic tourism in China comes down to 3 aspects:regional socio-economic attribute,regional transport accessibility level and regional development policy.

Spatial-temporal Pattern and Differences of Land Use Changes in the Three Gorges Reservoir Area of China during 1975-2005

Regional land use changes are an important part of global changes.The research on land use changes in the Three Gorges Reservoir Area of China attracts a lot of attention owing to the Three Gorges Dam building.The Three Gorges Reservoir Area becomes one of the important research areas.This study analyzed the transforming processes and traits of each land use type and the regional differences of land use changes during the past 30 years,summarized the distribution of different land use types in different buffer zones and regresses the equation areas and different buffer distances based on buffer analyses and regression analyses,and then analyzed the transforming rules in different buffer distances,got the optimal influence distances.The research results indicate that,(1) cultivated land lies at the northwest of the reservoir and was decreasing,however,the construction land was increasing,especially the urban construction land,a large number of land was flooded because of the reservoir water level rise;(2) urban area was sprawling quickly in developed and neighboring areas,and a great deal of cultivated land and a considerable amount of grassland were occupied;in the earlier time,rural settlements occupied lots of cultivated land and a sum of forestry land in the later time;(3) the optimum influenced distances for cultivated land and forestry land were 10-35 km,and for urban and rural settlements were in 5-20 km.Overall,this research can reflect the spatial-temporal characteristics of land use changes during the 30 years,and it is helpful for urban planning and land use planning in the reservoir area.

Intermonthly Difference of Multi-attribute Characteristics of Rainstorm in China Based on Spatial-temporal Differentiation

Based on the daily precipitation data of 545 meteorological stations in China from 1961 to 2016, the spatial-temporal variation characteristics of rainstorm rainfall and rainy days in different months of China were diagnosed from three aspects: climatic characteristics, variation trend and interannual variation. The results showed that:(1) Rainstorm rainfall and rainy days in different months of China from 1961 to 2016 had similar spatial characteristics in corresponding months. From January to July, the high-value areas of rainstorm rainfall and rainy days gradually expanded from southeast coast to northwest inland, but mainly distributed in the east area of Hu Huanyong Line. From August to December, it shrank from northwest to southeast coastal areas. Rainstorm rainfall and rainy days were less distributed in different months in the west area of Hu Huanyong Line;(2) From 1961 to 2016, the spatial-temporal variation characteristics of rainstorm rainfall in different months in China were basically consistent with that of rainy days. May to August was the most significant month for the variation trend of rainstorm rainfall and rainy days in China. It mainly distributed in the southeast monsoon area, and was mainly increasing trend. The trend of rainstorm rainfall and rainy days in northwest China changed slightly in different months;(3) The interannual variability of rainstorm rainfall in different months in China from 1961 to 2016 was similar to that of rainy days. The fluctuation characteristics from April to October were larger in the northern region. The southern region fluctuated greatly from November to December in January to March. With the development of the month, the high-value areas with large daily fluctuations of rainstorm rainfall and rainy days gradually expanded from southeast to northwest, northeast and southwest, and the fluctuations in southeast tended to decrease, then shrank from northwest, northeast and southwest to southeast, with the increasing fluctuations in southeast. The study has certain reference significance for flood control and disaster reduction and water resources planning and utilization.

TEMPORAL AND SPATIAL DIFFERENCES OF THE INDUSTRIAL RESTRUCTURING OF GUANGDONG PROVINCE IN THE 1990S

This paper focuses on the temporal and spatial change of industrial structure in Guangdong Province since the 1990s. The attention is initially paid to the change of industrial structure through analyzing the structure change of the three sectors, and the change of internal structures of agriculture, industry and tertiary sector respectively. Then the spatial differences of industrial restructuring are analyzed based on the four economic regions which have been agreed in the academic field: the Zhujiang(Pearl) River Delta, northern mountain region, east and west wings. Thirdly, the reasons for the spatial differences of industrial restructuring are examined. Finally a short conclusion is drawn and some suggestions for the regional integration are put forward.

Adaptive Modeling and Forecasting of Time Series by Combining the Methods of Temporal Differences with Neural Networks

This paper discusses the modeling method of time series with neural network. In order to improve the adaptability of direct multi-step prediction models, this paper proposes a method of combining the temporal differences methods with back-propagation algorithm for updating the parameters continuously on the basis of recent data. This method can make the neural network model fit the recent characteristic of the time series as close as possible, therefore improves the prediction accuracy. We built models and made predictions for the sunspot series. The prediction results of adaptive modeling method are better than that of non-adaptive modeling methods.

Influence of Sex Differences on Temporal Sequence of Sensations after Ingesting Fruit-Flavored Tea—A Preliminary Study

The goal of this preliminary study was to examine the effect of participants’ sexes on the temporal sequence of dominant sensations elicited by fruit-flavored teas. Twelve healthy young adults were assigned to male (n = 6) and female (n = 6) groups. Both groups were evaluated for four sensations using four fruit-flavored teas with 0.05 M sucrose using the temporal dominance of sensations (TDS) method. Only two sensations, sweetness and fruitiness, were consistently reported in the two groups. The male group first reported fruitiness and then sweetness as the dominant sensation following ingestion of the samples. Conversely, the female group reported these two dominant sensations in the reverse order. Significant dominant durations between the two sensations largely varied among the samples in both groups. These results suggest that there are sex-based differences in the temporal sequence of dominant sensations elicited by fruit-flavored teas as evaluated by the TDS method.

Exploration on Temporal-spatial Difference of Cultivated Land Pressure and Influence Factor Analysis: A Case Study of Guangxi

Based on related statistical data during 1980-2014,change rule of Guangxi cultivated land pressure level was studied. Taking each municipal administrative division as evaluation unit,temporal-spatial change trend of cultivated land pressure level was explored by establishing pressure index model of cultivated land,and principal component analysis was used to explore the driving force of cultivated land pressure. Results showed that from 1980 to 2014 in Guangxi,cultivated land pressure was at level one in 12 years,level two in 19 years and level three in 4 years; mean of cultivated land pressure in each city during 2005-2014 was taken as average level of cultivated land pressure in the city,in which cultivated land pressure values of Chongzuo City,Baise City,Laibin City,Liuzhou City,Fangchenggang City,Nanning City,Hechi City and Guigang City were all lower than average level in Guangxi at the same period. Driving factors of cultivated land pressure index mainly contained urbanization rate,Engel coefficient of rural households(ECRH),per capita cultivated land area,total population and rural per capita net income(RPFI).

Action Recognition Using Multi-Scale Temporal Shift Module and Temporal Feature Difference Extraction Based on 2D CNN

Convolutional neural networks, which have achieved outstanding performance in image recognition, have been extensively applied to action recognition. The mainstream approaches to video understanding can be categorized into two-dimensional and three-dimensional convolutional neural networks. Although three-dimensional convolutional filters can learn the temporal correlation between different frames by extracting the features of multiple frames simultaneously, it results in an explosive number of parameters and calculation cost. Methods based on two-dimensional convolutional neural networks use fewer parameters;they often incorporate optical flow to compensate for their inability to learn temporal relationships. However, calculating the corresponding optical flow results in additional calculation cost;further, it necessitates the use of another model to learn the features of optical flow. We proposed an action recognition framework based on the two-dimensional convolutional neural network;therefore, it was necessary to resolve the lack of temporal relationships. To expand the temporal receptive field, we proposed a multi-scale temporal shift module, which was then combined with a temporal feature difference extraction module to extract the difference between the features of different frames. Finally, the model was compressed to make it more compact. We evaluated our method on two major action recognition benchmarks: the HMDB51 and UCF-101 datasets. Before compression, the proposed method achieved an accuracy of 72.83% on the HMDB51 dataset and 96.25% on the UCF-101 dataset. Following compression, the accuracy was still impressive, at 95.57% and 72.19% on each dataset. The final model was more compact than most related works.

DIFFERENCES IN INTERICTAL PERFUSION PATTERNS BETWEEN REFRACTORY AND NON-REFRACTORY TEMPORAL LOBE EPILEPSIES EVALUATED WITH FAIR AT 3.0T

Objective To investigate the difference in interictal perfusion patterns between refractory and non-refractory temporal lobe epilepsies evaluated with flow-sensitive alternating inversion recovery (FAIR) magnetic resonance (MR) sequence. Methods Nine patients with refractory temporal lobe epilepsy, 21 patients with non-refractory temporal lobe epilepsy, and 13 normal volunteers underwent FAIR MR scanning. The relative cerebral blood flow (rCBF) in bilateral hemispheres and mesial temporal lobes were measured. And we also calculated the asymmetry index (AI) values. Results The AI values of bilateral hemispheres in refractory and non-refractory epilepsy patients were both significantly different from those of volunteers (P=0.012 and 0.029, respectively). There was significant difference in AI values of bilateral mesial temporal lobes between non-refractory epilepsy patients and volunteers (P=0.049), while no significant difference between refractory epilepsy patients and volunteers. Conclusions The hypoperfusion pattern of interictal refractory temporal lobe epilepsy patients is different from that of non-refractory patients. Although the hypoperfusion tends to extend out of temporal lobes in all patients, the refractory epilepsy patients have a preference of bilateral mesial temporal hypoperfusion, which may be valuable for evaluating prognosis.