Call for papers Journal of Control Theory and Applications Special issue on Approximate dynamic programming and reinforcement learning

Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain and nonconvex environments over time.

Application Strategy of PLC Technology in Energy-Saving Control of Tunnel Lighting

In this study,we investigated on the application of planar lightwave circuit(PLC)technology in energy-saving control of tunnel lighting.The application status of PLC in the field of energy saving followed by the necessity of energy saving in tunnel lighting was analyzed.Finally,the application of PLC in tunnel lighting energy-saving control around the three dimensions of system overall architecture design,control scheme,and program control process was investigated.The results showed that the system meets the requirements of control effect,robustness,and visual effect after trial operation,and is suitable for practical applications.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

Pollution Control Programs of Large-Scale Livestock and Poultry Farms in the Dianchi Lake Basin

With the enlarging scale and intensifying production of livestock and poultry breeding, the environment pollution becomes increasingly prominent in the Dianchi Lake Basin since 1990s. According to the survey of ＂The First National Census of Pollution Sources＂, occurrence and discharge of pollutants in large-scale livestock and poultry farms in this region were first understood. The pollution characteristics of large-scale live- stock and poultry breeding were also analyzed deeply. On this basis, the significance of pollution control programs for environment protection was investigated from aspects of pollution control policy, technology management and publicity.

A Newly Proposed Temperature Control Methods of Fuel Gas Shuttle Kiln

A program control was applied in the fuel gas shuttle kiln,and its principle and disadvantge were analyzed.An advanced set point control method,in which the change rate of temperature is the controlled variable,is also described,and the new control system makes the control precision of temperature improved.

Stochastic optimal control of cable vibration in plane by using axial support motion

A stochastic optimal control strategy for a slightly sagged cable using support motion in the cable axial direction is proposed. The nonlinear equation of cable motion in plane is derived and reduced to the equations for the first two modes of cable vibration by using the Galerkin method. The partially averaged Ito equation for controlled system energy is further derived by applying the stochastic averaging method for quasi-non-integrable Hamiltonian systems. The dynamical programming equation for the controlled system energy with a performance index is established by applying the stochastic dynamical programming principle and a stochastic optimal control law is obtained through solving the dynamical programming equation. A bilinear controller by using the direct method of Lyapunov is introduced. The comparison between the two controllers shows that the proposed stochastic optimal control strategy is superior to the bilinear control strategy in terms of higher control effectiveness and efficiency.

Off-policy integral reinforcement learning optimal tracking control for continuous-time chaotic systems

This paper estimates an off-policy integral reinforcement learning（IRL） algorithm to obtain the optimal tracking control of unknown chaotic systems. Off-policy IRL can learn the solution of the HJB equation from the system data generated by an arbitrary control. Moreover, off-policy IRL can be regarded as a direct learning method, which avoids the identification of system dynamics. In this paper, the performance index function is first given based on the system tracking error and control error. For solving the Hamilton–Jacobi–Bellman（HJB） equation, an off-policy IRL algorithm is proposed.It is proven that the iterative control makes the tracking error system asymptotically stable, and the iterative performance index function is convergent. Simulation study demonstrates the effectiveness of the developed tracking control method.

Developing of robot flexible processing system for shipbuilding profile steel

A robot flexible processing system of shipbuilding profile steel was developed. The system consists of computer integrated control and robot. An off line programming robot was used for marking and cutting of shipbuilding profile steel. In the system the deformation and position error of profile steel can be detected by precise sensors, and figure position coordinate error resulted from profile steel deformation can be compensated by modifying traveling track of robotic arm online. The practical operation results show that the system performance can meet the needs of profile steel processing.

新型微机调速器的应用

本文针对农村水电站水轮机调速器存在的问题及调速器选型进行探讨,对新型调速器原理进行了解析,通过计算机控制数字阀式调速器,在虎山电站的实际应用的良好效果,对新建电站以及老电站水轮机调速器的改造提出了具体的建议性意见。

可控停车器PLC控制系统

通过控制电路上的故障—安全技术和可控停车器结构上的故障—安全技术相结合来实现ＰＬＣ可编程控制器在可控停车器控制电路上的故障—安全技术。

Optimization of Numerical Control Program and Machining Simulation Based on VERICUT

In the machining process of large-scale complex curved surface,workers will encounter problems such as empty stroke of tool,collision interference,and overcut or undercut of the workpieces.This paper presents a method for generating the optimized tool path,compiling and checking the numerical control(NC)program.Taking the bogie frame as an example,the tool paths of all machining surface are optimized by the dynamic programming algorithm,Creo software is utilized to compile the optimized computerized numerical control(CNC)machining program,and VERICUT software is employed to simulate the machining process,optimize the amount of cutting and inspect the machining quality.The method saves the machining time,guarantees the correctness of NC program,and the overall machining efficiency is improved.The method lays a good theoretical and practical foundation for integration of the similar platform.

Improving tuberculosis case detection in underdeveloped multi-ethnic regions with high disease burden:a case study of integrated control program in China

Background:In the underdeveloped multi-ethnic regions of China,high tuberculosis(TB)burden and regional inequity in access to healthcare service increase the challenge of achieving the End TB goals.Among all the provinces,the highest TB burden is reported in Xinjiang,where ethnic minorities and older people have suffered most.However,current case-finding strategy is inadequate given the complex social determinants and suboptimal case detection rates.Thus,we developed an integrated TB control program to improve case detection and conducted a pilot in Xinjiang from 2014 to 2015.In this case study,we summarized the activities and key findings.We also shared the experiences and challenges of implementing interventions and provided recommendations to inform the TB control program in the future.Case presentation:The pilot interventions were implemented in one selected town in Yining based on local TB control programs.By applying tailor-made educational materials,outreach TB educational activities were conducted in diverse ways.In 22 Masjids,the trained imams promoted TB education to the Muslims,covering 20,440 persontimes in 88 delivered preaching sessions.In seven schools,1944 students were educated by the teachers and contributed to educating 6929 family members.In the village communities,13,073 residents participated in household education and screening.Among them,12,292 people aged under 65 years were investigated for suspicious pulmonary TB symptoms,where six TB patients were diagnosed out of 89 TB suspects;781 older people were mobilized for screening directly by chest X-ray,where 10 patients were diagnosed out of 692 participants.Supportive healthcare system,multi-sectoral cooperation and multi-channel financing mechanism were the successful experiences of implementation.The interventions were proved to be more effective than the previous performance:the number of TB suspects consulting doctors and patients detected increased by 50%and 26%,respectively.The potential challenges,implications and recommendations should been taken into account for further program improvement.Conclusions:In underdeveloped multi-ethnic regions with high TB burden,improving case detection is necessary and the interventions can be feasible and effective within a supportive system.More intensive educational and training approaches,a high index of TB suspicion and prioritization of older people in screening are recommended.To sustain and scale up the program,the impacts,cost-effectiveness,feasibility and acceptability of interventions warrant further research and evaluation in each specific context.

Fault-Tolerant Communication Network System for the Super Stored Program Control Exchange

With the increasing popularity of the computer network, more and more attention is paid to the reliability of the network. To improve the reliability of the communication network in the stored program control exchange, the author propounds a redundant method supported by a kind of the double local area network. Fault detection, hardware switching and system rebuilding are processed automatically by the administration of the software. The method can improve not only the system′s reliability but also its communication efficiency. It has been implemented in the pSOSystem environment in a super stored program control exchange.

Soil organic carbon dynamics in Xilingol grassland of northern China induced by the Beijing-Tianjin Sand Source Control Program

To mitigate impacts of sandstorms on northern China, the Chinese government launched the Beijing- Tianjin Sand Source Control Program （BTSSCP） in 2000. The associated practices （i.e., cultivation, enclosure, and aerial seeding） were expected to greatly enhance grassland carbon sequestration. However, the BTSSCP-induced soil organic carbon （SOC） dynamics remain elusive at a regional level. Using the Xilingol League in Inner Mongolia for a case study, we examined the impacts from 2000 to 2006 of the BTSSCP on SOC stocks using the IPCC carbon budget inventory method. Results indicated that over all practices SOC storage increased by 1.7%, but there were large differences between practices. SOC increased most rapidly at the rate of 0.3 Mg C.ha-1 ＂yr-1 under cultivation, but decreased signifi- cantly under aerial seeding with moderate or heavy grazing （0.3 vs.0.6 Mg C-ha-I .yr-1）. SOC increases varied slightly for grassland types, ranging from 0.10 Mg C-ha-1 .yr-a for temperate desert steppe to 0.16 Mg C.ha-l.yr-1 for temperate meadow steppe and lowland meadow. The overall economic benefits of the SOC sink were estimated to be 4.0 million CNY. Aerial seeding with no grazing was found to be the most cost-effective practice. Finally, we indicated that at least 55.5 years （shortest for cultivation） were needed for the grasslands to reach their potential carbon stocks. Our findings highlight the importance and effectiveness of BTSSCP in promoting terrestrial carbon sequestration which may help mitigate climate change, and further stress the need for more attention to the effective- ness of specific practices.

Original article Schistosomiasis in China： acute infections during 2005-2008

Background Significant progress has taken place over the past 50 years in the control of schistosomiasis japonica in China. However, the available data suggested that schistosomiasis has re-emerged shortly after the World Bank Loan Project which was conducted from 1992 to 2001. The national control program with a revised strategy to control schistosomiasis by using integrated measures has been implemented since 2005. In this study, we aimed to evaluate the effect of the national program on schistosomiasis control from 2005 to 2008.Methods A retrospective study was carried out to analyze the epidemic patterns of acute infections with Schistosoma japonicum （S. japonicum）, based on the number of acute cases annually collected from the web-based national communicable diseases reporting system from 2005 to 2008.Results A total of 564, 207, 83 and 57 acute cases infected with S. japonicum were reported nationwide in 2005, 2006, 2007 and 2008, respectively, with an average annual reduction rate of 46.35% during last four years. Six outbreaks of acute infection with S. japonicurn were reported in 2005 but none in the period of 2006 to 2008. All acute cases that were reported mainly came from the lake regions and became infected during the higher risk periods from the 27th to 43rd weeks of the year. Most of these cases are students （44.87%）, farmers （31.51 %） and fishermen （7.79%） who got the infection by water contact mainly through swimming （41.49%） and production activities （40.25%）. With time, the proportion of imported cases among all acute cases increased due to more frequent movement of people that has occurred with a more mobile population.Conclusions The national control program on schistosomiasis aliened with the revised control strategy has been effectively brought into effect. However, there is still a significant risk of infection among students, farmers and fishermen living in the lake regions. Therefore, it is important to strengthen control measures among risk populations in the high risk areas of transmission, or the lake regions.

Malaria elimination in Lao PDR:the challenges associated with population mobility

Although the Lao People’s Democratic Republic(Lao PDR)is comparatively small landlocked country with patterns of both in-and out-migration,its human migration situation has been poorly studied.This is despite all of the country’s 18 provinces sharing both official and unofficial border checkpoints with neighboring countries.Economic reforms in the last decade have seen a gradual increase in the promotion of foreign investment,and main towns and transportation networks have been expanding thus offering new opportunities for livelihoods and economic activities.In the last decade,there has also been a significant reduction of reported malaria cases in Lao PDR and while this is an important prerequisite for eliminating malaria in the country,malaria outbreaks reported in the last four years suggest that population mobility,particularly in the south,is an important factor challenging current control efforts.Bolder investment in social sector spending should be geared towards improving health service provision and utilization,ensuring equitable access to primary health care(including malaria)through efforts to achieve universal health coverage targets.This should be extended to populations that are mobile and migrants.The local government plays a critical role in supporting policy and enforcement issues related to private sector project development in the provinces.Cross-border initiatives with neighboring countries,especially in terms of data sharing,surveillance,and response,is essential.Mechanisms to engage the private sector,especially the informal private sector,needs to be explored within the context of existing regulations and laws.Existing and new interventions for outdoor transmission of malaria,especially in forest settings,for high-risk groups including short-and long-term forest workers and their families,mobile and migrant populations,as well as the military must be combined into integrated packages with innovative delivery mechanisms through social marketing approaches.This should happen at multiple points in the mobility pathway and involve the private sector rather than being fully reliant on the national malaria vertical program This article based on the review of existing literature from abstracts and full texts,includes published,peer-reviewed English language literature sourced through PubMed and grey literature sources through Google and Google Scholar.The review included also case reports,sector reports,conference proceedings,research reports,epidemiology studies,qualitative studies,and census reports in both Lao and English languages.The authors used the search terms:malaria and mobile populations,malaria control program and elimination,health system performance,malaria outbreak,Lao PDR;and included articles published until June 2015.

Operational research capacity building through the Structured Operational Research Training Initiative(SORT-IT)in China:implementation,outcomes and challenges

Background:Chinese Center for Disease Control and Prevention(China CDC)introduced the Structured Operational Research Training Initiative(SORT IT)into China to build a special capacity and equip public health professionals with an effective tool to support developing countries in strengthening their operational research.The paper aims to investigate and analyze the implementation,outcomes and challenges of the first cycle of SORT IT in China.

SIMPLE COMPUTING OF THE CUSTOMER LIFETIME VALUE:A FIXED LOCAL-OPTIMAL POLICY APPROACH

In this paper,we present a new method for finding a fixed local-optimal policy for computing the customer lifetime value.The method is developed for a class of ergodic controllable finite Markov chains.We propose an approach based on a non-converging state-value function that fluctuates（increases and decreases） between states of the dynamic process.We prove that it is possible to represent that function in a recursive format using a one-step-ahead fixed-optimal policy.Then,we provide an analytical formula for the numerical realization of the fixed local-optimal strategy.We also present a second approach based on linear programming,to solve the same problem,that implement the c-variable method for making the problem computationally tractable.At the end,we show that these two approaches are related：after a finite number of iterations our proposed approach converges to same result as the linear programming method.We also present a non-traditional approach for ergodicity verification.The validity of the proposed methods is successfully demonstrated theoretically and,by simulated credit-card marketing experiments computing the customer lifetime value for both an optimization and a game theory approach.

Automatic test system development for digital beam position monitor of HEPS and BEPCII

Purpose Hundreds of digital beam position monitor processors(DBPM)are required to be produced during the construction of projects such as High Energy Photon Source(HEPS)and the upgrade project of the Beijing Electron Positron Collider(BEPCII),which brings great challenges to the test work.In order to achieve accurate,fast,and complete mass production tests of DBPMs,an automatic test system(ATS)has been developed in this article.Methods According to the test items of DBPM,the standardized testing softwareflow is designed based on virtual instru-ment program control technology and experimental physics and industrial control system(EPICS),which realize automatic adjustment of test parameters and automatic acquisition of test result data.Results and conclusions The ATS can realize one-button testing of channel coefficients,channel linearity,attenuator linearity,beam current dependence(BCD)and sampling signal-to-noise ratio(SNR),and generate test reports.The total test time is less than 3 minutes,which is significantly more efficient compared to manual testing.More than 90 BEPCII DBPMs has been tested by this ATS in the lab.The test results proved that such a system could automatically recognize defective products and satisfy the requirements of mass testing.

Feature Selection and Feature Learning for High-dimensional Batch Reinforcement Learning: A Survey

Tremendous amount of data are being generated and saved in many complex engineering and social systems every day.It is significant and feasible to utilize the big data to make better decisions by machine learning techniques. In this paper, we focus on batch reinforcement learning(RL) algorithms for discounted Markov decision processes(MDPs) with large discrete or continuous state spaces, aiming to learn the best possible policy given a fixed amount of training data. The batch RL algorithms with handcrafted feature representations work well for low-dimensional MDPs. However, for many real-world RL tasks which often involve high-dimensional state spaces, it is difficult and even infeasible to use feature engineering methods to design features for value function approximation. To cope with high-dimensional RL problems, the desire to obtain data-driven features has led to a lot of works in incorporating feature selection and feature learning into traditional batch RL algorithms. In this paper, we provide a comprehensive survey on automatic feature selection and unsupervised feature learning for high-dimensional batch RL. Moreover, we present recent theoretical developments on applying statistical learning to establish finite-sample error bounds for batch RL algorithms based on weighted Lpnorms. Finally, we derive some future directions in the research of RL algorithms, theories and applications.