Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Progress of International Meridian Circle Program

In 2024,the Chinese Meridian Project(CMP)completed its construction,deploying 282 instruments across 31 stations.This achievement not only provides a robust foundation but also serves as a reference template for the International Meridian Circle Program(IMCP).The IMCP aims to integrate and establish a comprehensive network of ground-based monitoring stations designed to track the propagation of space weather events from the Sun to Earth.Additionally,it monitors various disturbances generated within the Earth system that impact geospace.Over the past two years,significant progress has been made on the IMCP.In particular,the second phase of construction for the China-Brazil Joint Laboratory for Space Weather has been completed,and the North Pole and Southeast Asia networks are under active construction.The 2024 IMCP joint observation campaign was successfully conducted.To facilitate these developments,the scientific program committee of IMCP was established,following the success of 2023 IMCP workshop and the space weather school,which was co-hosted with the Asia-Pacific Space Cooperation Organization(APSCO)and sponsored by Chinese Academy of Sciences(CAS)and Scientific Committee on Solar-Terrestrial Physics(SCOSTEP).Preparations are now underway for the 2024 workshop in collaboration with the National Institute for Space Research(INPE)in Brazil.

An Approach to Detect Structural Development Defects in Object-Oriented Programs

Structural development defects essentially refer to code structure that violates object-oriented design principles. They make program maintenance challenging and deteriorate software quality over time. Various detection approaches, ranging from traditional heuristic algorithms to machine learning methods, are used to identify these defects. Ensemble learning methods have strengthened the detection of these defects. However, existing approaches do not simultaneously exploit the capabilities of extracting relevant features from pre-trained models and the performance of neural networks for the classification task. Therefore, our goal has been to design a model that combines a pre-trained model to extract relevant features from code excerpts through transfer learning and a bagging method with a base estimator, a dense neural network, for defect classification. To achieve this, we composed multiple samples of the same size with replacements from the imbalanced dataset MLCQ1. For all the samples, we used the CodeT5-small variant to extract features and trained a bagging method with the neural network Roberta Classification Head to classify defects based on these features. We then compared this model to RandomForest, one of the ensemble methods that yields good results. Our experiments showed that the number of base estimators to use for bagging depends on the defect to be detected. Next, we observed that it was not necessary to use a data balancing technique with our model when the imbalance rate was 23%. Finally, for blob detection, RandomForest had a median MCC value of 0.36 compared to 0.12 for our method. However, our method was predominant in Long Method detection with a median MCC value of 0.53 compared to 0.42 for RandomForest. These results suggest that the performance of ensemble methods in detecting structural development defects is dependent on specific defects.

Design of Aided Decision-Making Program for Prioritizing Construction Projects in Urban Road Network Planning

The importance and complexity of prioritizing construction projects (PCP) in urban road network planning lead to the necessity to develop an aided decision making program (ADMP). Cost benefit ratio model and stage rolled method are chosen as the theoretical foundations of the program, and then benefit model is improved to accord with the actuality of urban traffic in China. Consequently, program flows, module functions and data structures are designed, and particularly an original data structure of road ...

ARTIFICIAL NEURAL NETWORKS-MODELING, PROGRAMMING AND APPLICATION IN MATERIAL HOT WORKING

The developments of modern mathematics and computer science make artificial neural networks become most useful tools in wide range of fields. Modeling methods of artificial neural networks are described in this paper. The programming technique by using Matlab neural networks toolbox is discussed. The application in Material Hot Working of neural networks is also introduced.

An Exact Virtual Network Embedding Algorithm Based on Integer Linear Programming for Virtual Network Request with Location Constraint

Network virtualization is known as a promising technology to tackle the ossification of current Internet and will play an important role in the future network area. Virtual network embedding(VNE) is a key issue in network virtualization. VNE is NP-hard and former VNE algorithms are mostly heuristic in the literature.VNE exact algorithms have been developed in recent years. However, the constraints of exact VNE are only node capacity and link bandwidth.Based on these, this paper presents an exact VNE algorithm, ILP-LC, which is based on Integer Linear Programming(ILP), for embedding virtual network request with location constraints. This novel algorithm is aiming at mapping virtual network request(VNR) successfully as many as possible and consuming less substrate resources.The topology of each VNR is randomly generated by Waxman model. Simulation results show that the proposed ILP-LC algorithm outperforms the typical heuristic algorithms in terms of the VNR acceptance ratio, at least 15%.

Fuzzy neural network control of underwater vehicles based on desired state programming

Due to the nonlinearity and uncertainty, the precise control of underwater vehicles in some intelligent operations hasn’t been solved very well yet. A novel method of control based on desired state programming was presented, which used the technique of fuzzy neural network. The structure of fuzzy neural network was constructed according to the moving characters and the back propagation algorithm was deduced. Simulation experiments were conducted on general detection remotely operated vehicle. The results show that there is a great improvement in response and precision over traditional control, and good robustness to the model’s uncertainty and external disturbance, which has theoretical and practical value.

A RECOGNITION PROBLEM IN CONVERTING LINEAR PROGRAMMING TO NETWORK FLOW MODELS

The main goal of this paper is to study the following combinatorial problem : given a finite set E = (e1, e2, ...,em} and a subset family a - [S1,S2, ... ,Sk} of E , does there exist a tree T with the edge set E such that each induced subgraph T[Si] of Si is precisely a path (1≤i≤k) ?

Using TCP/IP Programming to Realize the Communication among Different Network Platforms

This paper mainly introduces about how to use socket to design network program which can transfer data in different network platform in terms of TCP/IP Network Programming Standard.

Incremental Network Programming for Wireless Sensors

We present an incremental network programming mechanism which reprograms wireless sensors quickly by transmitting the incremental changes using the Rsync algorithm;we generate the difference of the two program images allowing us to distribute only the key changes. Unlike previous approaches, our design does not assume any prior knowledge of the program code structure and can be applied to any hardware platform. To meet the resource constraints of wireless sensors, we tuned the Rsync algorithm which was originally made for updating binary files among powerful host machines. The sensor node processes the delivery and the decoding of the difference script separately making it easy to extend for multi-hop network programming. We are able to get a speed-up of 9.1 for changing a constant and 2.1 to 2.5 for changing a few lines in the source code.

Two-stage robust optimization of power cost minimization problem in gunbarrel natural gas networks by approximate dynamic programming

In short-term operation of natural gas network,the impact of demand uncertainty is not negligible.To address this issue we propose a two-stage robust model for power cost minimization problem in gunbarrel natural gas networks.The demands between pipelines and compressor stations are uncertain with a budget parameter,since it is unlikely that all the uncertain demands reach the maximal deviation simultaneously.During solving the two-stage robust model we encounter a bilevel problem which is challenging to solve.We formulate it as a multi-dimensional dynamic programming problem and propose approximate dynamic programming methods to accelerate the calculation.Numerical results based on real network in China show that we obtain a speed gain of 7 times faster in average without compromising optimality compared with original dynamic programming algorithm.Numerical results also verify the advantage of robust model compared with deterministic model when facing uncertainties.These findings offer short-term operation methods for gunbarrel natural gas network management to handle with uncertainties.

A dynamic programming algorithm for network selection in 3G/WLAN

An essential characteristic of the 4th Generation(4G) wireless networks is integrating various heterogeneous wireless access networks.This paper considers the network selection for both admission and handoff strategy problems in heterogeneous network of 3G/WLAN.A novel dynamic programming algorithm is proposed by taking heterogeneous network characteristics,user mobility and different service types into account.The specificity of our approach is that it puts the situations in a new model and makes decisions in stages of different states.Simulation results validate that the proposed scheme can obtain better new call blocking and handoff dropping probability performance than traditional schemes while ensuring quality-of-services(QoS) for both real-time and data connections.

Simultaneous synthesis of heat-integrated water networks by a nonlinear program:Considering the wastewater regeneration reuse

Heat-integrated water network synthesis(HIWNS)has received considerable attention for the advantages of reducing water and energy consumptions.HIWNS is effective in water and energy sustainability.Mixed integer non-linear programming(MINLP)is usually applied in HIWNS.In this work,a novel nonlinear programming(NLP)was proposed for HIWNS by considering wastewater reuse and wastewater regeneration reuse.Integer variables are changed to non-linear equation by the methods for identifying stream roles and denoting the existence of process matches.The model is tested by examples with single and multiple regeneration unit problems.The testing results showed that the NLP is an alternative method for HIWNS with wastewater reuse and regeneration reuse.

The Construction of Linux System and Network Programming Course to Develop Engineering Ability

To meet society’s needs for undergraduate students to have engineering skills and to develop students’ability to operate Linux and engage in network software development,this paper proposes the construction of a new specialized course for network engineering major--Linux system and network programming.This paper analyzes the course’s advantages,presents the contents of this course,designs a series of teaching methods aimed at improving students’engineering ability,proposes a course assessment method that will encourage students to practice,lists the development requirements for an examination software designed for this course,and finally,presents the results of our practice in teaching this course.

Coordinated planning for flexible interconnection and energy storage system in low-voltage distribution networks to improve the accommodation capacity of photovoltaic

The increasing proportion of distributed photovoltaics(DPVs)and electric vehicle charging stations in low-voltage distribution networks(LVDNs)has resulted in challenges such as distribution transformer overloads and voltage violations.To address these problems,we propose a coordinated planning method for flexible interconnections and energy storage systems(ESSs)to improve the accommodation capacity of DPVs.First,the power-transfer characteristics of flexible interconnection and ESSs are analyzed.The equipment costs of the voltage source converters(VSCs)and ESSs are also analyzed comprehensively,considering the differences in installation and maintenance costs for different installation locations.Second,a bilevel programming model is established to minimize the annual comprehensive cost and yearly total PV curtailment capacity.Within this framework,the upper-level model optimizes the installation locations and capacities of the VSCs and ESSs,whereas the lower-level model optimizes the operating power of the VSCs and ESSs.The proposed model is solved using a non-dominated sorting genetic algorithm with an elite strategy(NSGA-II).The effectiveness of the proposed planning method is validated through an actual LVDN scenario,which demonstrates its advantages in enhancing PV accommodation capacity.In addition,the economic benefits of various planning schemes with different flexible interconnection topologies and different PV grid-connected forms are quantitatively analyzed,demonstrating the adaptability of the proposed coordinated planning method.

Homomorphic Hashing Verification for Wireless Sensor Networks Rateless Codes Over-the-Air Programming

The homomorphic hash algorithm(HHA)is introduced to help on-the-fly verify the vireless sensor network(WSN)over-the-air programming(OAP)data based on rateless codes.The receiver calculates the hash value of a group of data by homomorphic hash function,and then it compares the hash value with the receiving message digest.Because the feedback channel is deliberately removed during the distribution process,the rateless codes are often vulnerable when they face security issues such as packets contamination or attack.This method prevents contaminating or attack on rateless codes and reduces the potential risks of decoding failure.Compared with the SHA1 and MD5,HHA,which has a much shorter message digest,will deliver more data.The simulation results show that to transmit and verify the same amount of OAP data,HHA method sends 17.9% to 23.1%fewer packets than MD5 and SHA1 under different packet loss rates.

STUDIES ON PROGRAMMING FEATURES AND METHODS OF FLOATING GATE MOSFET AS ANALOG MEMORY FOR SYNAPTIC WEIGHTS IN NEURAL NETWORKS

The features of the floating gate devices as analog memory have been investigatedexperimentally.Programming properties of the devices,compatibility and endurance of program-ming,and programming methods are presented in this paper.The results illustrate that thedevice can be used to store the analog weights for the neural networks,and the method that thestored value is adjusted continuously to approach to a given analog values is a rather practicalmethod for storing weights of neural networks.

Semantic network based component organization model for program mining

Based on the definition of component ontology, an effective component classification mechanism and a facet named component relationship are proposed. Then an application domain oriented, hierarchical component organization model is established. At last a hierarchical component semantic network (HCSN) described by ontology interchange language(OIL) is presented and then its function is described. Using HCSN and cooperating with other components retrieving algorithms based on component description, other components information and their assembly or composite modes related to the key component can be found. Based on HCSN, component directory library is catalogued and a prototype system is constructed. The prototype system proves that component library organization based on this model gives guarantee to the reliability of component assembly during program mining.

Scalable Incremental Network Programming for Multihop Wireless Sensors

We present a network programming mechanism that can flexibly and quickly re-task a large multi-hop network of wireless sensor nodes. Our mechanism allows each sensor node to be incrementally reprogrammed with heterogeneous images of native program code using Rsync block comparison algorithm, point-to-point routing with the BLIP IPv6 stack, and image volume management with Deluge2. With our re-tasking method, we demonstrate an order of magnitude speed-up on small code changes over non-incremental delivery. Our mechanism also scales sub-linearly in the diameter of the network. Collectively, these advancements qualitatively change the software life cycle of the embedded networked systems.