Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.

Reinforcement Technology for Soft Soil Roadbed in the Widened Section of Expressway Expansion

This article examines the soft soil roadbed reinforcement technology for widened sections of highways in a specific project.It provides an overview of the project,the principles of soft soil roadbed reinforcement technology for wide sections,and its practical application.The analysis aims to offer guidance on applying soft soil roadbed wide section reinforcement technology and enhancing the overall quality of similar projects.

Structural Design of Roadbed and Pavement in Transition Section of Roads and Bridges

As the lifeline of social development,road and bridge projects are the main channel to realize resource transportation and economic circulation.Ensuring the quality of road and bridge project construction is crucial for the development of society,the economy,and people’s livelihoods.This paper studies the design of roadbed pavement structures in road and bridge transition sections.It aims to provide technical references and significance for China’s road and bridge engineering design and construction units,promoting scientific and standardized design in these actions.This will contribute to the safety and stable operation of road and bridge projects,offering effective technical support.Furthermore,it seeks to foster the sustainable and healthy development of China’s road and bridge engineering on a macro level.

Multi-Scale Design and Optimization of Composite Material Structure for Heavy-Duty Truck Protection Device

In this paper,to present a lightweight-developed front underrun protection device(FUPD)for heavy-duty trucks,plain weave carbon fiber reinforced plastic(CFRP)is used instead of the original high-strength steel.First,the mechanical and structural properties of plain carbon fiber composite anti-collision beams are comparatively analyzed from a multi-scale perspective.For studying the design capability of carbon fiber composite materials,we investigate the effects of TC-33 carbon fiber diameter(D),fiber yarn width(W)and height(H),and fiber yarn density(N)on the front underrun protective beam of carbon fiber compositematerials.Based on the investigation,a material-structure matching strategy suitable for the front underrun protective beam of heavy-duty trucks is proposed.Next,the composite material structure is optimized by applying size optimization and stack sequence optimization methods to obtain the higher performance carbon fiber composite front underrun protection beam of commercial vehicles.The results show that the fiber yarn height(H)has the greatest influence on the protective beam,and theH1matching scheme for the front underrun protective beamwith a carbon fiber composite structure exhibits superior performance.The proposed method achieves a weight reduction of 55.21% while still meeting regulatory requirements,which demonstrates its remarkable weight reduction effect.

Shape design of the reinforcement for bending load-carrying capacity of under-matched butt joint under four-point bending load

To improve the bending load-carrying capacity （ BLCC） of under-matched butt joint under four-point bending load in the elastic stage, the shape design of the reinforcement is studied based on the theoretics of mechanics of materials. The concept, criterion, realization condition and design proposal of equal bending load-carrying capacity （EBLCC） are put forward. The theoretical analysis results have been verified by the finite element method. The simulation results are coincident basically with the ones of theoretical analysis. The research results show that the shape design of the reinforcement of EBLCC can improve BLCC of under-matched butt joint and the unilateral-side type reinforcement can replace double-side symmetry

Study on determining of reinforced concrete false roof strength and design of reinforcement based on reliability theory

Study on efficient mining of the steep incline and fractured ore-bodies in Yongshaba mine of Guizhou Kailin Group shows that ore-body is fractured and difficult to support the roadways in-vein.After research of the actual conditions about the ore-bodies,we have made the initial decision to adopt reconstruction of roof downward sublevel cut-and-fill mining.The men work safely under the false roof supporting the top plate.However,the difficult problem is how to determine the strength of the false roof.In this case,the method based on reliability theory has been put forward.Combined with elastic mechanics and field practice,when practical value of reliable probability is 90 %,the value of the false roof strength has been calculated,and the study shows that stope span greatly influences the false roof strength.With the strength of artificial roof,the reasonable reinforcement design ensures the false roof which can supply the demand of strength under large span and load.

A novel contour design technique of super/hypersonic nozzle diverging section with sharing throat

A straightforward technique has been developed to quickly determine the wall contour of super/hypersonic nozzles working at multiply Mach number which share a common throat section.Mach number distribution along the centerline of the nozzle is specified in advance and divided into two sections,both of which are described by the b-spline function.The first section is shared by different exit Mach number nozzles.The nozzle contour is determined by the method of characteristics plus boundary layer correction.An example of this design method is employed to illustrate the technique with a computational fluid dynamics calculation.The simulation results indicate that desired Mach numbers are obtained at the nozzle exit,and the good flow quality is attained for different nozzles within δMa/Ma<±0.56% in the flow core region.This technique improves the design precision of the converging-diverging nozzle,cancels waves completely,and achieves nozzles with multiple Mach number exiting which share a common throat section.

Studies on Seismic Identification and Reinforcement Design of Building Structures

China is a country with many earthquakes. Seismic safety monitoring and building earthquake-proofing technique are important means to protect the safety of people’s property in China. However, up to now, China’s seismic reinforcement and identification technology is still not mature enough. In particular, the 2008 Wenchuan earthquake caused great loss of life and safety to the Chinese people. This paper, takes seismic identification and reinforcement technology of building structures as the research object and summarizes the main methods of building structure seismic resistance in China. This paper is based on an in-depth analysis of the main seismic reinforcement and identification techniques in China, deeply analyzes the crux of anti-seismic and reinforcement of building structure combining with the current building seismic reinforcement typical cases, and puts forward some reasonable suggestions and improvement methods for the future development of building seismic identification and reinforcement design.

The Challenges of Designing a Wooden Beam of Composed Section

This work describes details about the challenges and solutions encountered in the timber structure beam. The choice for wood is due to the fact that it is a better material suited for environments with high salinity and, in this context, to reduce maintenance costs. The entire roof was supported only by four columns, which forced the side and frontal beams to exceed spans over 18 meters without intermediate supports. This was the most important challenge of the project： to find a section to the beam which can attend the ultimate limit state and serviceability in wood to the dimensions in project. The answer was clear： It would need to have composed section. A computational model was constructed using the finite element method in order to obtain the efforts on the structural elements. It is important to emphasize that the design of a wood structure requires not only knowledge, but creativity as well, to overcome the challenges posed by the distinctive needs to the detailing of this material. In addition to that, it requires special attention during the construction execution, because of the difficulty that the carpenters have to read the specific plans.

Design Strategy of Highway Speed Transition Section

In this paper,combined with the relevant speed theory and characteristics of the law,the current highway speed transition design problems are studied and analyzed.In the process of specific analysis,mainly combined with the characteristics of different types of highway speed changes and road section design requirements,this paper studies and analyzes the design methods of different types of highway speed transition section.And on this basis,according to the design principles and requirements of highway operation speed transition section,the paper summarizes the matters needing attention in the design of highway operation speed transition section,in order to provide certain reference value for relevant personnel.

Ultimate flexural strength of normal section of FRP-confined RC circular columns

Numerical analysis is carried out to study the sectional properties of the fiber-reinforced polymer（FRP）-confined reinforced concrete（RC）circular columns. The axial load ratio, the FRP confinement ratio and the longitudinal reinforcement characteristic value are the three main parameters that can influence the neutral axis depth when concrete compression strain reaches an ultimate value. The formula for computing the central angle θ, corresponding to the compression zone, is established according to the data regression of the numerical analysis results. The numerical analysis results demonstrate that the concrete stress enhancement from transverse confinement and strain hardening of the longitudinal reinforcement can cause a much greater flexural strength than that defined by the design code. Based on the analytical studies and the test results of 36 large scale columns, the formula to calculate the flexural strength when columns fail under seismic loading is proposed, and the calculated results agree well with the test results. Finally, parametric studies are conducted on a typical column with different axial load ratios, longitudinal reinforcement characteristic value and FRP confinement ratios. Analysis of the results shows that the calculated flexural strength can be increased by 50% compared to that of unconfined columns defined by the code.

Materials Design of Microstructure in Grain Boundary and Second Phase Particles

A concept of microstructure design for materials or materials microstructure engineering is proposed. The argument was suggested based on literature review and. some our new research work on second phase strengthening mechanisms and mechanical property modeling of a particulate reinforced metal matrix composite. Due to development of computer technology, it is possible now for us to establish the relationship between microstructures and properties systematically and quantitatively by analytical and numerical modeling in the research scope of computerization materials. Discussions and examples on intellectual optimization of microstructure are presented on two aspects: grain boundary engineering and optimal geometry of particulate reinforcements in two-phase materials.

Coal pillar design when considered a reinforcement problem rather than a suspension problem

Current coal pillar design is the epitome of suspension design.A defined weight of unstable overburden material is estimated, and the dimensions of the pillars left behind are based on holding up that material to a prescribed factor of safety.In principle, this is no different to early roadway roof support design.However, for the most part, roadway roof stabilisation has progressed to reinforcement, whereby the roof strata is assisted in supporting itself.This is now the mainstay of efficient and effective underground coal production.Suspension and reinforcement are fundamentally different in roadway roof stabilisation and lead to substantially different requirements in terms of support hardware characteristics and their application.In suspension, the primary focus is the total load-bearing capacity of the installed support and ensuring that it is securely anchored outside of the unstable roof mass.In contrast, reinforcement recognises that roof de-stabilisation is a gradational process with ever-increasing roof displacement magnitude leading to ever-reducing stability.Key roof support characteristics relate to such issues as system stiffness, the location and pattern of support elements and mobilising a defined thickness of the immediate roof to create(or build) a stabilising strata beam.The objective is to ensure that horizontal stress is maintained at a level that prevents mass roof collapse.This paper presents a prototype coal pillar and overburden system representation where reinforcement, rather than suspension, of the overburden is the stabilising mechanism via the action of in situ horizontal stresses.Established roadway roof reinforcement principles can potentially be applied to coal pillar design under this representation.The merit of this is evaluated according to failed pillar cases as found in a series of published databases.Based on the findings, a series of coal pillar system design considerations for bord and pillar type mine workings are provided.This potentially allows a more flexible approach to coal pillar sizing within workable mining layouts, as compared to common industry practice of a single design factor of safety(Fo S) under defined overburden dead-loading to the exclusion of other relevant overburden stabilising influences.

Energy-Efficient UAV Trajectory Design for Backscatter Communication: A Deep Reinforcement Learning Approach

Recently,backscatter communication(BC)has been introduced as a green paradigm for Internet of Things(IoT).Meanwhile,unmanned aerial vehicles(UAVs)can serve as aerial base stations(BSs)to enhance the performance of BC system thanks to their high mobility and flexibility.In this paper,we investigate the problem of energy efficiency(EE)for an energy-limited backscatter communication(BC)network,where backscatter devices(BDs)on the ground harvest energy from the wireless signal of a flying rotary-wing quadrotor.Specifically,we first reformulate the EE optimization problem as a Markov decision process(MDP)and then propose a deep reinforcement learning(DRL)algorithm to design the UAV trajectory with the constraints of the BD scheduling,the power reflection coefficients,the transmission power,and the fairness among BDs.Simulation results show the proposed DRL algorithm achieves close-to-optimal performance and significant EE gains compared to the benchmark schemes.

Instability energy mechanism of super-large section crossing chambers in deep coal mines

The stress concentration and failure at chamber intersections in coal mine are intense,especially in deepburied,super-large section conditions.In this paper,the plastic radius of super-large section chamber under unequal pressure was corrected on the basis of the size effect.Then,stress and failure evolution of intersections under different crossing angles and equivalent angular bisectors were revealed.Furthermore,2 trajectory curves of failure and stress were analytically expressed,which divided the intersection into 5 influencing zones in the light of stress superposition degree.After determining instability trigger point and instability path,instability energy criterion of intersection can be obtained as K>1,which means that the external energy is greater than the sum of energy consumed by surrounding rock instability and supporting structure failure.Taking coal-gangue separation system of Longgu Coal Mine as example,it was found that there was instability risk under original parameters.For long-term stability,an optimization design method was proposed by considering safety factor,and optimal support scheme was obtained.Field monitoring showed intersections deformations were relatively small with the maximum of 125 mm,which verified the rationality of theoretical analysis.This study provides guidance for the stability control of the intersections under the same or similar conditions.

Reinforcement of Soft Foundation with Geotextile and Observation for Sea Dike Project of Zhapu Port

The design method of reinforcement of soft foundation with geotextile for the sea dike of the Zhapu Port is discussed in this paper. The prototype behaviours such as pore water pressure, settlement and so on were observed. The degree of consolidation is found out from observed pore water pressure and observed settlement respectively, then the strength increment of soil is calculated and compared with that obtained from vane shear tests. For the use of observed pore water pressure, the consolidation coefficient of soil is deduced approximately with a method named experimental exponential interpolation. The degree of consolidation of the ground is deduced theoretically from the dissipation of pore water pressure. Besides, the logarithmic curve and hyperbola are used to fit the observed time-settlement curve, and the degree of consolidation of soil is obtained according to the definition of the consolidation degree. After preliminary verification with observed prototype data, the method to reinforce the low dike with geotextile is considered to be simple and rational, and it can also reduce the construction cost.

Effect of seismic design details on hysteresis performance of SRC-RC transfer columns

Four types of seismic design details were tested using 11 transfer column specimens and one comparison specimen of RC under low cyclic reversed loading. Test results show that diagonal cracks control the failure pattern and damage occurs mainly in the RC section with weak shear capacity in the transfer columns. There is a large difference in the bearing capacity and ductility of the transfer columns according to the test results, which indicates that the strengthening effect of diverse structural measures is quite different. The section ratio of I-section-encased steel and the axial compression ratio also have a great influence on the bearing capacity and ductility. Although the bearing capacity of transfer columns with additional longitudinal bars and additional X bars is relatively large, they have poor deformation capacity. Setting more stirrups along the columns is the best structural measure to enhance the seismic performance. The studs on the I-sectionencased steel by welding can help to complete the stress transfer between the steel and concrete, and avoid performance degradation of the two materials due to bonding failure.

Multi-Agent Few-Shot Meta Reinforcement Learning for Trajectory Design and Channel Selection in UAV-Assisted Networks

Unmanned aerial vehicle(UAV)-assisted communications have been considered as a solution of aerial networking in future wireless networks due to its low-cost, high-mobility, and swift features. This paper considers a UAV-assisted downlink transmission,where UAVs are deployed as aerial base stations to serve ground users. To maximize the average transmission rate among the ground users, this paper formulates a joint optimization problem of UAV trajectory design and channel selection, which is NP-hard and non-convex. To solve the problem, we propose a multi-agent deep Q-network(MADQN) scheme.Specifically, the agents that the UAVs act as perform actions from their observations distributively and share the same reward. To tackle the tasks where the experience is insufficient, we propose a multi-agent meta reinforcement learning algorithm to fast adapt to the new tasks. By pretraining the tasks with similar distribution, the learning model can acquire general knowledge. Simulation results have indicated the MADQN scheme can achieve higher throughput than fixed allocation. Furthermore, our proposed multiagent meta reinforcement learning algorithm learns the new tasks much faster compared with the MADQN scheme.

Deep Reinforcement Learning for Multi-Phase Microstructure Design

This paper presents a de-novo computational design method driven by deep reinforcement learning to achieve reliable predictions and optimum properties for periodic microstructures.With recent developments in 3-D printing,microstructures can have complex geometries and material phases fabricated to achieve targeted mechanical performance.These material property enhancements are promising in improving the mechanical,thermal,and dynamic performance in multiple engineering systems,ranging from energy harvesting applications to spacecraft components.The study investigates a novel and efficient computational framework that integrates deep reinforcement learning algorithms into finite element-based material simulations to quantitatively model and design 3-D printed periodic microstructures.These algorithms focus on improving the mechanical and thermal performance of engineering components by optimizing a microstructural architecture to meet different design requirements.Additionally,the machine learning solutions demonstrated equivalent results to the physics-based simulations while significantly improving the computational time efficiency.The outcomes of the project show promise to the automation of the design and manufacturing of microstructures to enable their fabrication in large quantities with the utilization of the 3-D printing technology.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.