Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Design and Construction Technology of Prefabricated Reinforced Concrete Slab Culverts

Compared with traditional cast-in-situ concrete slab culverts,prefabricated reinforced concrete slab culverts can be produced more quickly and has strong quality controllability,strong earthquake resistance,and repeatability.They will be the primary production method of slab culverts in the future.This article offers a comprehensive review of the design and construction technology associated with prefabricated reinforced concrete slab culverts.The objective is to provide a valuable reference for related enterprises,enhance the quality of design and construction in precast pile configuration,and,in turn,contribute to the advancement of construction projects within our country.

Multi-Scale Design and Optimization of Composite Material Structure for Heavy-Duty Truck Protection Device

In this paper,to present a lightweight-developed front underrun protection device(FUPD)for heavy-duty trucks,plain weave carbon fiber reinforced plastic(CFRP)is used instead of the original high-strength steel.First,the mechanical and structural properties of plain carbon fiber composite anti-collision beams are comparatively analyzed from a multi-scale perspective.For studying the design capability of carbon fiber composite materials,we investigate the effects of TC-33 carbon fiber diameter(D),fiber yarn width(W)and height(H),and fiber yarn density(N)on the front underrun protective beam of carbon fiber compositematerials.Based on the investigation,a material-structure matching strategy suitable for the front underrun protective beam of heavy-duty trucks is proposed.Next,the composite material structure is optimized by applying size optimization and stack sequence optimization methods to obtain the higher performance carbon fiber composite front underrun protection beam of commercial vehicles.The results show that the fiber yarn height(H)has the greatest influence on the protective beam,and theH1matching scheme for the front underrun protective beamwith a carbon fiber composite structure exhibits superior performance.The proposed method achieves a weight reduction of 55.21% while still meeting regulatory requirements,which demonstrates its remarkable weight reduction effect.

Improving the design of reinforcing frames by simulating the arch and peltate venation structures

Based on the analyses on arch and peltate venation structures, the design of reinforcing frames was improved. First, distribution rules of the arch structure were summarized. According to the load condition and the structure of the frame, a mechanical model of arch structure was devel- oped, and two solutions for the model were analyzed and compared with each other. Through the a- nalysis, application rules of arch structure for improving the design were obtained. Then, distribu- tion rules of peltate venation structure were summarized. By using the same method, application rules of peltate venation structure for improving the design were also obtained. Finally, mechanical problem of the frame was described, and rib arrangement of the frame was redesigned. A parameter optimization for the widths of ribs in bionic arrangement was also carried out to accomplish the im- proving design. Comparison between bionic and conventional reinforcing frames shows that the weight is reduced by as much as 15.3%.

Coal pillar design when considered a reinforcement problem rather than a suspension problem

Current coal pillar design is the epitome of suspension design.A defined weight of unstable overburden material is estimated, and the dimensions of the pillars left behind are based on holding up that material to a prescribed factor of safety.In principle, this is no different to early roadway roof support design.However, for the most part, roadway roof stabilisation has progressed to reinforcement, whereby the roof strata is assisted in supporting itself.This is now the mainstay of efficient and effective underground coal production.Suspension and reinforcement are fundamentally different in roadway roof stabilisation and lead to substantially different requirements in terms of support hardware characteristics and their application.In suspension, the primary focus is the total load-bearing capacity of the installed support and ensuring that it is securely anchored outside of the unstable roof mass.In contrast, reinforcement recognises that roof de-stabilisation is a gradational process with ever-increasing roof displacement magnitude leading to ever-reducing stability.Key roof support characteristics relate to such issues as system stiffness, the location and pattern of support elements and mobilising a defined thickness of the immediate roof to create(or build) a stabilising strata beam.The objective is to ensure that horizontal stress is maintained at a level that prevents mass roof collapse.This paper presents a prototype coal pillar and overburden system representation where reinforcement, rather than suspension, of the overburden is the stabilising mechanism via the action of in situ horizontal stresses.Established roadway roof reinforcement principles can potentially be applied to coal pillar design under this representation.The merit of this is evaluated according to failed pillar cases as found in a series of published databases.Based on the findings, a series of coal pillar system design considerations for bord and pillar type mine workings are provided.This potentially allows a more flexible approach to coal pillar sizing within workable mining layouts, as compared to common industry practice of a single design factor of safety(Fo S) under defined overburden dead-loading to the exclusion of other relevant overburden stabilising influences.

Energy-Efficient UAV Trajectory Design for Backscatter Communication: A Deep Reinforcement Learning Approach

Recently,backscatter communication(BC)has been introduced as a green paradigm for Internet of Things(IoT).Meanwhile,unmanned aerial vehicles(UAVs)can serve as aerial base stations(BSs)to enhance the performance of BC system thanks to their high mobility and flexibility.In this paper,we investigate the problem of energy efficiency(EE)for an energy-limited backscatter communication(BC)network,where backscatter devices(BDs)on the ground harvest energy from the wireless signal of a flying rotary-wing quadrotor.Specifically,we first reformulate the EE optimization problem as a Markov decision process(MDP)and then propose a deep reinforcement learning(DRL)algorithm to design the UAV trajectory with the constraints of the BD scheduling,the power reflection coefficients,the transmission power,and the fairness among BDs.Simulation results show the proposed DRL algorithm achieves close-to-optimal performance and significant EE gains compared to the benchmark schemes.

Shape design of the reinforcement for bending load-carrying capacity of under-matched butt joint under four-point bending load

To improve the bending load-carrying capacity （ BLCC） of under-matched butt joint under four-point bending load in the elastic stage, the shape design of the reinforcement is studied based on the theoretics of mechanics of materials. The concept, criterion, realization condition and design proposal of equal bending load-carrying capacity （EBLCC） are put forward. The theoretical analysis results have been verified by the finite element method. The simulation results are coincident basically with the ones of theoretical analysis. The research results show that the shape design of the reinforcement of EBLCC can improve BLCC of under-matched butt joint and the unilateral-side type reinforcement can replace double-side symmetry

Multi-Agent Few-Shot Meta Reinforcement Learning for Trajectory Design and Channel Selection in UAV-Assisted Networks

Unmanned aerial vehicle(UAV)-assisted communications have been considered as a solution of aerial networking in future wireless networks due to its low-cost, high-mobility, and swift features. This paper considers a UAV-assisted downlink transmission,where UAVs are deployed as aerial base stations to serve ground users. To maximize the average transmission rate among the ground users, this paper formulates a joint optimization problem of UAV trajectory design and channel selection, which is NP-hard and non-convex. To solve the problem, we propose a multi-agent deep Q-network(MADQN) scheme.Specifically, the agents that the UAVs act as perform actions from their observations distributively and share the same reward. To tackle the tasks where the experience is insufficient, we propose a multi-agent meta reinforcement learning algorithm to fast adapt to the new tasks. By pretraining the tasks with similar distribution, the learning model can acquire general knowledge. Simulation results have indicated the MADQN scheme can achieve higher throughput than fixed allocation. Furthermore, our proposed multiagent meta reinforcement learning algorithm learns the new tasks much faster compared with the MADQN scheme.

Deep Reinforcement Learning for Multi-Phase Microstructure Design

This paper presents a de-novo computational design method driven by deep reinforcement learning to achieve reliable predictions and optimum properties for periodic microstructures.With recent developments in 3-D printing,microstructures can have complex geometries and material phases fabricated to achieve targeted mechanical performance.These material property enhancements are promising in improving the mechanical,thermal,and dynamic performance in multiple engineering systems,ranging from energy harvesting applications to spacecraft components.The study investigates a novel and efficient computational framework that integrates deep reinforcement learning algorithms into finite element-based material simulations to quantitatively model and design 3-D printed periodic microstructures.These algorithms focus on improving the mechanical and thermal performance of engineering components by optimizing a microstructural architecture to meet different design requirements.Additionally,the machine learning solutions demonstrated equivalent results to the physics-based simulations while significantly improving the computational time efficiency.The outcomes of the project show promise to the automation of the design and manufacturing of microstructures to enable their fabrication in large quantities with the utilization of the 3-D printing technology.

Optimal seismic design of reinforced concrete structures under timehistory earthquake loads using an intelligent hybrid algorithm

A reliable seismic-resistant design of structures is achieved in accordance with the seismic design codes by designing structures under seven or more pairs of earthquake records. Based on the recommendations of seismic design codes, the average time-history responses （ATHR） of structure is required. This paper focuses on the optimal seismic design of reinforced concrete （RC） structures against ten earthquake records using a hybrid of particle swarm optimization algorithm and an intelligent regression model （IRM）. In order to reduce the computational time of optimization procedure due to the computational efforts of time-history analyses, IRM is proposed to accurately predict ATHR of structures. The proposed IRM consists of the combination of the subtractive algorithm （SA）, K-means clustering approach and wavelet weighted least squares support vector machine （WWLS-SVM）. To predict ATHR of structures, first, the input-output samples of structures are classified by SA and K-means clustering approach. Then, WWLS-SVM is trained with few samples and high accuracy for each cluster. 9- and 18-storey RC frames are designed optimally to illustrate the effectiveness and practicality of the proposed IRM. The numerical results demonstrate the efficiency and computational advantages of IRM for optimal design of structures subjected to time-history earthquake loads.

Finite Element Method for Design of Reinforced Concrete Offshore Platforms

A design method of reinforced concrete (R. C.) offshore platforms with nonlinear finite element analysis is proposed. According to the method, a computer program is developed. In this program nonlinear constitutive relationships and strength criteria of concrete and steel bars are included, and the progressive cracking and crushing of the concrete are taken into account. Based on the stress distribution obtained by the nonlinear finite element analysis, the amount of reinforcement in the control sections can be computed and adjusted automatically by the program to satisfy the requirement of the design. The amount of reinforcement required in the control sections, which are obtained with the nonlinear finite element analysis, is agreeable to that obtained in the experiment. This shows that the design method of R. C. offshore platform with the nonlinear finite element method proposed by the authors is reliable for practical use.

Study on determining of reinforced concrete false roof strength and design of reinforcement based on reliability theory

Study on efficient mining of the steep incline and fractured ore-bodies in Yongshaba mine of Guizhou Kailin Group shows that ore-body is fractured and difficult to support the roadways in-vein.After research of the actual conditions about the ore-bodies,we have made the initial decision to adopt reconstruction of roof downward sublevel cut-and-fill mining.The men work safely under the false roof supporting the top plate.However,the difficult problem is how to determine the strength of the false roof.In this case,the method based on reliability theory has been put forward.Combined with elastic mechanics and field practice,when practical value of reliable probability is 90 %,the value of the false roof strength has been calculated,and the study shows that stope span greatly influences the false roof strength.With the strength of artificial roof,the reasonable reinforcement design ensures the false roof which can supply the demand of strength under large span and load.

Trajectory Design for UAV-Enabled Maritime Secure Communications:A Reinforcement Learning Approach

This paper investigates an unmanned aerial vehicle(UAV)-enabled maritime secure communication network,where the UAV aims to provide the communication service to a legitimate mobile vessel in the presence of multiple eavesdroppers.In this maritime communication networks(MCNs),it is challenging for the UAV to determine its trajectory on the ocean,since it cannot land or replenish energy on the sea surface,the trajectory should be pre-designed before the UAV takes off.Furthermore,the take-off location of the UAV and the sea lane of the vessel may be random,which leads to a highly dynamic environment.To address these issues,we propose two reinforcement learning schemes,Q-learning and deep deterministic policy gradient(DDPG)algorithms,to solve the discrete and continuous UAV trajectory design problem,respectively.Simulation results are provided to validate the effectiveness and superior performance of the proposed reinforcement learning schemes versus the existing schemes in the literature.Additionally,the proposed DDPG algorithm converges faster and achieves higher utilities for the UAV,compared to the Q-learning algorithm.

Studies on Seismic Identification and Reinforcement Design of Building Structures

China is a country with many earthquakes. Seismic safety monitoring and building earthquake-proofing technique are important means to protect the safety of people’s property in China. However, up to now, China’s seismic reinforcement and identification technology is still not mature enough. In particular, the 2008 Wenchuan earthquake caused great loss of life and safety to the Chinese people. This paper, takes seismic identification and reinforcement technology of building structures as the research object and summarizes the main methods of building structure seismic resistance in China. This paper is based on an in-depth analysis of the main seismic reinforcement and identification techniques in China, deeply analyzes the crux of anti-seismic and reinforcement of building structure combining with the current building seismic reinforcement typical cases, and puts forward some reasonable suggestions and improvement methods for the future development of building seismic identification and reinforcement design.

Advanced Welding Technology for Highly Stressable Multi-Material Designs with Fiber-Reinforced Plastics and Metals

Organic sheets made out of fiber-reinforced thermoplastics are able to make a crucial contribution to increase the lightweight potential of a design. They show high specific strength- and stiffness properties, good damping characteristics and recycling capabilities, while being able to show a higher energy absorption capacity than comparable metal constructions. Nowadays, multi-material designs are an established way in the automotive industry to combine the benefits of metal and fiber-reinforced plastics. Currently used technologies for the joining of organic sheets and metals in large-scale production are mechanical joining technologies and adhesive technologies. Both techniques require large overlapping areas that are not required in the design of the part. Additionally, mechanical joining is usually combined with “fiber-destroying” pre-drilling and punching processes. This will disturb the force flux at the joining location by causing unwanted fiber- and inter-fiber failure and inducing critical notch stresses. Therefore, the multi-material design with fiber-reinforced thermoplastics and metals needs optimized joining techniques that don’t interrupt the force flux, so that higher loads can be induced and the full benefit of the FRP material can be used. This article focuses on the characterization of a new joining technology, based on the Cold Metal Transfer (CMT) welding process that allows joining of organic sheets and metals in a load path optimized way, with short cycle times. This is achieved by redirecting the fibers around the joining area by the insertion of a thin metal pin. The path of the fibers will be similar to paths of fibers inside structures found in nature, e.g. a knothole inside of a tree. As a result of the bionic fiber design of the joint, high joining strengths can be achieved. The increase of the joint strength compared to blind riveting was performed and proven with stainless steel and orthotropic reinforced composites in shear-tests based on the DIN EN ISO 14273. Every specimen joined with the new CMT Pin joining technology showed a higher strength than specimens joined with one blind rivet. Specimens joined with two or three pin rows show a higher strength than specimens joined with two blind rivets.

Research on stability design of a bar-reinforced spherical bulkhead

In this paper, the stability of a concave spherical stem bulkhead under the pumping load when still lying at the slipway is analyzed. The stability of the spherical stem bulkhead with different shell thickness and reinforcing forms is discussed. According to the results of stability analysis, the optimization design of the spherical stem bulkhead stability is performed. On the basis of considering the machining technical requirements of the bulkhead, a rational design of the spherical stem bulkhead structure is obtained. This paper has a certain value to the design of submarine's spherical stem bulkhead.

纤维增强复合材料加固混凝土桥梁结构研究进展

为总结纤维增强复合材料(FRP)在混凝土桥梁加固中的研究成果,拓宽FRP加固桥梁的思路,推动其在桥梁加固领域的广泛应用,阐述了FRP加固方法的发展及其特点,系统梳理了FRP加固方法在混凝土桥梁的研究进展。根据加固机理的不同,从抗弯加固、抗剪加固、抗压加固三方面加以归纳总结,介绍了几种常见的FRP加固方式、影响因素、破坏模式及其设计理论;分析了当前研究的不足,由此提出了一些可以继续深入研究的问题。结果表明:目前研究较多的有外贴FRP法、预应力FRP技术、表层嵌入法、FRP网格加固技术,其在不同的设计参数和使用条件下呈现出不同的破坏模式;外贴FRP片材容易过早发生剥离破坏,可以通过一些锚固措施提升界面黏结性,其中HB-FRP加固技术的锚固效果最好;预应力FRP加固技术因具有较高的材料强度利用率而展现出良好的应用前景;后续应深入研究FRP加固设计理论,综合考虑各参数对加固结构性能的影响,建立统一可靠且实用的计算模型;应进一步研究既有混凝土加固结构在多因素耦合作用下的长期性能和耐久性能,完善FRP加固混凝土桥梁设计方法和规范体系。

面向中外学生的混凝土结构教学实践与思考

该文分析“一带一路”倡议背景下留学生混凝土结构教学面临的国际化教学挑战,传统的混凝土教学模式在国外留学生加入教育背景下亟需改进。该文首先分析天津大学土木工程系留学生构成比例及混凝土教学所面临的不同国家混凝土设计规范差异,分析作者所教授的混凝土结构设计和全英文高等混凝土结构等课程特点,并提出加强课程思政、丰富教学方法、重视实习基地、因材施教调整教学内容和多重考核方式的课程教学措施。以期达到提升混凝土教学质量,提升中外学生培养质量,增强中外学生人才培养素质,助力我国“一带一路”发展建设。