Reinforcement Technology for Soft Soil Roadbed in the Widened Section of Expressway Expansion

This article examines the soft soil roadbed reinforcement technology for widened sections of highways in a specific project.It provides an overview of the project,the principles of soft soil roadbed reinforcement technology for wide sections,and its practical application.The analysis aims to offer guidance on applying soft soil roadbed wide section reinforcement technology and enhancing the overall quality of similar projects.

A new horizon for neuroscience:terahertz biotechnology in brain research

Terahertz biotechnology has been increasingly applied in various biomedical fields and has especially shown great potential for application in brain sciences.In this article,we review the development of terahertz biotechnology and its applications in the field of neuropsychiatry.Available evidence indicates promising prospects for the use of terahertz spectroscopy and terahertz imaging techniques in the diagnosis of amyloid disease,cerebrovascular disease,glioma,psychiatric disease,traumatic brain injury,and myelin deficit.In vitro and animal experiments have also demonstrated the potential therapeutic value of terahertz technology in some neuropsychiatric diseases.Although the precise underlying mechanism of the interactions between terahertz electromagnetic waves and the biosystem is not yet fully understood,the research progress in this field shows great potential for biomedical noninvasive diagnostic and therapeutic applications.However,the biosafety of terahertz radiation requires further exploration regarding its two-sided efficacy in practical applications.This review demonstrates that terahertz biotechnology has the potential to be a promising method in the field of neuropsychiatry based on its unique advantages.

Application of a combined supporting technology with U-shaped steel support and anchor-grouting to surrounding soft rock reinforcement in roadway

Soft rock surrounding deep roadway has poor stability and long-term rheological effect. More and larger deformation problems of surrounding rock occur due to adverse supporting measures for such roadways, which not only affects the engineering safety critically but also improves the maintenance costs. This paper takes the main rail roadway with severely deformation in China＇s Zaoquan coal mine as an example to study the long-term deformation tendency and damage zone by means of in-situ deformation monitoring and acoustic wave testing technique. A three-dimensional finite element model reflecting the engineering geological condition and initial design scheme is established by ABAQUS. Then, on the basis of field monitoring deformation data, the surrounding rock geotechnical and theological parameters of the roadway are obtained by back analysis. A combined supporting technology with U-shaped steel support and anchor-grouting is proposed for the surrounding soft rock. The numerical simulation of the combined supporting technology and in-situ deformation monitoring results show that the soft rock surrounding the roadway has been held effectively.

Deformation characteristics and reinforcement technology for entry subjected to mining-induced stresses

The entry at Zhangcun coal mine in Lu＇an coal mining area in Shanxi Province suffered from severe mining-induced stresses with the heading face driven oppositely to an adjacent working face. In this paper, the characteristics of deformation and failure of the entry were investigated in terms of the tempo-spatial relations between heading and working faces through field study and numerical modeling. The three-dimensional （3D） finite difference models were built to investigate stresses, displacements and damages in the surrounding rocks of the entry and the working face. The field study includes selection of reinforcing methods and materials, design parameters, and determination of cable prestress. The monitoring data of entry deformation and stress along the cables during every stage were presented. The state of the reinforced entry was evaluated based on the monitoring data. The results demonstrate that before the heading face of the entry crosses the adjacent working face, the influence of advanced abutment pressure caused by adjacent working face upon the entry is not significant. After they cross each other, however, the lateral abutment pressure will have an evident impact on the entry. The displacement rate of the entry will be greatly increased and reaches a certain value within a certain distance between the heading face and the working face. Then, it will increase again with the presence of secondary mining-induced pressure on the entry when the present working face advances. The fully-grouted cable with short length, high strength and high prestress is an effective way to reinforce the entry suffering from severe mining-induced stresses, which greatly reduces the displacement and failure possibility of the entry. Finally, the principles and recommendations for reinforcing design of entries suffering from severe mining-induced stresses were proposed according to field study, numerical modeling and experiences from other coal mines. Problems encountered in field study and suggestions for reinforcement were also discussed.

Analysis on Construction Technology and Reinforcement Technology of Building Foundation

In the field of construction engineering,foundation engineering plays a critical role.In actual construction,we must first effectively regulate the foundation construction to ensure the safety and stability of the entire building in order to improve the overall quality of the project.It's also important to look into the technologies that go into building foundations.The construction technology and reinforcing technology of building foundations are examined in this study as a reference.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Cognitive interference decision method for air defense missile fuze based on reinforcement learning

To solve the problem of the low interference success rate of air defense missile radio fuzes due to the unified interference form of the traditional fuze interference system,an interference decision method based Q-learning algorithm is proposed.First,dividing the distance between the missile and the target into multiple states to increase the quantity of state spaces.Second,a multidimensional motion space is utilized,and the search range of which changes with the distance of the projectile,to select parameters and minimize the amount of ineffective interference parameters.The interference effect is determined by detecting whether the fuze signal disappears.Finally,a weighted reward function is used to determine the reward value based on the range state,output power,and parameter quantity information of the interference form.The effectiveness of the proposed method in selecting the range of motion space parameters and designing the discrimination degree of the reward function has been verified through offline experiments involving full-range missile rendezvous.The optimal interference form for each distance state has been obtained.Compared with the single-interference decision method,the proposed decision method can effectively improve the success rate of interference.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

The Whole Pre-stressing Reinforcement Technology Directed by Conception Reinforcement and the Prospect of Engineering Application

The paper puts forward a new thinking and thought in engineering structural reinforcement-“conception reinforcement”. The “conception reinforcement”means using the human being’s thinking and judgment, deciding the fundamental question and principle of engineering structural reinforcement in macrocosm. It expounds the whole prestressing reinforcement technology owing to the direction of conception reinforcement thought, and introduces the prospect of engineering application.

A frequency servo SoC with output power stabilization loop technology for miniaturized atomic clocks

A frequency servo system-on-chip(FS-SoC)featuring output power stabilization technology is introduced in this study for high-precision and miniaturized cesium(Cs)atomic clocks.The proposed power stabilization loop(PSL)technique,incorporating an off-chip power detector(PD),ensures that the output power of the FS-SoC remains stable,mitigating the impact of power fluctuations on the atomic clock's stability.Additionally,a one-pulse-per-second(1PPS)is employed to syn-chronize the clock with GPS.Fabricated using 65 nm CMOS technology,the measured phase noise of the FS-SoC stands at-69.5 dBc/Hz@100 Hz offset and-83.9 dBc/Hz@1 kHz offset,accompanied by a power dissipation of 19.7 mW.The Cs atomic clock employing the proposed FS-SoC and PSL obtains an Allan deviation of 1.7×10^(-11) with 1-s averaging time.

Development and technology status of energy storage in depleted gas reservoirs

Utilizing energy storage in depleted oil and gas reservoirs can improve productivity while reducing power costs and is one of the best ways to achieve synergistic development of"Carbon Peak–Carbon Neutral"and"Underground Resource Utiliza-tion".Starting from the development of Compressed Air Energy Storage(CAES)technology,the site selection of CAES in depleted gas and oil reservoirs,the evolution mechanism of reservoir dynamic sealing,and the high-flow CAES and injection technology are summarized.It focuses on analyzing the characteristics,key equipment,reservoir construction,application scenarios and cost analysis of CAES projects,and sorting out the technical key points and existing difficulties.The devel-opment trend of CAES technology is proposed,and the future development path is scrutinized to provide reference for the research of CAES projects in depleted oil and gas reservoirs.

Key Technology and Complete-set Equipment for Intelligent Construction of HSR Simply-supported Girder Reinforcement Framework

In view of the problems such as backward production mode,poor quality stability,high safety risk and incomplete control system during erection of the reinforcement framework of simply-supported box girders for high-speed railway(HSR),and in combination with the key points and main challenges in the reinforcement framework construction of Guangzhou-Zhanjiang HSR,the overall technical route for the intelligent manufacturing of reinforcement framework of simply-supported box girders is put forward.The component design of reinforcement framework of simply supported box girder is carried out based on BIM,and the feasibility of the scheme is verified through segment assembly test.The assembly techniques are studied in combination with the mesh design scheme to achieve rapid forming of the reinforcement framework.R&D of automatic processing equipment for components,material transshipment equipment,automatic hoisting equipment and technological equipment for assembly clamping fixture are carried out to realize the overall design of equipment production line.An intelligent control system is developed for the whole-process intelligent construction of reinforcement framework to realize the full life-cycle applications for the workshop production and visual management including intelligent layout and quality traceability.The research results systematically optimize and innovate the assembly and forming technologies of reinforcement framework in the prefabrication beam yard of high-speed railway,realize the component processing,automatic assembly and information technology management,improve the construction quality,efficiency and information technology level of intelligent manufacturing of reinforcement framework of railway prefabricated beam as a whole,and reduce the construction cost of the project.The research has realized a major breakthrough in the construction technology of railway prefabricated box girders,which has the extensive technical and market promotion values.

Bridge Bidding via Deep Reinforcement Learning and Belief Monte Carlo Search

Contract Bridge,a four-player imperfect information game,comprises two phases:bidding and playing.While computer programs excel at playing,bidding presents a challenging aspect due to the need for information exchange with partners and interference with communication of opponents.In this work,we introduce a Bridge bidding agent that combines supervised learning,deep reinforcement learning via self-play,and a test-time search approach.Our experiments demonstrate that our agent outperforms WBridge5,a highly regarded computer Bridge software that has won multiple world championships,by a performance of 0.98 IMPs(international match points)per deal over 10000 deals,with a much cost-effective approach.The performance significantly surpasses previous state-of-the-art(0.85 IMPs per deal).Note 0.1 IMPs per deal is a significant improvement in Bridge bidding.

Distributed Graph Database Load Balancing Method Based on Deep Reinforcement Learning

This paper focuses on the scheduling problem of workflow tasks that exhibit interdependencies.Unlike indepen-dent batch tasks,workflows typically consist of multiple subtasks with intrinsic correlations and dependencies.It necessitates the distribution of various computational tasks to appropriate computing node resources in accor-dance with task dependencies to ensure the smooth completion of the entire workflow.Workflow scheduling must consider an array of factors,including task dependencies,availability of computational resources,and the schedulability of tasks.Therefore,this paper delves into the distributed graph database workflow task scheduling problem and proposes a workflow scheduling methodology based on deep reinforcement learning(DRL).The method optimizes the maximum completion time(makespan)and response time of workflow tasks,aiming to enhance the responsiveness of workflow tasks while ensuring the minimization of the makespan.The experimental results indicate that the Q-learning Deep Reinforcement Learning(Q-DRL)algorithm markedly diminishes the makespan and refines the average response time within distributed graph database environments.In quantifying makespan,Q-DRL achieves mean reductions of 12.4%and 11.9%over established First-fit and Random scheduling strategies,respectively.Additionally,Q-DRL surpasses the performance of both DRL-Cloud and Improved Deep Q-learning Network(IDQN)algorithms,with improvements standing at 4.4%and 2.6%,respectively.With reference to average response time,the Q-DRL approach exhibits a significantly enhanced performance in the scheduling of workflow tasks,decreasing the average by 2.27%and 4.71%when compared to IDQN and DRL-Cloud,respectively.The Q-DRL algorithm also demonstrates a notable increase in the efficiency of system resource utilization,reducing the average idle rate by 5.02%and 9.30%in comparison to IDQN and DRL-Cloud,respectively.These findings support the assertion that Q-DRL not only upholds a lower average idle rate but also effectively curtails the average response time,thereby substantially improving processing efficiency and optimizing resource utilization within distributed graph database systems.

Development status and prospect of underground thermal energy storage technology

Underground Thermal Energy Storage(UTES)store unstable and non-continuous energy underground,releasing stable heat energy on demand.This effectively improve energy utilization and optimize energy allocation.As UTES technology advances,accommodating greater depth,higher temperature and multi-energy complementarity,new research challenges emerge.This paper comprehensively provides a systematic summary of the current research status of UTES.It categorized different types of UTES systems,analyzes the applicability of key technologies of UTES,and evaluate their economic and environmental benefits.Moreover,this paper identifies existing issues with UTES,such as injection blockage,wellbore scaling and corrosion,seepage and heat transfer in cracks,etc.It suggests deepening the research on blockage formation mechanism and plugging prevention technology,improving the study of anticorrosive materials and water treatment technology,and enhancing the investigation of reservoir fracture network characterization technology and seepage heat transfer.These recommendations serve as valuable references for promoting the high-quality development of UTES.

6G New Multiple Access Technology

As a basic technology at physical layer of mobile communications,non-orthogonal multiple access has been attracting wide attention across the academia and the industry.During the standardization of the fifth-generation(5G)of mobile communications,3GPP conducted preliminary study on non-orthogonal multiple access without reaching the consensus to standardize the technology.

Nano-scale Reinforcements and Properties of Al-Si-Cu Alloy Processed by High-Pressure Torsion

To improve the comprehensive mechanical properties of Al-Si-Cu alloy,it was treated by a high-pressure torsion process,and the effect of the deformation degree on the microstructure and properties of the Al-Si-Cu alloy was studied.The results show that the reinforcements(β-Si andθ-CuAl_(2)phases)of the Al-Si-Cu alloy are dispersed in theα-Al matrix phase with finer phase size after the treatment.The processed samples exhibit grain sizes in the submicron or even nanometer range,which effectively improves the mechanical properties of the material.The hardness and strength of the deformed alloy are both significantly raised to 268 HV and 390.04 MPa by 10 turns HPT process,and the fracture morphology shows that the material gradually transits from brittle to plastic before and after deformation.The elements interdiffusion at the interface between the phases has also been effectively enhanced.In addition,it is found that the severe plastic deformation at room temperature induces a ternary eutectic reaction,resulting in the formation of ternary Al+Si+CuAl_(2)eutectic.

QoS Routing Optimization Based on Deep Reinforcement Learning in SDN

To enhance the efficiency and expediency of issuing e-licenses within the power sector, we must confront thechallenge of managing the surging demand for data traffic. Within this realm, the network imposes stringentQuality of Service (QoS) requirements, revealing the inadequacies of traditional routing allocation mechanismsin accommodating such extensive data flows. In response to the imperative of handling a substantial influx of datarequests promptly and alleviating the constraints of existing technologies and network congestion, we present anarchitecture forQoS routing optimizationwith in SoftwareDefinedNetwork (SDN), leveraging deep reinforcementlearning. This innovative approach entails the separation of SDN control and transmission functionalities, centralizingcontrol over data forwardingwhile integrating deep reinforcement learning for informed routing decisions. Byfactoring in considerations such as delay, bandwidth, jitter rate, and packet loss rate, we design a reward function toguide theDeepDeterministic PolicyGradient (DDPG) algorithmin learning the optimal routing strategy to furnishsuperior QoS provision. In our empirical investigations, we juxtapose the performance of Deep ReinforcementLearning (DRL) against that of Shortest Path (SP) algorithms in terms of data packet transmission delay. Theexperimental simulation results show that our proposed algorithm has significant efficacy in reducing networkdelay and improving the overall transmission efficiency, which is superior to the traditional methods.

Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Reinforcement Learning in Process Industries:Review and Perspective

This survey paper provides a review and perspective on intermediate and advanced reinforcement learning(RL)techniques in process industries. It offers a holistic approach by covering all levels of the process control hierarchy. The survey paper presents a comprehensive overview of RL algorithms,including fundamental concepts like Markov decision processes and different approaches to RL, such as value-based, policy-based, and actor-critic methods, while also discussing the relationship between classical control and RL. It further reviews the wide-ranging applications of RL in process industries, such as soft sensors, low-level control, high-level control, distributed process control, fault detection and fault tolerant control, optimization,planning, scheduling, and supply chain. The survey paper discusses the limitations and advantages, trends and new applications, and opportunities and future prospects for RL in process industries. Moreover, it highlights the need for a holistic approach in complex systems due to the growing importance of digitalization in the process industries.