Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

A review of artificial intelligence applications in high-speed railway systems

In recent years,the global surge of High-speed Railway(HSR)revolutionized ground transportation,providing secure,comfortable,and punctual services.The next-gen HSR,fueled by emerging services like video surveillance,emergency communication,and real-time scheduling,demands advanced capabilities in real-time perception,automated driving,and digitized services,which accelerate the integration and application of Artificial Intelligence(AI)in the HSR system.This paper first provides a brief overview of AI,covering its origin,evolution,and breakthrough applications.A comprehensive review is then given regarding the most advanced AI technologies and applications in three macro application domains of the HSR system:mechanical manufacturing and electrical control,communication and signal control,and transportation management.The literature is categorized and compared across nine application directions labeled as intelligent manufacturing of trains and key components,forecast of railroad maintenance,optimization of energy consumption in railroads and trains,communication security,communication dependability,channel modeling and estimation,passenger scheduling,traffic flow forecasting,high-speed railway smart platform.Finally,challenges associated with the application of AI are discussed,offering insights for future research directions.

Cooperative User-Scheduling and Resource Allocation Optimization for Intelligent Reflecting Surface Enhanced LEO Satellite Communication

Lower Earth Orbit(LEO) satellite becomes an important part of complementing terrestrial communication due to its lower orbital altitude and smaller propagation delay than Geostationary satellite. However, the LEO satellite communication system cannot meet the requirements of users when the satellite-terrestrial link is blocked by obstacles. To solve this problem, we introduce Intelligent reflect surface(IRS) for improving the achievable rate of terrestrial users in LEO satellite communication. We investigated joint IRS scheduling, user scheduling, power and bandwidth allocation(JIRPB) optimization algorithm for improving LEO satellite system throughput.The optimization problem of joint user scheduling and resource allocation is formulated as a non-convex optimization problem. To cope with this problem, the nonconvex optimization problem is divided into resource allocation optimization sub-problem and scheduling optimization sub-problem firstly. Second, we optimize the resource allocation sub-problem via alternating direction multiplier method(ADMM) and scheduling sub-problem via Lagrangian dual method repeatedly.Third, we prove that the proposed resource allocation algorithm based ADMM approaches sublinear convergence theoretically. Finally, we demonstrate that the proposed JIRPB optimization algorithm improves the LEO satellite communication system throughput.

Low-Cost Federated Broad Learning for Privacy-Preserved Knowledge Sharing in the RIS-Aided Internet of Vehicles

High-efficiency and low-cost knowledge sharing can improve the decision-making ability of autonomous vehicles by mining knowledge from the Internet of Vehicles(IoVs).However,it is challenging to ensure high efficiency of local data learning models while preventing privacy leakage in a high mobility environment.In order to protect data privacy and improve data learning efficiency in knowledge sharing,we propose an asynchronous federated broad learning(FBL)framework that integrates broad learning(BL)into federated learning(FL).In FBL,we design a broad fully connected model(BFCM)as a local model for training client data.To enhance the wireless channel quality for knowledge sharing and reduce the communication and computation cost of participating clients,we construct a joint resource allocation and reconfigurable intelligent surface(RIS)configuration optimization framework for FBL.The problem is decoupled into two convex subproblems.Aiming to improve the resource scheduling efficiency in FBL,a double Davidon–Fletcher–Powell(DDFP)algorithm is presented to solve the time slot allocation and RIS configuration problem.Based on the results of resource scheduling,we design a reward-allocation algorithm based on federated incentive learning(FIL)in FBL to compensate clients for their costs.The simulation results show that the proposed FBL framework achieves better performance than the comparison models in terms of efficiency,accuracy,and cost for knowledge sharing in the IoV.

Ensuring Secure Platooning of Constrained Intelligent and Connected Vehicles Against Byzantine Attacks:A Distributed MPC Framework

This study investigates resilient platoon control for constrained intelligent and connected vehicles(ICVs)against F-local Byzantine attacks.We introduce a resilient distributed model-predictive platooning control framework for such ICVs.This framework seamlessly integrates the predesigned optimal control with distributed model predictive control(DMPC)optimization and introduces a unique distributed attack detector to ensure the reliability of the transmitted information among vehicles.Notably,our strategy uses previously broadcasted information and a specialized convex set,termed the“resilience set”,to identify unreliable data.This approach significantly eases graph robustness prerequisites,requiring only an(F+1)-robust graph,in contrast to the established mean sequence reduced algorithms,which require a minimum(2F+1)-robust graph.Additionally,we introduce a verification algorithm to restore trust in vehicles under minor attacks,further reducing communication network robustness.Our analysis demonstrates the recursive feasibility of the DMPC optimization.Furthermore,the proposed method achieves exceptional control performance by minimizing the discrepancies between the DMPC control inputs and predesigned platoon control inputs,while ensuring constraint compliance and cybersecurity.Simulation results verify the effectiveness of our theoretical findings.

Intelligent Recognition Using Ultralight Multifunctional Nano‑Layered Carbon Aerogel Sensors with Human‑Like Tactile Perception

Humans can perceive our complex world through multi-sensory fusion.Under limited visual conditions,people can sense a variety of tactile signals to identify objects accurately and rapidly.However,replicating this unique capability in robots remains a significant challenge.Here,we present a new form of ultralight multifunctional tactile nano-layered carbon aerogel sensor that provides pressure,temperature,material recognition and 3D location capabilities,which is combined with multimodal supervised learning algorithms for object recognition.The sensor exhibits human-like pressure(0.04–100 kPa)and temperature(21.5–66.2℃)detection,millisecond response times(11 ms),a pressure sensitivity of 92.22 kPa^(−1)and triboelectric durability of over 6000 cycles.The devised algorithm has universality and can accommodate a range of application scenarios.The tactile system can identify common foods in a kitchen scene with 94.63%accuracy and explore the topographic and geomorphic features of a Mars scene with 100%accuracy.This sensing approach empowers robots with versatile tactile perception to advance future society toward heightened sensing,recognition and intelligence.

Label Recovery and Trajectory Designable Network for Transfer Fault Diagnosis of Machines With Incorrect Annotation

The success of deep transfer learning in fault diagnosis is attributed to the collection of high-quality labeled data from the source domain.However,in engineering scenarios,achieving such high-quality label annotation is difficult and expensive.The incorrect label annotation produces two negative effects:1)the complex decision boundary of diagnosis models lowers the generalization performance on the target domain,and2)the distribution of target domain samples becomes misaligned with the false-labeled samples.To overcome these negative effects,this article proposes a solution called the label recovery and trajectory designable network(LRTDN).LRTDN consists of three parts.First,a residual network with dual classifiers is to learn features from cross-domain samples.Second,an annotation check module is constructed to generate a label anomaly indicator that could modify the abnormal labels of false-labeled samples in the source domain.With the training of relabeled samples,the complexity of diagnosis model is reduced via semi-supervised learning.Third,the adaptation trajectories are designed for sample distributions across domains.This ensures that the target domain samples are only adapted with the pure-labeled samples.The LRTDN is verified by two case studies,in which the diagnosis knowledge of bearings is transferred across different working conditions as well as different yet related machines.The results show that LRTDN offers a high diagnosis accuracy even in the presence of incorrect annotation.

Smart prediction of liquefaction-induced lateral spreading

The prediction of liquefaction-induced lateral spreading/displacement(Dh)is a challenging task for civil/geotechnical engineers.In this study,a new approach is proposed to predict Dh using gene expression programming(GEP).Based on statistical reasoning,individual models were developed for two topographies:free-face and gently sloping ground.Along with a comparison with conventional approaches for predicting the Dh,four additional regression-based soft computing models,i.e.Gaussian process regression(GPR),relevance vector machine(RVM),sequential minimal optimization regression(SMOR),and M5-tree,were developed and compared with the GEP model.The results indicate that the GEP models predict Dh with less bias,as evidenced by the root mean square error(RMSE)and mean absolute error(MAE)for training(i.e.1.092 and 0.815;and 0.643 and 0.526)and for testing(i.e.0.89 and 0.705;and 0.773 and 0.573)in free-face and gently sloping ground topographies,respectively.The overall performance for the free-face topology was ranked as follows:GEP>RVM>M5-tree>GPR>SMOR,with a total score of 40,32,24,15,and 10,respectively.For the gently sloping condition,the performance was ranked as follows:GEP>RVM>GPR>M5-tree>SMOR with a total score of 40,32,21,19,and 8,respectively.Finally,the results of the sensitivity analysis showed that for both free-face and gently sloping ground,the liquefiable layer thickness(T_(15))was the major parameter with percentage deterioration(%D)value of 99.15 and 90.72,respectively.

IRS Assisted UAV Communications against Proactive Eavesdropping in Mobile Edge Computing Networks

In this paper,we consider mobile edge computing(MEC)networks against proactive eavesdropping.To maximize the transmission rate,IRS assisted UAV communications are applied.We take the joint design of the trajectory of UAV,the transmitting beamforming of users,and the phase shift matrix of IRS.The original problem is strong non-convex and difficult to solve.We first propose two basic modes of the proactive eavesdropper,and obtain the closed-form solution for the boundary conditions of the two modes.Then we transform the original problem into an equivalent one and propose an alternating optimization(AO)based method to obtain a local optimal solution.The convergence of the algorithm is illustrated by numerical results.Further,we propose a zero forcing(ZF)based method as sub-optimal solution,and the simulation section shows that the proposed two schemes could obtain better performance compared with traditional schemes.

Current status and construction scheme of smart geothermal field technology

To address the key problems in the application of intelligent technology in geothermal development,smart application scenarios for geothermal development are constructed.The research status and existing challenges of intelligent technology in each scenario are analyzed,and the construction scheme of smart geothermal field system is proposed.The smart geothermal field is an organic integration of geothermal development engineering and advanced technologies such as the artificial intelligence.At present,the technology of smart geothermal field is still in the exploratory stage.It has been tested for application in scenarios such as intelligent characterization of geothermal reservoirs,dynamic intelligent simulation of geothermal reservoirs,intelligent optimization of development schemes and smart management of geothermal development.However,it still faces many problems,including the high computational cost,difficult real-time response,multiple solutions and strong model dependence,difficult real-time optimization of dynamic multi-constraints,and deep integration of multi-source data.The construction scheme of smart geothermal field system is proposed,which consists of modules including the full database,intelligent characterization,intelligent simulation and intelligent optimization control.The connection between modules is established through the data transmission and the model interaction.In the next stage,it is necessary to focus on the basic theories and key technologies in each module of the smart geothermal field system,to accelerate the lifecycle intelligent transformation of the geothermal development and utilization,and to promote the intelligent,stable,long-term,optimal and safe production of geothermal resources.

Multi-level intelligence empowering lithium-ion batteries

With the significant and widespread application of lithium-ion batteries,there is a growing demand for improved performances of lithium-ion batteries.The intricate degradation throughout the whole lifecycle profoundly impacts the safety,durability,and reliability of lithium-ion batteries.To ensure the long-term,safe,and efficient operation of lithium-ion batteries in various fields,there is a pressing need for enhanced battery intelligence that can withstand extreme events.This work reviews the current status of intelligent battery technology from three perspectives:intelligent response,intelligent sensing,and intelligent management.The intelligent response of battery materials forms the foundation for battery stability,the intelligent sensing of multi-dimensional signals is essential for battery management,and the intelligent management ensures the long-term stable operation of lithium-ion batteries.The critical challenges encountered in the development of intelligent battery technology from each perspective are thoroughly analyzed,and potential solutions are proposed,aiming to facilitate the rapid development of intelligent battery technologies.

Intelligent casting:Empowering the future foundry industry

Emerging technological advances are reshaping the casting sector in latest decades.Casting technology is evolving towards intelligent casting paradigm that involves automation,greenization and intelligentization,which attracts more and more attention from the academic and industry communities.In this paper,the main features of casting technology were briefly summarized and forecasted,and the recent developments of key technologies and the innovative efforts made in promoting intelligent casting process were discussed.Moreover,the technical visions of intelligent casting process were also put forward.The key technologies for intelligent casting process comprise 3D printing technologies,intelligent mold technologies and intelligent process control technologies.In future,the intelligent mold that derived from mold with sensors,control devices and actuators will probably incorporate the Internet of Things,online inspection,embedded simulation,decision-making and control system,and other technologies to form intelligent cyber-physical casting system,which may pave the way to realize intelligent casting.It is promising that the intelligent casting process will eventually achieve the goal of real-time process optimization and full-scale control,with the defects,microstructure,performance,and service life of the fabricated castings can be accurately predicted and tailored.

Holistic and localized preparation methods for triboelectric sensors:principles,applications and perspectives

With the arrival of intelligent terminals,triboelectric nanogenerators,as a new kind of energy converter,are considered one of the most important technologies for the next generation of intelligent electronics.As a self-powered sensor,it can greatly reduce the power consumption of the entire sensing system by transforming external mechanical energy to electricity.However,the fabrication method of triboelectric sensors largely determines their functionality and performance.This review provides an overview of various methods used to fabricate triboelectric sensors,with a focus on the processes of micro-electro-mechanical systems technology,three-dimensional printing,textile methods,template-assisted methods,and material synthesis methods for manufacturing.The working mechanisms and suitable application scenarios of various methods are outlined.Subsequently,the advantages and disadvantages of various methods are summarized,and reference schemes for the subsequent application of these methods are included.Finally,the opportunities and challenges faced by different methods are discussed,as well as their potential for application in various intelligent systems in the Internet of Things.

An Optimal and Fast Precoding for RIS-Assisted MIMO Communications

In addition to conventional antenna-based array,the reconfigurable intelligent surface(RIS)holds promise as an alternative technology for manufacturing massive multi-input multi-output(MIMO)array for beyond 5G communications.This paper designs a fast algorithm to optimize the RIS-based MIMO precoder for maximizing the spectral efficiency,which includes the digital precoder and the RIS reflection phases.We evaluate the optimality of the algorithm by deriving an RIS channel capacity upper bound utilizing majorization theory.Our scheme can work for an RIS in both frequency flat and frequency selective channels,with either continuously or discretely tunable phases.The simulation results show that the proposed algorithm can achieve the capacity upper bound in some scenarios,which empirically proves its optimality.It is also shown that our algorithm is one-to-two orders of magnitude faster than the state-of-the-art methods in the literature.

A real-time intelligent lithology identification method based on a dynamic felling strategy weighted random forest algorithm

Real-time intelligent lithology identification while drilling is vital to realizing downhole closed-loop drilling. The complex and changeable geological environment in the drilling makes lithology identification face many challenges. This paper studies the problems of difficult feature information extraction,low precision of thin-layer identification and limited applicability of the model in intelligent lithologic identification. The author tries to improve the comprehensive performance of the lithology identification model from three aspects: data feature extraction, class balance, and model design. A new real-time intelligent lithology identification model of dynamic felling strategy weighted random forest algorithm(DFW-RF) is proposed. According to the feature selection results, gamma ray and 2 MHz phase resistivity are the logging while drilling(LWD) parameters that significantly influence lithology identification. The comprehensive performance of the DFW-RF lithology identification model has been verified in the application of 3 wells in different areas. By comparing the prediction results of five typical lithology identification algorithms, the DFW-RF model has a higher lithology identification accuracy rate and F1 score. This model improves the identification accuracy of thin-layer lithology and is effective and feasible in different geological environments. The DFW-RF model plays a truly efficient role in the realtime intelligent identification of lithologic information in closed-loop drilling and has greater applicability, which is worthy of being widely used in logging interpretation.

Intelligent responsive self-assembled micro-nanocapsules:Used to delay gel gelation time

In the application of polymer gels to profile control and water shutoff,the gelation time will directly determine whether the gel can"go further"in the formation,but the most of the methods for delaying gel gelation time are complicated or have low responsiveness.There is an urgent need for an effective method for delaying gel gelation time with intelligent response.Inspired by the slow-release effect of drug capsules,this paper uses the self-assembly effect of gas-phase hydrophobic SiO_(2) in aqueous solution as a capsule to prepare an intelligent responsive self-assembled micro-nanocapsules.The capsule slowly releases the cross-linking agent under the stimulation of external conditions such as temperature and pH value,thus delaying gel gelation time.When the pH value is 2 and the concentration of gas-phase hydrophobic SiO_(2) particles is 10%,the gelation time of the capsule gel system at 30,60,90,and 120℃is12.5,13.2,15.2,and 21.1 times longer than that of the gel system without containing capsule,respectively.Compared with other methods,the yield stress of the gel without containing capsules was 78 Pa,and the yield stress after the addition of capsules was 322 Pa.The intelligent responsive self-assembled micronanocapsules prepared by gas-phase hydrophobic silica nanoparticles can not only delay the gel gelation time,but also increase the gel strength.The slow release of cross-linking agent from capsule provides an effective method for prolongating the gelation time of polymer gels.

Distributed IRS-Aided DF Relaying Systems:Performance Analysis and Optimization

Intelligent reflecting surface(IRS)is a newly emerged and promising paradigm to substantially improve the performance of wireless communications by constructing favorable communication channels via properly tuning massive reflecting elements.This paper considers a distributed IRS aided decode-and-forward(DF)relaying system over Nakagami-m fading channels.Based on a tight approximation for the distribution of the received signalto-noise ratio(SNR),we first derive exact closed-form expressions of the outage probability,ergodic capacity,and energy efficiency for the considered system.Moreover,we propose the optimal IRS configuration considering the energy efficiency and pilot overhead.Finally,we compare the performance between the distributed IRS-aided DF relaying and multi-IRS-only systems,and verify the analytical results by using monte carlo simulations.

Privacy-Preserving Large-Scale AI Models for Intelligent Railway Transportation Systems:Hierarchical Poisoning Attacks and Defenses in Federated Learning

The development of Intelligent Railway Transportation Systems necessitates incorporating privacy-preserving mechanisms into AI models to protect sensitive information and enhance system efficiency.Federated learning offers a promising solution by allowing multiple clients to train models collaboratively without sharing private data.However,despite its privacy benefits,federated learning systems are vulnerable to poisoning attacks,where adversaries alter local model parameters on compromised clients and send malicious updates to the server,potentially compromising the global model’s accuracy.In this study,we introduce PMM(Perturbation coefficient Multiplied by Maximum value),a new poisoning attack method that perturbs model updates layer by layer,demonstrating the threat of poisoning attacks faced by federated learning.Extensive experiments across three distinct datasets have demonstrated PMM’s ability to significantly reduce the global model’s accuracy.Additionally,we propose an effective defense method,namely CLBL(Cluster Layer By Layer).Experiment results on three datasets have confirmed CLBL’s effectiveness.

Introduction to the Special Issue on Machine Learning-Guided Intelligent Modeling with Its Industrial Applications

With the advancement of Artificial Intelligence(AI)technology,traditional industrial systems are undergoing an intelligent transformation,bringing together advanced computing,communication and control technologies,Machine Learning(ML)-based intelligentmodelling has become a newparadigm for solving problems in the industrial domain[1–3].With numerous applications and diverse data types in the industrial domain,algorithmic and data-driven ML techniques can intelligently learn potential correlations between complex data and make efficient decisions while reducing human intervention.However,in real-world application scenarios,existing algorithms may have a variety of limitations,such as small data volumes,small detection targets,low efficiency,and algorithmic gaps in specific application domains[4].Therefore,many new algorithms and strategies have been proposed to address the challenges in industrial applications[5–8].

Safety control strategy of spinal lamina cutting based on force and cutting depth signals

Laminectomy is one of the most common posterior spinal operations. Since the lamina is adjacent to important tissues such as nerves, once damaged, it can cause serious com-plications and even lead to paralysis. In order to prevent the above injuries and com-plications, ultrasonic bone scalpel and surgical robots have been introduced into spinal laminectomy, and many scholars have studied the recognition method of the bone tissue status. Currently, almost all methods to achieve recognition of bone tissue are based on sensor signals collected by high‐precision sensors installed at the end of surgical robots. However, the previous methods could not accurately identify the state of spinal bone tissue. Innovatively, the identification of bone tissue status was regarded as a time series classification task, and the classification algorithm LSTM‐FCN was used to process fusion signals composed of force and cutting depth signals, thus achieving an accurate classi-fication of the lamina bone tissue status. In addition, it was verified that the accuracy of the proposed method could reach 98.85% in identifying the state of porcine spinal laminectomy. And the maximum penetration distance can be controlled within 0.6 mm, which is safe and can be used in practice.