基于Transformer结构增强的神经网络架构搜索性能预测器

神经网络架构搜索(Neural Architecture Search,NAS)作为一种通过搜索算法设计神经网络架构的方法,在计算机视觉和自然语言处理等领域得到广泛应用,相较于人工设计网络,NAS方法可以减少设计成本并提高模型性能.但是NAS的性能评估需要对候选架构进行大量训练,由此带来的计算量占整个NAS的80%以上.为降低计算开销和时间成本,近年来已提出许多基于Transformer的NAS预测器,由于Transformer出色的结构编码能力可以更好地表示拓扑信息,因而得到广泛应用.但是,现有基于Transformer的NAS预测器依然存在三个问题:其一是在预处理阶段,传统的One-hot编码方式描述节点特征的能力较弱,只能区分不同操作节点类型,而难以表达操作的细节特征,如卷积核尺寸等.其二是在编码阶段,Transformer的自注意力机制导致模型结构信息缺失;其三是在评估阶段,现有的Transformer预测器仅使用多层感知机(Multilayer Perceptron,MLP)对前向传播图进行精度预测,忽略了反向传播梯度流对预测精度的影响,因此难以真正拟合NAS评估中的正、反向交替信息流图,导致预测器精度与实际运行精度误差波动极大(10%~90%).为解决上述问题,本文提出了一种基于Transformer结构增强的NAS性能预测方法.首先,在预处理阶段,本文提出了一种超维嵌入方法增加输入数据维度以强化节点操作的参数描述能力,其次,在编码阶段将Transformer编码后的信息与图结构信息共同输人一个图卷积网络(Graph Convolutional Network,GCN),弥补由自注意力机制引起的结构缺失.最后,在性能评估阶段,本文构建了同时包含前向传播和反向传播的全训练图,并将数据集信息、图结构编码与梯度编码共同输入到GCN网络预测器中,使预测结果更贴近模型真实性能.实验结果表明,本方法与目前最先进方法相比,肯德尔相关系数提高了7.45%,训练时间减少了1.55倍。

Oxygen functionalization-assisted anionic exchange toward unique construction of flower-like transition metal chalcogenide embedded carbon fabric for ultra-long life flexible energy storage and conversion

The metal-organic framework(MOF)derived Ni–Co–C–N composite alloys(NiCCZ)were“embedded”inside the carbon cloth(CC)strands as opposed to the popular idea of growing them upward to realize ultrastable energy storage and conversion application.The NiCCZ was then oxygen functionalized,facilitating the next step of stoichiometric sulfur anion diffusion during hydrothermal sulfurization,generating a flower-like metal hydroxysulfide structure(NiCCZOS)with strong partial implantation inside CC.Thus obtained NiCCZOS shows an excellent capacity when tested as a supercapacitor electrode in a three-electrode configuration.Moreover,when paired with the biomass-derived nitrogen-rich activated carbon,the asymmetric supercapacitor device shows almost 100%capacity retention even after 45,000 charge–discharge cycles with remarkable energy density(59.4 Wh kg^(-1)/263.8μWh cm^(–2))owing to a uniquely designed cathode.Furthermore,the same electrode performed as an excellent bifunctional water-splitting electrocatalyst with an overpotential of 271 mV for oxygen evolution reaction(OER)and 168.4 mV for hydrogen evolution reaction(HER)at 10 mA cm−2 current density along with 30 h of unhinged chronopotentiometric stability performance for both HER and OER.Hence,a unique metal chalcogenide composite electrode/substrate configuration has been proposed as a highly stable electrode material for flexible energy storage and conversion applications.

Application of buried auricular point combined with Wenjing Sanhan prescription in arteriosclerosis obliterans patients with resting pain

BACKGROUND Research on the combined use of ear acupoint embedding beans and warming meridians with cold-dispersing formulas for alleviating resting pain in patients with arteriosclerosis obliterans(ASO)remains limited.AIM To explore the therapeutic efficacy of auricular point embedding beans combined with Wenjing Sanhan prescription in alleviating resting pain in patients with lower-limb ASO.METHODS A total of 100 patients with ASO experiencing resting pain who were treated at our hospital from January 2022 to January 2023 were enrolled.They were randomly allocated into two groups using a double-blind approach.The control group was treated using a warming meridian with a cold-dispersing formula,while the study group received additional treatment with ear acupoint embedding beans.The clinical efficacy,ankle-brachial artery pressure ratio,hemorheological indicators,and traditional Chinese medicine symptom scores were compared between the two groups.RESULTS The clinical efficacy rate in the study group was significantly higher(94.00%)than that in the control group(72.00%,P<0.05).Moreover,the ankle-brachial artery pressure ratio was significantly higher in the study group after treatment(P<0.05).Hemorheological parameters,including whole blood viscosity,plasma viscosity(1.83±0.11)mPa/s,fibrinogen levels(3.30±0.21)g/L,platelet adhesion rate(49.87%±10.51%),and erythrocyte aggregation index(1.79±0)were improved in the study group compared to the control group.In addition,the scores for decreased skin temperature(1.41±0.26),intermittent claudication(1.30±0.20),and resting pain(1.23±0.31)were significantly lower in the study group than those in the control group(all P<0.05).The level of oxidative stress in the study group also exhibited significant improvement(P<0.05),and the levels of inflammatory factors were considerably lower than those in the control group.CONCLUSION The combination of ear point embedding beans and Wenjing Sanhan prescription demonstrates promising clinical efficacy in alleviating resting pain associated with ASO.

A Linked List Encryption Scheme for Image Steganography without Embedding

Information steganography has received more and more attention from scholars nowadays,especially in the area of image steganography,which uses image content to transmit information and makes the existence of secret information undetectable.To enhance concealment and security,the Steganography without Embedding(SWE)method has proven effective in avoiding image distortion resulting from cover modification.In this paper,a novel encrypted communication scheme for image SWE is proposed.It reconstructs the image into a multi-linked list structure consisting of numerous nodes,where each pixel is transformed into a single node with data and pointer domains.By employing a special addressing algorithm,the optimal linked list corresponding to the secret information can be identified.The receiver can restore the secretmessage fromthe received image using only the list header position information.The scheme is based on the concept of coverless steganography,eliminating the need for any modifications to the cover image.It boasts high concealment and security,along with a complete message restoration rate,making it resistant to steganalysis.Furthermore,this paper proposes linked-list construction schemeswithin theproposedframework,which caneffectively resist a variety of attacks,includingnoise attacks and image compression,demonstrating a certain degree of robustness.To validate the proposed framework,practical tests and comparisons are conducted using multiple datasets.The results affirm the framework’s commendable performance in terms of message reduction rate,hidden writing capacity,and robustness against diverse attacks.

Intelligent casting:Empowering the future foundry industry

Emerging technological advances are reshaping the casting sector in latest decades.Casting technology is evolving towards intelligent casting paradigm that involves automation,greenization and intelligentization,which attracts more and more attention from the academic and industry communities.In this paper,the main features of casting technology were briefly summarized and forecasted,and the recent developments of key technologies and the innovative efforts made in promoting intelligent casting process were discussed.Moreover,the technical visions of intelligent casting process were also put forward.The key technologies for intelligent casting process comprise 3D printing technologies,intelligent mold technologies and intelligent process control technologies.In future,the intelligent mold that derived from mold with sensors,control devices and actuators will probably incorporate the Internet of Things,online inspection,embedded simulation,decision-making and control system,and other technologies to form intelligent cyber-physical casting system,which may pave the way to realize intelligent casting.It is promising that the intelligent casting process will eventually achieve the goal of real-time process optimization and full-scale control,with the defects,microstructure,performance,and service life of the fabricated castings can be accurately predicted and tailored.

Enhancing visual security: An image encryption scheme based on parallel compressive sensing and edge detection embedding

A novel image encryption scheme based on parallel compressive sensing and edge detection embedding technology is proposed to improve visual security. Firstly, the plain image is sparsely represented using the discrete wavelet transform.Then, the coefficient matrix is scrambled and compressed to obtain a size-reduced image using the Fisher–Yates shuffle and parallel compressive sensing. Subsequently, to increase the security of the proposed algorithm, the compressed image is re-encrypted through permutation and diffusion to obtain a noise-like secret image. Finally, an adaptive embedding method based on edge detection for different carrier images is proposed to generate a visually meaningful cipher image. To improve the plaintext sensitivity of the algorithm, the counter mode is combined with the hash function to generate keys for chaotic systems. Additionally, an effective permutation method is designed to scramble the pixels of the compressed image in the re-encryption stage. The simulation results and analyses demonstrate that the proposed algorithm performs well in terms of visual security and decryption quality.

Randomization Strategies in Image Steganography Techniques:A Review

Image steganography is one of the prominent technologies in data hiding standards.Steganographic system performance mostly depends on the embedding strategy.Its goal is to embed strictly confidential information into images without causing perceptible changes in the original image.The randomization strategies in data embedding techniques may utilize random domains,pixels,or region-of-interest for concealing secrets into a cover image,preventing information from being discovered by an attacker.The implementation of an appropriate embedding technique can achieve a fair balance between embedding capability and stego image imperceptibility,but it is challenging.A systematic approach is used with a standard methodology to carry out this study.This review concentrates on the critical examination of several embedding strategies,incorporating experimental results with state-of-the-art methods emphasizing the robustness,security,payload capacity,and visual quality metrics of the stego images.The fundamental ideas of steganography are presented in this work,along with a unique viewpoint that sets it apart from previous works by highlighting research gaps,important problems,and difficulties.Additionally,it offers a discussion of suggested directions for future study to advance and investigate uncharted territory in image steganography.

CO_(2)flooding in shale oil reservoir with radial borehole fracturing for CO_(2)storage and enhanced oil recovery

This study introduces a novel method integrating CO_(2)flooding with radial borehole fracturing for enhanced oil recovery and CO_(2)underground storage,a solution to the limited vertical stimulation reservoir volume in horizontal well fracturing.A numerical model is established to investigate the production rate,reservoir pressure field,and CO_(2)saturation distribution corresponding to changing time of CO_(2)flooding with radial borehole fracturing.A sensitivity analysis on the influence of CO_(2)injection location,layer spacing,pressure difference,borehole number,and hydraulic fractures on oil production and CO_(2)storage is conducted.The CO_(2)flooding process is divided into four stages.Reductions in layer spacing will significantly improve oil production rate and gas storage capacity.However,serious gas channeling can occur when the spacing is lower than 20 m.Increasing the pressure difference between the producer and injector,the borehole number,the hydraulic fracture height,and the fracture width can also increase the oil production rate and gas storage rate.Sensitivity analysis shows that layer spacing and fracture height greatly influence gas storage and oil production.Research outcomes are expected to provide a theoretical basis for the efficient development of shale oil reservoirs in the vertical direction.

Advancements in machine learning for material design and process optimization in the field of additive manufacturing

Additive manufacturing technology is highly regarded due to its advantages,such as high precision and the ability to address complex geometric challenges.However,the development of additive manufacturing process is constrained by issues like unclear fundamental principles,complex experimental cycles,and high costs.Machine learning,as a novel artificial intelligence technology,has the potential to deeply engage in the development of additive manufacturing process,assisting engineers in learning and developing new techniques.This paper provides a comprehensive overview of the research and applications of machine learning in the field of additive manufacturing,particularly in model design and process development.Firstly,it introduces the background and significance of machine learning-assisted design in additive manufacturing process.It then further delves into the application of machine learning in additive manufacturing,focusing on model design and process guidance.Finally,it concludes by summarizing and forecasting the development trends of machine learning technology in the field of additive manufacturing.

Advances of embedded resistive random access memory in industrial manufacturing and its potential applications

Embedded memory,which heavily relies on the manufacturing process,has been widely adopted in various industrial applications.As the field of embedded memory continues to evolve,innovative strategies are emerging to enhance performance.Among them,resistive random access memory(RRAM)has gained significant attention due to its numerousadvantages over traditional memory devices,including high speed(<1 ns),high density(4 F^(2)·n^(-1)),high scalability(~nm),and low power consumption(~pJ).This review focuses on the recent progress of embedded RRAM in industrial manufacturing and its potentialapplications.It provides a brief introduction to the concepts and advantages of RRAM,discusses the key factors that impact its industrial manufacturing,and presents the commercial progress driven by cutting-edge nanotechnology,which has been pursued by manysemiconductor giants.Additionally,it highlights the adoption of embedded RRAM in emerging applications within the realm of the Internet of Things and future intelligent computing,with a particular emphasis on its role in neuromorphic computing.Finally,the review discusses thecurrent challenges and provides insights into the prospects of embedded RRAM in the era of big data and artificial intelligence.

Binary Program Vulnerability Mining Based on Neural Network

Software security analysts typically only have access to the executable program and cannot directly access the source code of the program.This poses significant challenges to security analysis.While it is crucial to identify vulnerabilities in such non-source code programs,there exists a limited set of generalized tools due to the low versatility of current vulnerability mining methods.However,these tools suffer from some shortcomings.In terms of targeted fuzzing,the path searching for target points is not streamlined enough,and the completely random testing leads to an excessively large search space.Additionally,when it comes to code similarity analysis,there are issues with incomplete code feature extraction,which may result in information loss.In this paper,we propose a cross-platform and cross-architecture approach to exploit vulnerabilities using neural network obfuscation techniques.By leveraging the Angr framework,a deobfuscation technique is introduced,along with the adoption of a VEX-IR-based intermediate language conversion method.This combination allows for the unified handling of binary programs across various architectures,compilers,and compilation options.Subsequently,binary programs are processed to extract multi-level spatial features using a combination of a skip-gram model with self-attention mechanism and a bidirectional Long Short-Term Memory(LSTM)network.Finally,the graph embedding network is utilized to evaluate the similarity of program functionalities.Based on these similarity scores,a target function is determined,and symbolic execution is applied to solve the target function.The solved content serves as the initial seed for targeted fuzzing.The binary program is processed by using the de-obfuscation technique and intermediate language transformation method,and then the similarity of program functions is evaluated by using a graph embedding network,and symbolic execution is performed based on these similarity scores.This approach facilitates cross-architecture analysis of executable programs without their source codes and concurrently reduces the risk of symbolic execution path explosion.

Identification of partial differential equations from noisy data with integrated knowledge discovery and embedding using evolutionary neural networks

Identification of underlying partial differential equations(PDEs)for complex systems remains a formidable challenge.In the present study,a robust PDE identification method is proposed,demonstrating the ability to extract accurate governing equations under noisy conditions without prior knowledge.Specifically,the proposed method combines gene expression programming,one type of evolutionary algorithm capable of generating unseen terms based solely on basic operators and functional terms,with symbolic regression neural networks.These networks are designed to represent explicit functional expressions and optimize them with data gradients.In particular,the specifically designed neural networks can be easily transformed to physical constraints for the training data,embedding the discovered PDEs to further optimize the metadata used for iterative PDE identification.The proposed method has been tested in four canonical PDE cases,validating its effectiveness without preliminary information and confirming its suitability for practical applications across various noise levels.

Ice-Induced Vibrational Response of Single-Pile Offshore Wind-Turbine Foundations

Important challenges must be addressed to make wind turbines sustainable renewable energy sources.A typical problem concerns the design of the foundation.If the pile diameter is larger than that of the jacket platform,traditional mechanical models cannot be used.In this study,relying on the seabed soil data of an offshore wind farm,the m-method and the equivalent embedded method are used to address the single-pile wind turbine foundation problem for different pile diameters.An approach to determine the equivalent pile length is also proposed accordingly.The results provide evidence for the effectiveness and reliability of the model based on the equivalent embedded method.

EG-STC: An Efficient Secure Two-Party Computation Scheme Based on Embedded GPU for Artificial Intelligence Systems

This paper presents a comprehensive exploration into the integration of Internet of Things(IoT),big data analysis,cloud computing,and Artificial Intelligence(AI),which has led to an unprecedented era of connectivity.We delve into the emerging trend of machine learning on embedded devices,enabling tasks in resource-limited environ-ments.However,the widespread adoption of machine learning raises significant privacy concerns,necessitating the development of privacy-preserving techniques.One such technique,secure multi-party computation(MPC),allows collaborative computations without exposing private inputs.Despite its potential,complex protocols and communication interactions hinder performance,especially on resource-constrained devices.Efforts to enhance efficiency have been made,but scalability remains a challenge.Given the success of GPUs in deep learning,lever-aging embedded GPUs,such as those offered by NVIDIA,emerges as a promising solution.Therefore,we propose an Embedded GPU-based Secure Two-party Computation(EG-STC)framework for Artificial Intelligence(AI)systems.To the best of our knowledge,this work represents the first endeavor to fully implement machine learning model training based on secure two-party computing on the Embedded GPU platform.Our experimental results demonstrate the effectiveness of EG-STC.On an embedded GPU with a power draw of 5 W,our implementation achieved a secure two-party matrix multiplication throughput of 5881.5 kilo-operations per millisecond(kops/ms),with an energy efficiency ratio of 1176.3 kops/ms/W.Furthermore,leveraging our EG-STC framework,we achieved an overall time acceleration ratio of 5–6 times compared to solutions running on server-grade CPUs.Our solution also exhibited a reduced runtime,requiring only 60%to 70%of the runtime of previously best-known methods on the same platform.In summary,our research contributes to the advancement of secure and efficient machine learning implementations on resource-constrained embedded devices,paving the way for broader adoption of AI technologies in various applications.

A chaotic hierarchical encryption/watermark embedding scheme for multi-medical images based on row-column confusion and closed-loop bi-directional diffusion

Security during remote transmission has been an important concern for researchers in recent years.In this paper,a hierarchical encryption multi-image encryption scheme for people with different security levels is designed,and a multiimage encryption(MIE)algorithm with row and column confusion and closed-loop bi-directional diffusion is adopted in the paper.While ensuring secure communication of medical image information,people with different security levels have different levels of decryption keys,and differentiated visual effects can be obtained by using the strong sensitivity of chaotic keys.The highest security level can obtain decrypted images without watermarks,and at the same time,patient information and copyright attribution can be verified by obtaining watermark images.The experimental results show that the scheme is sufficiently secure as an MIE scheme with visualized differences and the encryption and decryption efficiency is significantly improved compared to other works.

CL2ES-KDBC:A Novel Covariance Embedded Selection Based on Kernel Distributed Bayes Classifier for Detection of Cyber-Attacks in IoT Systems

The Internet of Things(IoT)is a growing technology that allows the sharing of data with other devices across wireless networks.Specifically,IoT systems are vulnerable to cyberattacks due to its opennes The proposed work intends to implement a new security framework for detecting the most specific and harmful intrusions in IoT networks.In this framework,a Covariance Linear Learning Embedding Selection(CL2ES)methodology is used at first to extract the features highly associated with the IoT intrusions.Then,the Kernel Distributed Bayes Classifier(KDBC)is created to forecast attacks based on the probability distribution value precisely.In addition,a unique Mongolian Gazellas Optimization(MGO)algorithm is used to optimize the weight value for the learning of the classifier.The effectiveness of the proposed CL2ES-KDBC framework has been assessed using several IoT cyber-attack datasets,The obtained results are then compared with current classification methods regarding accuracy(97%),precision(96.5%),and other factors.Computational analysis of the CL2ES-KDBC system on IoT intrusion datasets is performed,which provides valuable insight into its performance,efficiency,and suitability for securing IoT networks.

Functional Optical Fiber Sensors Detecting Imperceptible Physical/Chemical Changes for Smart Batteries

The battery technology progress has been a contradictory process in which performance improvement and hidden risks coexist.Now the battery is still a“black box”,thus requiring a deep understanding of its internal state.The battery should“sense its internal physical/chemical conditions”,which puts strict requirements on embedded sensing parts.This paper summarizes the application of advanced optical fiber sensors in lithium-ion batteries and energy storage technologies that may be mass deployed,focuses on the insights of advanced optical fiber sensors into the processes of one-dimensional nano-micro-level battery material structural phase transition,electrolyte degradation,electrode-electrolyte interface dynamics to three-dimensional macro-safety evolution.The paper contributes to understanding how to use optical fiber sensors to achieve“real”and“embedded”monitoring.Through the inherent advantages of the advanced optical fiber sensor,it helps clarify the battery internal state and reaction mechanism,aiding in the establishment of more detailed models.These advancements can promote the development of smart batteries,with significant importance lying in essentially promoting the improvement of system consistency.Furthermore,with the help of smart batteries in the future,the importance of consistency can be weakened or even eliminated.The application of advanced optical fiber sensors helps comprehensively improve the battery quality,reliability,and life.

NeurstrucEnergy:A bi-directional GNN model for energy prediction of neural networks in IoT

A significant demand rises for energy-efficient deep neural networks to support power-limited embedding devices with successful deep learning applications in IoT and edge computing fields.An accurate energy prediction approach is critical to provide measurement and lead optimization direction.However,the current energy prediction approaches lack accuracy and generalization ability due to the lack of research on the neural network structure and the excessive reliance on customized training dataset.This paper presents a novel energy prediction model,NeurstrucEnergy.NeurstrucEnergy treats neural networks as directed graphs and applies a bi-directional graph neural network training on a randomly generated dataset to extract structural features for energy prediction.NeurstrucEnergy has advantages over linear approaches because the bi-directional graph neural network collects structural features from each layer's parents and children.Experimental results show that NeurstrucEnergy establishes state-of-the-art results with mean absolute percentage error of 2.60%.We also evaluate NeurstrucEnergy in a randomly generated dataset,achieving the mean absolute percentage error of 4.83%over 10 typical convolutional neural networks in recent years and 7 efficient convolutional neural networks created by neural architecture search.Our code is available at https://github.com/NEUSoftGreenAI/NeurstrucEnergy.git.

TCM-HIN2Vec:A strategy for uncovering biological basis of heart qi deficiency pattern based on network embedding and transcriptomic experiment

Objective:To elucidate the biological basis of the heart qi deficiency(HQD)pattern,an in-depth understanding of which is essential for improving clinical herbal therapy.Methods: We predicted and characterized HQD pattern genes using the new strategy,TCM-HIN2Vec,which involves heterogeneous network embedding and transcriptomic experiments.First,a heterogeneous network of traditional Chinese medicine(TCM)patterns was constructed using public databases.Next,we predicted HQD pattern genes using a heterogeneous network-embedding algorithm.We then analyzed the functional characteristics of HQD pattern genes using gene enrichment analysis and examined gene expression levels using RNA-seq.Finally,we identified TCM herbs that demonstrated enriched interactions with HQD pattern genes via herbal enrichment analysis.Results: Our TCM-HIN2Vec strategy revealed that candidate genes associated with HQD pattern were significantly enriched in energy metabolism,signal transduction pathways,and immune processes.Moreover,we found that these candidate genes were significantly differentially expressed in the transcriptional profile of mice model with heart failure with a qi deficiency pattern.Furthermore,herbal enrichment analysis identified TCM herbs that demonstrated enriched interactions with the top 10 candidate genes and could potentially serve as drug candidates for treating HQD.Conclusion: Our results suggested that TCM-HIN2Vec is capable of not only accurately identifying HQD pattern genes,but also deciphering the basis of HQD pattern.Furthermore our finding indicated that TCM-HIN2Vec may be further expanded to develop other patterns,leading to a new approach aimed at elucidating general TCM patterns and developing precision medicine.

High-Precision Flow Numerical Simulation and Productivity Evaluation of Shale Oil Considering Stress Sensitivity

Continental shale oil reservoirs,characterized by numerous bedding planes and micro-nano scale pores,feature significantly higher stress sensitivity compared to other types of reservoirs.However,research on suitable stress sensitivity characterization models is still limited.In this study,three commonly used stress sensitivity models for shale oil reservoirs were considered,and experiments on representative core samples were conducted.By fitting and comparing the data,the“exponential model”was identified as a characterization model that accurately represents stress sensitivity in continental shale oil reservoirs.To validate the accuracy of the model,a two-phase seepage mathematical model for shale oil reservoirs coupled with the exponential model was introduced.The model was discretely solved using the finite volume method,and its accuracy was verified through the commercial simulator CMG.The study evaluated the productivity of a typical horizontal well under different engineering,geological,and fracture conditions.The results indicate that considering stress sensitivity leads to a 13.57%reduction in production for the same matrix permeability.Additionally,as the fracture half-length and the number of fractures increase,and the bottomhole flowing pressure decreases,the reservoir stress sensitivity becomes higher.