Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Attention mechanism based multi-scale feature extraction of bearing fault diagnosis

Effective bearing fault diagnosis is vital for the safe and reliable operation of rotating machinery.In practical applications,bearings often work at various rotational speeds as well as load conditions.Yet,the bearing fault diagnosis under multiple conditions is a new subject,which needs to be further explored.Therefore,a multi-scale deep belief network(DBN)method integrated with attention mechanism is proposed for the purpose of extracting the multi-scale core features from vibration signals,containing four primary steps:preprocessing of multi-scale data,feature extraction,feature fusion,and fault classification.The key novelties include multi-scale feature extraction using multi-scale DBN algorithm,and feature fusion using attention mecha-nism.The benchmark dataset from University of Ottawa is applied to validate the effectiveness as well as advantages of this method.Furthermore,the aforementioned method is compared with four classical fault diagnosis methods reported in the literature,and the comparison results show that our pro-posed method has higher diagnostic accuracy and better robustness.

Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Transfer learning framework for multi-scale crack type classification with sparse microseismic networks

Rock fracture mechanisms can be inferred from moment tensors(MT)inverted from microseismic events.However,MT can only be inverted for events whose waveforms are acquired across a network of sensors.This is limiting for underground mines where the microseismic stations often lack azimuthal coverage.Thus,there is a need for a method to invert fracture mechanisms using waveforms acquired by a sparse microseismic network.Here,we present a novel,multi-scale framework to classify whether a rock crack contracts or dilates based on a single waveform.The framework consists of a deep learning model that is initially trained on 2400000+manually labelled field-scale seismic and microseismic waveforms acquired across 692 stations.Transfer learning is then applied to fine-tune the model on 300000+MT-labelled labscale acoustic emission waveforms from 39 individual experiments instrumented with different sensor layouts,loading,and rock types in training.The optimal model achieves over 86%F-score on unseen waveforms at both the lab-and field-scale.This model outperforms existing empirical methods in classification of rock fracture mechanisms monitored by a sparse microseismic network.This facilitates rapid assessment of,and early warning against,various rock engineering hazard such as induced earthquakes and rock bursts.

YOLO-MFD:Remote Sensing Image Object Detection with Multi-Scale Fusion Dynamic Head

Remote sensing imagery,due to its high altitude,presents inherent challenges characterized by multiple scales,limited target areas,and intricate backgrounds.These inherent traits often lead to increased miss and false detection rates when applying object recognition algorithms tailored for remote sensing imagery.Additionally,these complexities contribute to inaccuracies in target localization and hinder precise target categorization.This paper addresses these challenges by proposing a solution:The YOLO-MFD model(YOLO-MFD:Remote Sensing Image Object Detection withMulti-scale Fusion Dynamic Head).Before presenting our method,we delve into the prevalent issues faced in remote sensing imagery analysis.Specifically,we emphasize the struggles of existing object recognition algorithms in comprehensively capturing critical image features amidst varying scales and complex backgrounds.To resolve these issues,we introduce a novel approach.First,we propose the implementation of a lightweight multi-scale module called CEF.This module significantly improves the model’s ability to comprehensively capture important image features by merging multi-scale feature information.It effectively addresses the issues of missed detection and mistaken alarms that are common in remote sensing imagery.Second,an additional layer of small target detection heads is added,and a residual link is established with the higher-level feature extraction module in the backbone section.This allows the model to incorporate shallower information,significantly improving the accuracy of target localization in remotely sensed images.Finally,a dynamic head attentionmechanism is introduced.This allows themodel to exhibit greater flexibility and accuracy in recognizing shapes and targets of different sizes.Consequently,the precision of object detection is significantly improved.The trial results show that the YOLO-MFD model shows improvements of 6.3%,3.5%,and 2.5%over the original YOLOv8 model in Precision,map@0.5 and map@0.5:0.95,separately.These results illustrate the clear advantages of the method.

Multi-Scale Mixed Attention Tea Shoot Instance Segmentation Model

Tea leaf picking is a crucial stage in tea production that directly influences the quality and value of the tea.Traditional tea-picking machines may compromise the quality of the tea leaves.High-quality teas are often handpicked and need more delicate operations in intelligent picking machines.Compared with traditional image processing techniques,deep learning models have stronger feature extraction capabilities,and better generalization and are more suitable for practical tea shoot harvesting.However,current research mostly focuses on shoot detection and cannot directly accomplish end-to-end shoot segmentation tasks.We propose a tea shoot instance segmentation model based on multi-scale mixed attention(Mask2FusionNet)using a dataset from the tea garden in Hangzhou.We further analyzed the characteristics of the tea shoot dataset,where the proportion of small to medium-sized targets is 89.9%.Our algorithm is compared with several mainstream object segmentation algorithms,and the results demonstrate that our model achieves an accuracy of 82%in recognizing the tea shoots,showing a better performance compared to other models.Through ablation experiments,we found that ResNet50,PointRend strategy,and the Feature Pyramid Network(FPN)architecture can improve performance by 1.6%,1.4%,and 2.4%,respectively.These experiments demonstrated that our proposed multi-scale and point selection strategy optimizes the feature extraction capability for overlapping small targets.The results indicate that the proposed Mask2FusionNet model can perform the shoot segmentation in unstructured environments,realizing the individual distinction of tea shoots,and complete extraction of the shoot edge contours with a segmentation accuracy of 82.0%.The research results can provide algorithmic support for the segmentation and intelligent harvesting of premium tea shoots at different scales.

Keyphrase Generation Based on Self-Attention Mechanism

Keyphrase greatly provides summarized and valuable information.This information can help us not only understand text semantics,but also organize and retrieve text content effectively.The task of automatically generating it has received considerable attention in recent decades.From the previous studies,we can see many workable solutions for obtaining keyphrases.One method is to divide the content to be summarized into multiple blocks of text,then we rank and select the most important content.The disadvantage of this method is that it cannot identify keyphrase that does not include in the text,let alone get the real semantic meaning hidden in the text.Another approach uses recurrent neural networks to generate keyphrases from the semantic aspects of the text,but the inherently sequential nature precludes parallelization within training examples,and distances have limitations on context dependencies.Previous works have demonstrated the benefits of the self-attention mechanism,which can learn global text dependency features and can be parallelized.Inspired by the above observation,we propose a keyphrase generation model,which is based entirely on the self-attention mechanism.It is an encoder-decoder model that can make up the above disadvantage effectively.In addition,we also consider the semantic similarity between keyphrases,and add semantic similarity processing module into the model.This proposed model,which is demonstrated by empirical analysis on five datasets,can achieve competitive performance compared to baseline methods.

Surrogate-based modeling and dimension reduction techniques for multi-scale mechanics problems

Successful modeling and/or design of engineering systems often requires one to address the impact of multiple ＂design variables＂ on the prescribed outcome.There are often multiple,competing objectives based on which we assess the outcome of optimization.Since accurate,high fidelity models are typically time consuming and computationally expensive,comprehensive evaluations can be conducted only if an efficient framework is available.Furthermore,informed decisions of the model/hardware＇s overall performance rely on an adequate understanding of the global,not local,sensitivity of the individual design variables on the objectives.The surrogate-based approach,which involves approximating the objectives as continuous functions of design variables from limited data,offers a rational framework to reduce the number of important input variables,i.e.,the dimension of a design or modeling space.In this paper,we review the fundamental issues that arise in surrogate-based analysis and optimization,highlighting concepts,methods,techniques,as well as modeling implications for mechanics problems.To aid the discussions of the issues involved,we summarize recent efforts in investigating cryogenic cavitating flows,active flow control based on dielectric barrier discharge concepts,and lithium（Li）-ion batteries.It is also stressed that many multi-scale mechanics problems can naturally benefit from the surrogate approach for ＂scale bridging.＂

Mechanical properties of multi-scale germanium specimens from space solar cells under electron irradiation

During long-term service in space,Gallium Arsenide(GaAs)solar cells are directly exposed to electron irradiation which usually causes a dramatic decrease in their performance.In the multilayer structure of solar cells,the germanium(Ge)layer occupies the majority of the thickness as the substrate.Due to the intrinsic brittleness of semiconductor material,there exist various defects during the preparation and assembly of solar cells,the influences of which tend to be intensified by the irradiation effect.In this work,first,Ge specimens for mechanical tests were prepared at scales from microscopic to macroscopic.Then,after different doses of electron irradiation,the mechanical properties of the Ge specimens were investigated.The experimental results demonstrate that electron irradiation has an obvious effect on the mechanical property variation of Ge in diverse scales.The four-point bending test indicates that the elastic modulus,fracture strength,and maximum displacement of the Ge specimens all increase,and reach the maximum value at the irradiation dose of 1×10^(15)e/cm^(2).The micrometer scale cantilever and nanoindentation tests present similar trends for Ge specimens after irradiation.Atomic Force Microscope(AFM)also observed the change in surface roughness.Finally,a fitting model was established to characterize the relation between modulus change and electron irradiation dose.

Numerical failure analysis of a continuous reinforced concrete bridge under strong earthquakes using multi-scale models

Previous failure analyses of bridges typically focus on substructure failure or superstructure failure separately. However, in an actual bridge, the seismic induced substructure failure and superstructure failure may influence each other. Moreover, previous studies typically use simplified models to analyze the bridge failure; however, there are inherent defects in the calculation accuracy compared with using a detailed three-dimensional （3D） finite element （FE） model. Conversely, a detailed 3D FE model requires more computational costs, and a proper erosion criterion of the 3D elements is necessary. In this paper, a multi-scale FE model, including a corresponding erosion criterion, is proposed and validated that can significantly reduce computational costs with high precision by modelling a pseudo-dynamic test of an reinforced concrete （RC） pier. Numerical simulations of the seismic failures of a continuous RC bridge based on the multi-scale FE modeling method using LS-DYNA are performed. The nonlinear properties of the bridge, various connection strengths and bidirectional excitations are considered. The numerical results demonstrate that the failure of the connections will induce large pounding responses of the girders. The nonlinear deformation of the piers will aggravate the pounding damages. Furthermore, bidirectional earthquakes will induce eccentric poundingsto the girders and different failure modes to the adjacent piers.

Multi-Scale Superhydrophobic Anti-Icing Coating for Wind Turbine Blades

As a surface functional material,super-hydrophobic coating has great application potential in wind turbine blade anti-icing,self-cleaning and drag reduction.In this study,ZnO and SiO2 multi-scale superhydrophobic coatings with mechanical flexibility were prepared by embedding modified ZnO and SiO2 nanoparticles in PDMS.The prepared coating has a higher static water contact angle(CA is 153°)and a lower rolling angle(SA is 3.3°),showing excellent super-hydrophobicity.Because of its excellent superhydrophobic ability and micro-nano structure,the coating has good anti-icing ability.Under the conditions of−10
C and 60%relative humidity,the coating can delay the freezing time by 1511S,which is 10.7 times slower than the normal freezing time.More importantly,due to the mechanical properties provided by SiO2 and the synergistic effect of micro-nano particles,the coating has excellent mechanical durability.After 10 wear tests,the contact angle of the coating is still as high as 141°and the rolling angle is 6.8°.This research provides a theoretical reference for the preparation of a mechanically stable coating with a simple preparation process,as well as a basic research on the anti-icing behavior of the coating.

MULTI-SCALE AND MULTI-PHASE NANOCOMPOSITE CERAMIC TOOLS AND CUTTING PERFORMANCE

An advanced ceramic cutting tool material Al2O3/TiC/TiN （LTN） is developed by incorporation and dispersion of micro-scale TiC particle and nano-scale TiN particle in alumina matrix. With the optimal dispersing and fabricating technology, this multi-scale and multi-phase nanocomposite ceramic tool material can get both higher flexural strength and fracture toughness than that of A1203/TiC （LZ） ceramic tool material without nano-scale TiN particle, especially the fracture toughness can reach to 7.8 MPa . m^0.5. The nano-scale TiN can lead to the grain fining effect and promote the sintering process to get a higher density. The coexisting transgranular and intergranular fracture mode induced by micro-scale TiC and nano-scale TiN, and the homogeneous and densified microstructure can result in a remarkable strengthening and toughening effect. The cutting performance and wear mechanisms of the advanced multi-scale and multi-phase nanocomposite ceramic cutting tool are researched.

A multi-scale grained microstructure of the surface nanocrystallized 304 stainless steel sheets after warm-rolling

An ultrafine grained microstructure was obtained for 304 stainless steel(304SS)sheets by using surface nanocrystallization and warm-rolling.The microstructure and mechanical properties were determined by X-ray diffraction(XRD),transmission electron microscope(TEM)and a test on microhardness.Experimental results were shown that the microstructure was featured by a continuous distribution from the nanocrystalline on the surface to micro-grains in the center,in which the volume fraction of the micro-sized grains is about 40% in the surface layer.This multi-scale grained microstructure was composed of austenite and martensite phases with a gradient increasing volume fraction of austenite from the surface to the centre.The microhardness of the resultant steel was higher than 150% of that as received,due to the refined grains and strain-induced martensitic transformation.The hardness distribution was consistent with the microstructural variation,suggesting a good combination of high strength and improved ductility.

NFHP-RN:AMethod of Few-Shot Network Attack Detection Based on the Network Flow Holographic Picture-ResNet

Due to the rapid evolution of Advanced Persistent Threats(APTs)attacks,the emergence of new and rare attack samples,and even those never seen before,make it challenging for traditional rule-based detection methods to extract universal rules for effective detection.With the progress in techniques such as transfer learning and meta-learning,few-shot network attack detection has progressed.However,challenges in few-shot network attack detection arise from the inability of time sequence flow features to adapt to the fixed length input requirement of deep learning,difficulties in capturing rich information from original flow in the case of insufficient samples,and the challenge of high-level abstract representation.To address these challenges,a few-shot network attack detection based on NFHP(Network Flow Holographic Picture)-RN(ResNet)is proposed.Specifically,leveraging inherent properties of images such as translation invariance,rotation invariance,scale invariance,and illumination invariance,network attack traffic features and contextual relationships are intuitively represented in NFHP.In addition,an improved RN network model is employed for high-level abstract feature extraction,ensuring that the extracted high-level abstract features maintain the detailed characteristics of the original traffic behavior,regardless of changes in background traffic.Finally,a meta-learning model based on the self-attention mechanism is constructed,achieving the detection of novel APT few-shot network attacks through the empirical generalization of high-level abstract feature representations of known-class network attack behaviors.Experimental results demonstrate that the proposed method can learn high-level abstract features of network attacks across different traffic detail granularities.Comparedwith state-of-the-artmethods,it achieves favorable accuracy,precision,recall,and F1 scores for the identification of unknown-class network attacks through cross-validation onmultiple datasets.

A New Industrial Intrusion Detection Method Based on CNN-BiLSTM

Nowadays,with the rapid development of industrial Internet technology,on the one hand,advanced industrial control systems(ICS)have improved industrial production efficiency.However,there are more and more cyber-attacks targeting industrial control systems.To ensure the security of industrial networks,intrusion detection systems have been widely used in industrial control systems,and deep neural networks have always been an effective method for identifying cyber attacks.Current intrusion detection methods still suffer from low accuracy and a high false alarm rate.Therefore,it is important to build a more efficient intrusion detection model.This paper proposes a hybrid deep learning intrusion detection method based on convolutional neural networks and bidirectional long short-term memory neural networks(CNN-BiLSTM).To address the issue of imbalanced data within the dataset and improve the model’s detection capabilities,the Synthetic Minority Over-sampling Technique-Edited Nearest Neighbors(SMOTE-ENN)algorithm is applied in the preprocessing phase.This algorithm is employed to generate synthetic instances for the minority class,simultaneously mitigating the impact of noise in the majority class.This approach aims to create a more equitable distribution of classes,thereby enhancing the model’s ability to effectively identify patterns in both minority and majority classes.In the experimental phase,the detection performance of the method is verified using two data sets.Experimental results show that the accuracy rate on the CICIDS-2017 data set reaches 97.7%.On the natural gas pipeline dataset collected by Lan Turnipseed from Mississippi State University in the United States,the accuracy rate also reaches 85.5%.

Intelligent Fault Diagnosis Method of Rolling Bearings Based on Transfer Residual Swin Transformer with Shifted Windows

Due to their robust learning and expression ability for complex features,the deep learning(DL)model plays a vital role in bearing fault diagnosis.However,since there are fewer labeled samples in fault diagnosis,the depth of DL models in fault diagnosis is generally shallower than that of DL models in other fields,which limits the diagnostic performance.To solve this problem,a novel transfer residual Swin Transformer(RST)is proposed for rolling bearings in this paper.RST has 24 residual self-attention layers,which use the hierarchical design and the shifted window-based residual self-attention.Combined with transfer learning techniques,the transfer RST model uses pre-trained parameters from ImageNet.A new end-to-end method for fault diagnosis based on deep transfer RST is proposed.Firstly,wavelet transform transforms the vibration signal into a wavelet time-frequency diagram.The signal’s time-frequency domain representation can be represented simultaneously.Secondly,the wavelet time-frequency diagram is the input of the RST model to obtain the fault type.Finally,our method is verified on public and self-built datasets.Experimental results show the superior performance of our method by comparing it with a shallow neural network.

An Affective EEG Analysis Method Without Feature Engineering

Emotional electroencephalography(EEG)signals are a primary means of recording emotional brain activity.Currently,the most effective methods for analyzing emotional EEG signals involve feature engineering and neural networks.However,neural networks possess a strong ability for automatic feature extraction.Is it possible to discard feature engineering and directly employ neural networks for end-to-end recognition?Based on the characteristics of EEG signals,this paper proposes an end-to-end feature extraction and classification method for a dynamic self-attention network(DySAT).The study reveals significant differences in brain activity patterns associated with different emotions across various experimenters and time periods.The results of this experiment can provide insights into the reasons behind these differences.

Trans-scale mechanics: looking for the missing links between continuum and micro/nanoscopic reality

Problems involving coupled multiple space and time scales offer a real challenge for conventional frame-works of either particle or continuum mechanics. In this paper, four cases studies （shear band formation in bulk metallic glasses, spallation resulting from stress wave, interaction between a probe tip and sample, the simulation of nanoindentation with molecular statistical thermodynamics） are provided to illustrate the three levels of trans-scale problems （problems due to various physical mechanisms at macro-level, problems due to micro-structural evolution at macro/micro-level, problems due to the coupling of atoms/ molecules and a finite size body at micro/nano-level） and their formulations. Accordingly, non-equilibrium statistical mechanics, coupled trans-scale equations and simultaneous solutions, and trans-scale algorithms based on atomic/molecular interaction are suggested as the three possible modes of trans-scale mechanics.

Multi-scale crystal plasticity finite element simulations of the microstructural evolution and formation mechanism of adiabatic shear bands in dual-phase Ti20C alloy under complex dynamic loading

A dynamic compression test was performed on α+β dual-phase titanium alloy Ti20C using a split Hopkinson pressure bar.The formation of adiabatic shear bands generated during the compression process was studied by combining the proposed multi-scale crystal plasticity finite element method with experimental measurements.The complex local micro region load was progressively extracted from the simulation results of a macro model and applied to an established three-dimensional multi-grain microstructure model.Subsequently,the evolution histories of the grain shape,size,and orientation inside the adiabatic shear band were quantitatively simulated.The results corresponded closely to the experimental results obtained via transmission electron microscopy and precession electron diffraction.Furthermore,by calculating the grain rotation and temperature rise inside the adiabatic shear band,the microstructural softening and thermal softening effects of typical heavily-deformed α grains were successfully decoupled.The results revealed that the microstructural softening stress was triggered and then stabilized(in general)at a relatively high value.This indicated that the mechanical strength was lowered mainly by the grain orientation evolution or dynamic recrystallization occurring during early plastic deformation.Subsequently,thermal softening increased linearly and became the main softening mechanism.Noticeably,in the final stage,the thermal softening stress accounted for 78.4% of the total softening stress due to the sharp temperature increase,which inevitably leads to the stress collapse and potential failure of the alloy.

NANO SCIENCE AND ENGINEERING IN SOLID MECHANICS

According to National Science Foundation （NSF） Director A. Bement, ‘Transformative research is... research driven by ideas that stand a reasonable chance of radically changing our understanding of an important existing scientific concept or leading to the creation of a new paradigm or field of science is also characterized by its challenge to current understanding or its pathway to new frontiers.＇ Nanotechnology is one of such frontiers. It is the creation of new materials, devices and systems at the molecular level--phenomena associated with atomic and molecular interactions strongly influence macroscopic material properties with significantly improved mechanical, optical, chemical, electrical... properties. Former NSF Director Rita Colwell in 2002 declared, ‘nanoscale technology will have an impact equal to the Industrial Revolution＇. The transcendent technologies include nanotechnology, microelectronics, information technology and biotechnology as well as the enabling and supporting mechanical and civil infrastructure systems and materials. These technologies are the primary drivers of the twenty first century and the new economy. Mechanics is an essential eleraent in all of the transcendent technologies. Research opportunities, education and challenges in mechanics, including experimental, numerical and analytical methods in nanomechanics, carbon nano-tubes, bio-inspired materials, fuel cells, as well as improved engineering and design of materials are presented and discussed in this paper.