MSADCN:Multi-Scale Attentional Densely Connected Network for Automated Bone Age Assessment

Bone age assessment(BAA)helps doctors determine how a child’s bones grow and develop in clinical medicine.Traditional BAA methods rely on clinician expertise,leading to time-consuming predictions and inaccurate results.Most deep learning-based BAA methods feed the extracted critical points of images into the network by providing additional annotations.This operation is costly and subjective.To address these problems,we propose a multi-scale attentional densely connected network(MSADCN)in this paper.MSADCN constructs a multi-scale dense connectivity mechanism,which can avoid overfitting,obtain the local features effectively and prevent gradient vanishing even in limited training data.First,MSADCN designs multi-scale structures in the densely connected network to extract fine-grained features at different scales.Then,coordinate attention is embedded to focus on critical features and automatically locate the regions of interest(ROI)without additional annotation.In addition,to improve the model’s generalization,transfer learning is applied to train the proposed MSADCN on the public dataset IMDB-WIKI,and the obtained pre-trained weights are loaded onto the Radiological Society of North America(RSNA)dataset.Finally,label distribution learning(LDL)and expectation regression techniques are introduced into our model to exploit the correlation between hand bone images of different ages,which can obtain stable age estimates.Extensive experiments confirm that our model can converge more efficiently and obtain a mean absolute error(MAE)of 4.64 months,outperforming some state-of-the-art BAA methods.

Artificial neural network-based subgrid-scale models for LES of compressible turbulent channel flow

Fully connected neural networks(FCNNs)have been developed for the closure of subgrid-scale(SGS)stress and SGS heat flux in large-eddy simulations of compressible turbulent channel flow.The FCNNbased SGS model trained using data with Mach number Ma=3.0 and Reynolds number Re=3000 was applied to situations with different Mach numbers and Reynolds numbers.The input variables of the neural network model were the filtered velocity gradients and temperature gradients at a single spatial grid point.The a priori test showed that the FCNN model had a correlation coefficient larger than 0.91 and a relative error smaller than 0.43,with much better reconstructions of SGS unclosed terms than the dynamic Smagorinsky model(DSM).In a posteriori test,the behavior of the FCNN model was marginally better than that of the DSM in predicting the mean velocity profiles,mean temperature profiles,turbulent intensities,total Reynolds stress,total Reynolds heat flux,and mean SGS flux of kinetic energy,and outperformed the Smagorinsky model.

Attention-based neural network for end-to-end music separation

The end-to-end separation algorithm with superior performance in the field of speech separation has not been effectively used in music separation.Moreover,since music signals are often dual channel data with a high sampling rate,how to model longsequence data and make rational use of the relevant information between channels is also an urgent problem to be solved.In order to solve the above problems,the performance of the end-to-end music separation algorithm is enhanced by improving the network structure.Our main contributions include the following:(1)A more reasonable densely connected U-Net is designed to capture the long-term characteristics of music,such as main melody,tone and so on.(2)On this basis,the multi-head attention and dualpath transformer are introduced in the separation module.Channel attention units are applied recursively on the feature map of each layer of the network,enabling the network to perform long-sequence separation.Experimental results show that after the introduction of the channel attention,the performance of the proposed algorithm has a stable improvement compared with the baseline system.On the MUSDB18 dataset,the average score of the separated audio exceeds that of the current best-performing music separation algorithm based on the time-frequency domain(T-F domain).

A NewMalicious Code ClassificationMethod for the Security of Financial Software

The field of finance heavily relies on cybersecurity to safeguard its systems and clients from harmful software.The identification of malevolent code within financial software is vital for protecting both the financial system and individual clients.Nevertheless,present detection models encounter limitations in their ability to identify malevolent code and its variations,all while encompassing a multitude of parameters.To overcome these obsta-cles,we introduce a lean model for classifying families of malevolent code,formulated on Ghost-DenseNet-SE.This model integrates the Ghost module,DenseNet,and the squeeze-and-excitation(SE)channel domain attention mechanism.It substitutes the standard convolutional layer in DenseNet with the Ghost module,thereby diminishing the model’s size and augmenting recognition speed.Additionally,the channel domain attention mechanism assigns distinctive weights to feature channels,facilitating the extraction of pivotal characteristics of malevolent code and bolstering detection precision.Experimental outcomes on the Malimg dataset indicate that the model attained an accuracy of 99.14%in discerning families of malevolent code,surpassing AlexNet(97.8%)and The visual geometry group network(VGGNet)(96.16%).The proposed model exhibits reduced parameters,leading to decreased model complexity alongside enhanced classification accuracy,rendering it a valuable asset for categorizing malevolent code.

Interpretation and characterization of rate of penetration intelligent prediction model

Accurate prediction of the rate of penetration(ROP)is significant for drilling optimization.While the intelligent ROP prediction model based on fully connected neural networks(FNN)outperforms traditional ROP equations and machine learning algorithms,its lack of interpretability undermines its credibility.This study proposes a novel interpretation and characterization method for the FNN ROP prediction model using the Rectified Linear Unit(ReLU)activation function.By leveraging the derivative of the ReLU function,the FNN function calculation process is transformed into vector operations.The FNN model is linearly characterized through further simplification,enabling its interpretation and analysis.The proposed method is applied in ROP prediction scenarios using drilling data from three vertical wells in the Tarim Oilfield.The results demonstrate that the FNN ROP prediction model with ReLU as the activation function performs exceptionally well.The relative activation frequency curve of hidden layer neurons aids in analyzing the overfitting of the FNN ROP model and determining drilling data similarity.In the well sections with similar drilling data,averaging the weight parameters enables linear characterization of the FNN ROP prediction model,leading to the establishment of a corresponding linear representation equation.Furthermore,the quantitative analysis of each feature's influence on ROP facilitates the proposal of drilling parameter optimization schemes for the current well section.The established linear characterization equation exhibits high precision,strong stability,and adaptability through the application and validation across multiple well sections.

Fully Connected Feedforward Neural Networks Based CSI Feedback Algorithm

In modern wireless communication systems,the accurate acquisition of channel state information(CSI)is critical to the performance of beamforming,non-orthogonal multiple access(NOMA),etc.However,with the application of massive MIMO in 5G,the number of antennas increases by hundreds or even thousands times,which leads to excessive feedback overhead and poses a huge challenge to the conventional channel state information feedback scheme.In this paper,by using deep learning technology,we develop a system framework for CSI feedback based on fully connected feedforward neural networks(FCFNN),named CF-FCFNN.Through learning the training set composed of CSI,CF-FCFNN is able to recover the original CSI from the compressed CSI more accurately compared with the existing method based on deep learning without increasing the algorithm complexity.

Diagnosis of focal liver lesions with deep learning-based multichannel analysis of hepatocyte-specific contrast-enhanced magnetic resonance imaging

BACKGROUND The nature of input data is an essential factor when training neural networks.Research concerning magnetic resonance imaging(MRI)-based diagnosis of liver tumors using deep learning has been rapidly advancing.Still,evidence to support the utilization of multi-dimensional and multi-parametric image data is lacking.Due to higher information content,three-dimensional input should presumably result in higher classification precision.Also,the differentiation between focal liver lesions(FLLs)can only be plausible with simultaneous analysis of multisequence MRI images.AIM To compare diagnostic efficiency of two-dimensional(2D)and three-dimensional(3D)-densely connected convolutional neural networks(DenseNet)for FLLs on multi-sequence MRI.METHODS We retrospectively collected T2-weighted,gadoxetate disodium-enhanced arterial phase,portal venous phase,and hepatobiliary phase MRI scans from patients with focal nodular hyperplasia(FNH),hepatocellular carcinomas(HCC)or liver metastases(MET).Our search identified 71 FNH,69 HCC and 76 MET.After volume registration,the same three most representative axial slices from all sequences were combined into four-channel images to train the 2D-DenseNet264 network.Identical bounding boxes were selected on all scans and stacked into 4D volumes to train the 3D-DenseNet264 model.The test set consisted of 10-10-10 tumors.The performance of the models was compared using area under the receiver operating characteristic curve(AUROC),specificity,sensitivity,positive predictive values(PPV),negative predictive values(NPV),and f1 scores.RESULTS The average AUC value of the 2D model(0.98)was slightly higher than that of the 3D model(0.94).Mean PPV,sensitivity,NPV,specificity and f1 scores(0.94,0.93,0.97,0.97,and 0.93)of the 2D model were also superior to metrics of the 3D model(0.84,0.83,0.92,0.92,and 0.83).The classification metrics of FNH were 0.91,1.00,1.00,0.95,and 0.95 using the 2D and 0.90,0.90,0.95,0.95,and 0.90 using the 3D models.The 2D and 3D networks'performance in the diagnosis of HCC were 1.00,0.80,0.91,1.00,and 0.89 and 0.88,0.70,0.86,0.95,and 0.78,respectively;while the evaluation of MET lesions resulted in 0.91,1.00,1.00,0.95,and 0.95 and 0.75,0.90,0.94,0.85,and 0.82 using the 2D and 3D networks,respectively.CONCLUSION Both 2D and 3D-DenseNets can differentiate FNH,HCC and MET with good accuracy when trained on hepatocyte-specific contrast-enhanced multi-sequence MRI volumes.

Road Damage Detection and Classification Using Mask R-CNN with DenseNet Backbone

Automatic road damage detection using image processing is an important aspect of road maintenance.It is also a challenging problem due to the inhomogeneity of road damage and complicated background in the road images.In recent years,deep convolutional neural network based methods have been used to address the challenges of road damage detection and classification.In this paper,we propose a new approach to address those challenges.This approach uses densely connected convolution networks as the backbone of the Mask R-CNN to effectively extract image feature,a feature pyramid network for combining multiple scales features,a region proposal network to generate the road damage region,and a fully convolutional neural network to classify the road damage region and refine the region bounding box.This method can not only detect and classify the road damage,but also create a mask of the road damage.Experimental results show that the proposed approach can achieve better results compared with other existing methods.

Analyzing the Reliability of Degradable Networks

The reachability of a strongly connected network may be destroyed after link damage.Since many networds have directed links with the potential for reversal,the reachabilty may be restored by reversing the direction of links.In this paper,the reliability of a network that allows reversal of links is dis- cussed.

Diffusion logistic regression algorithms over multiagent networks

In this paper,a distributed scheme is proposed for ensemble learning method of bagging,which aims to address the classification problems for large dataset by developing a group of cooperative logistic regression learners in a connected network.Moveover,each weak learner/agent can share the local weight vector with its immediate neighbors through diffusion strategy in a fully distributed manner Our diffusion logistic regression algorithms can effectively avoid overfitting and obtain high classification accuracy compared to the non-cooperation mode.Furthermore,simulations with a real dataset are given to demonstrate the effectiveness of the proposed methods in comparison with the centralized one.

Microstructure Evolution and Strain Softening of Carbon Black Filled Natural Rubber Vulcanizates

Incorporation of carbon black(CB)in natural rubber(NR)enhances the Mullins effect and Payne effect of their vulcanizates,but the strain softening mechanisms and the microstructure evolution in the vulcanizates have not been clearly concluded so far.We investigate the Mullins effect and Payne effect of CB filled NR vulcanizates by using cyclic tensile tests at different temperatures and dynamic rheological measurements combined with simultaneous electric conduction.During cyclic stretching,the normalized recovery hysteresis energy and accumulative softening energy for NR/CB vulcanizates with different loadings can be both superimposed on a master curve,indicating that the Mullins effect is mainly dominated by the rubber matrix.The irreversible simultaneous resistance evolution also reveals that the structural evolution of nanoparticles(NPs)network is not directly related to the Mullins effect.Moreover,the extension of linear viscoelastic region and the hysteresis of Payne effect for filled vulcanizates subjected to cyclic stretching indicate the destruction of CB aggregated structure and the interfacial layers between CB and rubber chains during cyclic stretching.This investigation would be illuminating for the microstructure evolution and strain softening of rubber nanocomposites under harsh service conditions.

Applying Big Data Based Deep Learning System to Intrusion Detection

With vast amounts of data being generated daily and the ever increasing interconnectivity of the world’s internet infrastructures,a machine learning based Intrusion Detection Systems(IDS)has become a vital component to protect our economic and national security.Previous shallow learning and deep learning strategies adopt the single learning model approach for intrusion detection.The single learning model approach may experience problems to understand increasingly complicated data distribution of intrusion patterns.Particularly,the single deep learning model may not be effective to capture unique patterns from intrusive attacks having a small number of samples.In order to further enhance the performance of machine learning based IDS,we propose the Big Data based Hierarchical Deep Learning System(BDHDLS).BDHDLS utilizes behavioral features and content features to understand both network traffic characteristics and information stored in the payload.Each deep learning model in the BDHDLS concentrates its efforts to learn the unique data distribution in one cluster.This strategy can increase the detection rate of intrusive attacks as compared to the previous single learning model approaches.Based on parallel training strategy and big data techniques,the model construction time of BDHDLS is reduced substantially when multiple machines are deployed.