A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

NEURAL NETWORK PREDICTIVE CONTROL WITH HIERARCHICAL GENETIC ALGORITHM

A kind of predictive control based on the neural network(NN) for nonlinear systems with time delay is addressed.The off line NN model is obtained by using hierarchical genetic algorithms (HGA) to train a sequence data of input and output.Output predictions are obtained by recursively mapping the NN model.The error rectification term is introduced into a performance function that is directly optimized while on line control so that it overcomes influences of the mismatched model and disturbances,etc.Simulations show the system has good dynamic responses and robustness.

Soft sensor of chemical processes with large numbers of input parameters using auto-associative hierarchical neural network

To explore the problems of monitoring chemical processes with large numbers of input parameters, a method based on Auto-associative Hierarchical Neural Network(AHNN) is proposed. AHNN focuses on dealing with datasets in high-dimension. AHNNs consist of two parts: groups of subnets based on well trained Autoassociative Neural Networks(AANNs) and a main net. The subnets play an important role on the performance of AHNN. A simple but effective method of designing the subnets is developed in this paper. In this method,the subnets are designed according to the classification of the data attributes. For getting the classification, an effective method called Extension Data Attributes Classification(EDAC) is adopted. Soft sensor using AHNN based on EDAC(EDAC-AHNN) is introduced. As a case study, the production data of Purified Terephthalic Acid(PTA) solvent system are selected to examine the proposed model. The results of the EDAC-AHNN model are compared with the experimental data extracted from the literature, which shows the efficiency of the proposed model.

A Novel Real-Time Fault Diagnostic System for Steam Turbine Generator Set by Using Strata Hierarchical Artificial Neural Network

The real-time fault diagnosis system is very great important for steam turbine generator set due to a serious fault results in a reduced amount of electricity supply in power plant. A novel real-time fault diagnosis system is proposed by using strata hierarchical fuzzy CMAC neural network. A framework of the fault diagnosis system is described. Hierarchical fault diagnostic structure is discussed in detail. The model of a novel fault diagnosis system by using fuzzy CMAC are built and analyzed. A case of the diagnosis is simulated. The results show that the real-time fault diagnostic system is of high accuracy, quick convergence, and high noise rejection. It is also found that this model is feasible in real-time fault diagnosis.

Neighborhood fusion-based hierarchical parallel feature pyramid network for object detection

In order to improve the detection accuracy of small objects,a neighborhood fusion-based hierarchical parallel feature pyramid network(NFPN)is proposed.Unlike the layer-by-layer structure adopted in the feature pyramid network(FPN)and deconvolutional single shot detector(DSSD),where the bottom layer of the feature pyramid network relies on the top layer,NFPN builds the feature pyramid network with no connections between the upper and lower layers.That is,it only fuses shallow features on similar scales.NFPN is highly portable and can be embedded in many models to further boost performance.Extensive experiments on PASCAL VOC 2007,2012,and COCO datasets demonstrate that the NFPN-based SSD without intricate tricks can exceed the DSSD model in terms of detection accuracy and inference speed,especially for small objects,e.g.,4%to 5%higher mAP(mean average precision)than SSD,and 2%to 3%higher mAP than DSSD.On VOC 2007 test set,the NFPN-based SSD with 300×300 input reaches 79.4%mAP at 34.6 frame/s,and the mAP can raise to 82.9%after using the multi-scale testing strategy.

Research on Global Higher Education Quality Based on BP Neural Network and Analytic Hierarchy Process

Having a universal, fair, democratic and practical higher education system plays a particularly important role in the future development of the country. However, the higher education system in various countries is uneven. It is of great significance to establish a general evaluation system for the development of global education. In this paper, 23 indicators are preliminarily selected from the education data of Universitas 21 and Global Statistical Yearbook. After the gray correlation analysis, 12 indicators were selected. On the one hand, principal component analysis is used to reduce the dimension of these 12 indicators in 50 countries, and the first four principal components with cumulative contribution rate of 99% are finally selected as the input parameters of BP neural network. On the other hand, 12 indicators are divided into four aspects as the standard of scheme decision-making. Finally, a higher education quality evaluation and decision-making model based on BP neural network and analytic hierarchy process are established. Then eight countries are selected to use the model to evaluate their current higher education quality. Based on the input and evaluation results of the four aspects of higher education in various countries, the analytic hierarchy process is used to make program decision, and several improvement suggestions are put forward for the current education policies of various countries.

Corporate Credit Ratings Based on Hierarchical Heterogeneous Graph Neural Networks

order to help investors understand the credit status of target corporations and reduce investment risks,the corporate credit rating model has become an important evaluation tool in the financial market.These models are based on statistical learning,machine learning and deep learning especially graph neural networks(GNNs).However,we found that only few models take the hierarchy,heterogeneity or unlabeled data into account in the actual corporate credit rating process.Therefore,we propose a novel framework named hierarchical heterogeneous graph neural networks(HHGNN),which can fully model the hierarchy of corporate features and the heterogeneity of relationships between corporations.In addition,we design an adversarial learning block to make full use of the rich unlabeled samples in the financial data.Extensive experiments conducted on the public-listed corporate rating dataset prove that HHGNN achieves SOTA compared to the baseline methods.

APPLICATION OF NEURAL NETWORK WITH MULTI-HIERARCHIC STRUCTURE TO EVALUATE SUSTAINABLE DEVELOPMENT OF THE COAL MINES

The neural network with multi hierarchic structure is provided in this paper to evaluate sustainable development of the coal mines based on analyzing its effect factors. The whole evaluating system is composed of 5 neural networks.The feasibility of this method has been proved by case study. This study will provide a scientfic and theoretic foundation for evaluating the sustainable development of coal mines.

OTT Messages Modeling and Classification Based on Recurrent Neural Networks

A vast amount of information has been produced in recent years,which brings a huge challenge to information management.The better usage of big data is of important theoretical and practical significance for effectively addressing and managing messages.In this paper,we propose a nine-rectangle-grid information model according to the information value and privacy,and then present information use policies based on the rough set theory.Recurrent neural networks were employed to classify OTT messages.The content of user interest is effectively incorporated into the classification process during the annotation of OTT messages,ending with a reliable trained classification model.Experimental results showed that the proposed method yielded an accurate classification performance and hence can be used for effective distribution and control of OTT messages.

Enhancing Collaborative and Geometric Multi-Kernel Learning Using Deep Neural Network

This research proposes a method called enhanced collaborative andgeometric multi-kernel learning (E-CGMKL) that can enhance the CGMKLalgorithm which deals with multi-class classification problems with non-lineardata distributions. CGMKL combines multiple kernel learning with softmaxfunction using the framework of multi empirical kernel learning (MEKL) inwhich empirical kernel mapping (EKM) provides explicit feature constructionin the high dimensional kernel space. CGMKL ensures the consistent outputof samples across kernel spaces and minimizes the within-class distance tohighlight geometric features of multiple classes. However, the kernels constructed by CGMKL do not have any explicit relationship among them andtry to construct high dimensional feature representations independently fromeach other. This could be disadvantageous for learning on datasets with complex hidden structures. To overcome this limitation, E-CGMKL constructskernel spaces from hidden layers of trained deep neural networks (DNN).Due to the nature of the DNN architecture, these kernel spaces not onlyprovide multiple feature representations but also inherit the compositionalhierarchy of the hidden layers, which might be beneficial for enhancing thepredictive performance of the CGMKL algorithm on complex data withnatural hierarchical structures, for example, image data. Furthermore, ourproposed scheme handles image data by constructing kernel spaces from aconvolutional neural network (CNN). Considering the effectiveness of CNNarchitecture on image data, these kernel spaces provide a major advantageover the CGMKL algorithm which does not exploit the CNN architecture forconstructing kernel spaces from image data. Additionally, outputs of hiddenlayers directly provide features for kernel spaces and unlike CGMKL, do notrequire an approximate MEKL framework. E-CGMKL combines the consistency and geometry preserving aspects of CGMKL with the compositionalhierarchy of kernel spaces extracted from DNN hidden layers to enhance the predictive performance of CGMKL significantly. The experimental results onvarious data sets demonstrate the superior performance of the E-CGMKLalgorithm compared to other competing methods including the benchmarkCGMKL.

Hierarchical Representations Feature Deep Learning for Face Recognition

Most modern face recognition and classification systems mainly rely on hand-crafted image feature descriptors. In this paper, we propose a novel deep learning algorithm combining unsupervised and supervised learning named deep belief network embedded with Softmax regress (DBNESR) as a natural source for obtaining additional, complementary hierarchical representations, which helps to relieve us from the complicated hand-crafted feature-design step. DBNESR first learns hierarchical representations of feature by greedy layer-wise unsupervised learning in a feed-forward (bottom-up) and back-forward (top-down) manner and then makes more efficient recognition with Softmax regress by supervised learning. As a comparison with the algorithms only based on supervised learning, we again propose and design many kinds of classifiers: BP, HBPNNs, RBF, HRBFNNs, SVM and multiple classification decision fusion classifier (MCDFC)—hybrid HBPNNs-HRBFNNs-SVM classifier. The conducted experiments validate: Firstly, the proposed DBNESR is optimal for face recognition with the highest and most stable recognition rates;second, the algorithm combining unsupervised and supervised learning has better effect than all supervised learning algorithms;third, hybrid neural networks have better effect than single model neural network;fourth, the average recognition rate and variance of these algorithms in order of the largest to the smallest are respectively shown as DBNESR, MCDFC, SVM, HRBFNNs, RBF, HBPNNs, BP and BP, RBF, HBPNNs, HRBFNNs, SVM, MCDFC, DBNESR;at last, it reflects hierarchical representations of feature by DBNESR in terms of its capability of modeling hard artificial intelligent tasks.

基于PSO-BP神经网络的磨机传动系统模型修正

针对磨机传动系统结构的复杂性、部件间约束条件的不确定性以及非线性等因素,提出了一种基于PSO-BP神经网络的有限元模型修正方法。通过改进BP神经网络逼近设计参数和特征量间的非线性映射关系,结合实际结构响应,利用神经网络的泛化特性,得到了模型设计参数值。修正后频率误差从最高18%降到4%左右,修正系数误差范围均在0.5%以内,明显提高了有限元模型精度;同时,又不需要大量迭代求解步骤,避开了传统反问题模型修正法的复杂非线性优化过程,提升了效率,验证了PSO-BP神经网络法应用于大型磨机传动系统上的可行性,为后续传动系统整体分析奠定了基础。

一种基于BiLSTM 的混合层次化图分类模型

图分类在化学和生物信息学等诸多领域中是一个非常重要且极具挑战的问题,GNN模型是图分类问题的主流方法。现有的GNN模型采用卷积操作来实现邻域节点信息聚集,再通过池化操作生成粗化图。然而,仅通过池化方法不能捕获到每次卷积后读出图的双向依赖关系。为了提取到更充分的特征信息,提出一种混合层次化模型,首先分别提取节点特征信息和结构特征信息,再将特征信息融合,然后采用BiLSTM捕获不同层次读出图之间的双向依赖关系,从而提取到更丰富的特征信息。实验结果表明,与对比模型相比,上述模型的准确度有着明显的提升。

基于节点采样的子结构代表层次池化图卷积网络模型

为解决目前基于节点采样的图池化方法中所存在的评估节点重要性的策略过于简单以及子结构特征信息大量丢失等问题,提出了基于节点采样的子结构代表层次池化模型(sub-structure representative hierarchical pooling model based on node sampling,SsrPool)。该模型主要包括子结构代表节点选择模块和子结构代表节点特征生成模块2个部分。首先,子结构代表节点选择模块同时考虑了节点特征信息以及结构信息,利用不同方法评估节点重要性并通过不同重要性分数协作产生鲁棒的节点排名以指导节点选择。其次,子结构代表节点特征生成模块通过特征融合保留局部子结构特征信息。通过将SsrPool与现有神经网络相结合,在不同规模公共数据集上的图分类实验结果证明了SsrPool的有效性。

基于改进分层注意网络和TextCNN联合建模的暴力犯罪分级算法

为了科学、智能地对服刑人员的暴力倾向分级,将自然语言处理(NLP)中的文本分类方法引入犯罪心理学领域,提出一种基于改进分层注意网络(HAN)与TextCNN(Text Convolutional Neural Network)两通道联合建模的犯罪语义卷积分层注意网络(CCHA-Net),通过分别挖掘犯罪事实与服刑人员基本情况的语义信息,完成暴力犯罪气质分级。首先,采用Focal Loss同时替代两通道中的Cross-Entropy函数,优化样本数量不均衡问题。其次,在两通道输入层中,同时引入位置编码,改进对位置信息的感知能力;改进HAN通道,采用最大池化构建显著向量。最后,输出层都采用全局平均池化替代全连接方法,以避免过拟合。实验结果表明,与AC-BiLSTM(Attention-based Bidirectional Long Short-Term Memory with Convolution layer)、支持向量机(SVM)等17种相关基线模型相比,CCHA-Net各项指标均最优,微平均F1(Micro_F1)为99.57%,宏平均和微平均下的曲线下面积(AUC)分别为99.45%和99.89%,相较于次优的AC-BiLSTM提高了4.08、5.59和0.74个百分点,验证了CCHA-Net能有效胜任暴力犯罪气质分级任务。

邻域信息分层感知的知识图谱补全方法

知识图谱补全(KGC)旨在利用知识图谱的现有知识推断三元组的缺失值。近期的一些研究表明,将图卷积网络(GCN)应用于KGC任务有助于改善模型的推理性能。针对目前大多数GCN模型存在的同等对待邻域信息、忽略了邻接实体对中心实体的不同贡献度、采用简单的线性变换更新关系嵌入等问题,提出了一个邻域信息分层感知的图神经网络模型NAHAT,在关系更新中引入实体特征信息,通过聚合实体和关系表征来丰富异质关系语义,提高模型的表达能力。同时,将自我对立的负样本训练应用到损失计算中,实现模型的高效训练。实验结果表明,与图卷积网络模型COMPGCN相比,所提出的模型在FB15K-237数据集上Hits@1、Hits@10指标分别提高了3%、2.6%;在WN18RR数据集上分别提高了0.9%、2.2%。验证了所提出的模型的有效性。

结合层次图神经网络与长短期记忆的产业链风险评估预警模型

产业链风险评估预警是有效保护产业链上下游公司利益和减轻公司风险的重要措施。然而,现有方法由于忽视了产业链上下游公司之间的传播效应和公司信息的不透明性,无法准确评估公司风险,且忽略了动态财务数据对产业链的影响,无法提前感知风险,进行风险预警。针对以上问题,提出一种结合层次图(HG)神经网络与长短期记忆(LSTM)的产业链风险评估预警模型(HiGNN)。首先,利用产业链上下游关系和投融资关系构建“产业链-投资”HG;其次,利用财务特征提取模块提取公司多季度财务数据的特征;再次,利用投资特征提取模块提取投资关系图特征;最后,利用注意力机制融合财务特征和投资特征,通过图表示学习方法对公司节点进行风险分类。在真实的集成电路制造业数据集上的实验结果表明,与图注意力网络(GAT)模型、循环神经网络(RNN)模型相比,当训练比率为60%时,所提模型的准确率分别提升了14.87%、22.10%,F1值提升了12.63%、16.67%。所提模型能够有效捕捉产业链中的传染效应,提高风险识别能力,优于传统的机器学习方法和图神经网络方法。

基于图神经网络的SSL/TLS加密恶意流量检测算法研究

为实现SSL/TLS加密恶意流量的精准检测,针对传统机器学习方法过分依赖专家经验的问题,提出一种基于图神经网络的恶意加密流量检测模型。通过对SSL/TLS加密会话进行分析,利用图结构对流量会话交互信息进行表征,将恶意加密流量检测问题转化为图分类问题。生成的模型基于分层图池化架构,通过多层卷积池化的聚合,结合注意力机制,充分挖掘图中节点特征和图结构信息,实现了端到端的恶意加密流量检测方法。基于公开数据集CICAndMal2017进行验证,实验结果表明,所提模型在加密恶意流量二分类检测中,准确率高达97.1%,相较于其他模型,准确率、召回率、精确率、F1分数分别提升了2.1%,3.2%,1.6%,2.1%,说明所提方法对于恶意加密流量的表征能力和检测能力优于其他方法。

基于分层强化学习的多智能体博弈策略生成方法

典型基于深度强化学习的多智能体对抗策略生成方法采用“分总”框架,各智能体基于部分可观测信息生成策略并进行决策,缺乏从整体角度生成对抗策略的能力,大大限制了决策能力。为了解决该问题,基于分层强化学习提出改进的多智能体博弈策略生成方法。基于分层强化学习构建观测信息到整体价值的决策映射,以最大化整体价值作为目标构建优化问题,并推导了策略优化过程,为后续框架结构和方法实现的设计提供了理论依据;基于决策映射与优化问题构建,采用神经网络设计了模型框架,详细阐述了顶层策略控制模型和个体策略执行模型;基于策略优化方法,给出详细训练流程和算法流程;采用星际争霸多智能体对抗(StarCraft Multi-Agent Challenge,SMAC)环境,与典型多智能体方法进行性能对比。实验结果表明,该方法能够有效生成对抗策略,控制异构多智能体战胜预设对手策略,相比典型多智能体强化学习方法性能提升明显。