Tri-party deep network representation learning using inductive matrix completion

Most existing network representation learning algorithms focus on network structures for learning.However,network structure is only one kind of view and feature for various networks,and it cannot fully reflect all characteristics of networks.In fact,network vertices usually contain rich text information,which can be well utilized to learn text-enhanced network representations.Meanwhile,Matrix-Forest Index(MFI)has shown its high effectiveness and stability in link prediction tasks compared with other algorithms of link prediction.Both MFI and Inductive Matrix Completion(IMC)are not well applied with algorithmic frameworks of typical representation learning methods.Therefore,we proposed a novel semi-supervised algorithm,tri-party deep network representation learning using inductive matrix completion(TDNR).Based on inductive matrix completion algorithm,TDNR incorporates text features,the link certainty degrees of existing edges and the future link probabilities of non-existing edges into network representations.The experimental results demonstrated that TFNR outperforms other baselines on three real-world datasets.The visualizations of TDNR show that proposed algorithm is more discriminative than other unsupervised approaches.

HCRVD: A Vulnerability Detection System Based on CST-PDG Hierarchical Code Representation Learning

Prior studies have demonstrated that deep learning-based approaches can enhance the performance of source code vulnerability detection by training neural networks to learn vulnerability patterns in code representations.However,due to limitations in code representation and neural network design,the validity and practicality of the model still need to be improved.Additionally,due to differences in programming languages,most methods lack cross-language detection generality.To address these issues,in this paper,we analyze the shortcomings of previous code representations and neural networks.We propose a novel hierarchical code representation that combines Concrete Syntax Trees(CST)with Program Dependence Graphs(PDG).Furthermore,we introduce a Tree-Graph-Gated-Attention(TGGA)network based on gated recurrent units and attention mechanisms to build a Hierarchical Code Representation learning-based Vulnerability Detection(HCRVD)system.This system enables cross-language vulnerability detection at the function-level.The experiments show that HCRVD surpasses many competitors in vulnerability detection capabilities.It benefits from the hierarchical code representation learning method,and outperforms baseline in cross-language vulnerability detection by 9.772%and 11.819%in the C/C++and Java datasets,respectively.Moreover,HCRVD has certain ability to detect vulnerabilities in unknown programming languages and is useful in real open-source projects.HCRVD shows good validity,generality and practicality.

Homogeneity Analysis of Multiairport System Based on Airport Attributed Network Representation Learning

The homogeneity analysis of multi-airport system can provide important decision-making support for the route layout and cooperative operation.Existing research seldom analyzes the homogeneity of multi-airport system from the perspective of route network analysis,and the attribute information of airport nodes in the airport route network is not appropriately integrated into the airport network.In order to solve this problem,a multi-airport system homogeneity analysis method based on airport attribute network representation learning is proposed.Firstly,the route network of a multi-airport system with attribute information is constructed.If there are flights between airports,an edge is added between airports,and regional attribute information is added for each airport node.Secondly,the airport attributes and the airport network vector are represented respectively.The airport attributes and the airport network vector are embedded into the unified airport representation vector space by the network representation learning method,and then the airport vector integrating the airport attributes and the airport network characteristics is obtained.By calculating the similarity of the airport vectors,it is convenient to calculate the degree of homogeneity between airports and the homogeneity of the multi-airport system.The experimental results on the Beijing-Tianjin-Hebei multi-airport system show that,compared with other existing algorithms,the homogeneity analysis method based on attributed network representation learning can get more consistent results with the current situation of Beijing-Tianjin-Hebei multi-airport system.

A malware propagation prediction model based on representation learning and graph convolutional networks

The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.

Heterogeneous graph construction and node representation learning method of Treatise on Febrile Diseases based on graph convolutional network

Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.

Multi-task Joint Sparse Representation Classification Based on Fisher Discrimination Dictionary Learning

Recently,sparse representation classification(SRC)and fisher discrimination dictionary learning(FDDL)methods have emerged as important methods for vehicle classification.In this paper,inspired by recent breakthroughs of discrimination dictionary learning approach and multi-task joint covariate selection,we focus on the problem of vehicle classification in real-world applications by formulating it as a multi-task joint sparse representation model based on fisher discrimination dictionary learning to merge the strength of multiple features among multiple sensors.To improve the classification accuracy in complex scenes,we develop a new method,called multi-task joint sparse representation classification based on fisher discrimination dictionary learning,for vehicle classification.In our proposed method,the acoustic and seismic sensor data sets are captured to measure the same physical event simultaneously by multiple heterogeneous sensors and the multi-dimensional frequency spectrum features of sensors data are extracted using Mel frequency cepstral coefficients(MFCC).Moreover,we extend our model to handle sparse environmental noise.We experimentally demonstrate the benefits of joint information fusion based on fisher discrimination dictionary learning from different sensors in vehicle classification tasks.

Enhanced Deep Autoencoder Based Feature Representation Learning for Intelligent Intrusion Detection System

In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.

Connected Components-based Colour Image Representations of Vibrations for a Two-stage Fault Diagnosis of Roller Bearings Using Convolutional Neural Networks

Roller bearing failure is one of the most common faults in rotating machines.Various techniques for bearing fault diagnosis based on faults feature extraction have been proposed.But feature extraction from fault signals requires expert prior information and human labour.Recently,deep learning algorithms have been applied extensively in the condition monitoring of rotating machines to learn features automatically from the input data.Given its robust performance in image recognition,the convolutional neural network(CNN)architecture has been widely used to learn automatically discriminative features from vibration images and classify health conditions.This paper proposes and evaluates a two-stage method RGBVI-CNN for roller bearings fault diagnosis.The first stage in the proposed method is to generate the RGB vibration images(RGBVIs)from the input vibration signals.To begin this process,first,the 1-D vibration signals were converted to 2-D grayscale vibration Images.Once the conversion was completed,the regions of interest(ROI)were found in the converted 2-D grayscale vibration images.Finally,to produce vibration images with more discriminative characteristics,an algorithm was applied to the 2-D grayscale vibration images to produce connected components-based RGB vibration images(RGBVIs)with sets of colours and texture features.In the second stage,with these RGBVIs a CNN-based architecture was employed to learn automatically features from the RGBVIs and to classify bearing health conditions.Two cases of fault classification of rolling element bearings are used to validate the proposed method.Experimental results of this investigation demonstrate that RGBVI-CNN can generate advantageous health condition features from bearing vibration signals and classify the health conditions under different working loads with high accuracy.Moreover,several classification models trained using RGBVI-CNN offered high performance in the testing results of the overall classification accuracy,precision,recall,and F-score.

Early Detection of Diabetic Retinopathy Using Machine Intelligence throughDeep Transfer and Representational Learning

Diabetic retinopathy (DR) is a retinal disease that causes irreversible blindness.DR occurs due to the high blood sugar level of the patient, and it is clumsy tobe detected at an early stage as no early symptoms appear at the initial level. To preventblindness, early detection and regular treatment are needed. Automated detectionbased on machine intelligence may assist the ophthalmologist in examining thepatients’ condition more accurately and efficiently. The purpose of this study is toproduce an automated screening system for recognition and grading of diabetic retinopathyusing machine learning through deep transfer and representational learning.The artificial intelligence technique used is transfer learning on the deep neural network,Inception-v4. Two configuration variants of transfer learning are applied onInception-v4: Fine-tune mode and fixed feature extractor mode. Both configurationmodes have achieved decent accuracy values, but the fine-tuning method outperformsthe fixed feature extractor configuration mode. Fine-tune configuration modehas gained 96.6% accuracy in early detection of DR and 97.7% accuracy in gradingthe disease and has outperformed the state of the art methods in the relevant literature.

Research of Dynamic Competitive Learning in Neural Networks

Introduce a method of generation of new units within a cluster and aalgorithm of generating new clusters. The model automatically builds up its dynamically growinginternal representation structure during the learning process. Comparing model with other typicalclassification algorithm such as the Kohonen's self-organizing map, the model realizes a multilevelclassification of the input pattern with an optional accuracy and gives a strong support possibilityfor the parallel computational main processor. The idea is suitable for the high-level storage ofcomplex datas structures for object recognition.

CoLM^(2)S:Contrastive self‐supervised learning on attributed multiplex graph network with multi‐scale information

Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.

Hierarchical Representations Feature Deep Learning for Face Recognition

Most modern face recognition and classification systems mainly rely on hand-crafted image feature descriptors. In this paper, we propose a novel deep learning algorithm combining unsupervised and supervised learning named deep belief network embedded with Softmax regress (DBNESR) as a natural source for obtaining additional, complementary hierarchical representations, which helps to relieve us from the complicated hand-crafted feature-design step. DBNESR first learns hierarchical representations of feature by greedy layer-wise unsupervised learning in a feed-forward (bottom-up) and back-forward (top-down) manner and then makes more efficient recognition with Softmax regress by supervised learning. As a comparison with the algorithms only based on supervised learning, we again propose and design many kinds of classifiers: BP, HBPNNs, RBF, HRBFNNs, SVM and multiple classification decision fusion classifier (MCDFC)—hybrid HBPNNs-HRBFNNs-SVM classifier. The conducted experiments validate: Firstly, the proposed DBNESR is optimal for face recognition with the highest and most stable recognition rates;second, the algorithm combining unsupervised and supervised learning has better effect than all supervised learning algorithms;third, hybrid neural networks have better effect than single model neural network;fourth, the average recognition rate and variance of these algorithms in order of the largest to the smallest are respectively shown as DBNESR, MCDFC, SVM, HRBFNNs, RBF, HBPNNs, BP and BP, RBF, HBPNNs, HRBFNNs, SVM, MCDFC, DBNESR;at last, it reflects hierarchical representations of feature by DBNESR in terms of its capability of modeling hard artificial intelligent tasks.

Meta-Path-Based Deep Representation Learning for Personalized Point of Interest Recommendation

With the wide application of location-based social networks(LBSNs),personalized point of interest(POI)recommendation becomes popular,especially in the commercial field.Unfortunately,it is challenging to accurately recommend POIs to users because the user-POI matrix is extremely sparse.In addition,a user's check-in activities are affected by many influential factors.However,most of existing studies capture only few influential factors.It is hard for them to be extended to incorporate other heterogeneous information in a unified way.To address these problems,we propose a meta-path-based deep representation learning(MPDRL)model for personalized POI recommendation.In this model,we design eight types of meta-paths to fully utilize the rich heterogeneous information in LBSNs for the representations of users and POIs,and deeply mine the correlations between users and POIs.To further improve the recommendation performance,we design an attention-based long short-term memory(LSTM)network to learn the importance of different influential factors on a user's specific check-in activity.To verify the effectiveness of our proposed method,we conduct extensive experiments on a real-world dataset,Foursquare.Experimental results show that the MPDRL model improves at least 16.97%and 23.55%over all comparison methods in terms of the metric Precision@N(Pre@N)and Recall@N(Rec@N)respectively.

LC-NPLA: Label and Community Information-Based Network Presentation Learning Algorithm

Many network presentation learning algorithms(NPLA)have originated from the process of the random walk between nodes in recent years.Despite these algorithms can obtain great embedding results,there may be also some limitations.For instance,only the structural information of nodes is considered when these kinds of algorithms are constructed.Aiming at this issue,a label and community information-based network presentation learning algorithm(LC-NPLA)is proposed in this paper.First of all,by using the community information and the label information of nodes,the first-order neighbors of nodes are reconstructed.In the next,the random walk strategy is improved by integrating the degree information and label information of nodes.Then,the node sequence obtained from random walk sampling is transformed into the node representation vector by the Skip-Gram model.At last,the experimental results on ten real-world networks demonstrate that the proposed algorithm has great advantages in the label classification,network reconstruction and link prediction tasks,compared with three benchmark algorithms.

PUNet:A Semi-Supervised Anomaly Detection Model for Network Anomaly Detection Based on Positive Unlabeled Data

Network anomaly detection plays a vital role in safeguarding network security.However,the existing network anomaly detection task is typically based on the one-class zero-positive scenario.This approach is susceptible to overfitting during the training process due to discrepancies in data distribution between the training set and the test set.This phenomenon is known as prediction drift.Additionally,the rarity of anomaly data,often masked by normal data,further complicates network anomaly detection.To address these challenges,we propose the PUNet network,which ingeniously combines the strengths of traditional machine learning and deep learning techniques for anomaly detection.Specifically,PUNet employs a reconstruction-based autoencoder to pre-train normal data,enabling the network to capture potential features and correlations within the data.Subsequently,PUNet integrates a sampling algorithm to construct a pseudo-label candidate set among the outliers based on the reconstruction loss of the samples.This approach effectively mitigates the prediction drift problem by incorporating abnormal samples.Furthermore,PUNet utilizes the CatBoost classifier for anomaly detection to tackle potential data imbalance issues within the candidate set.Extensive experimental evaluations demonstrate that PUNet effectively resolves the prediction drift and data imbalance problems,significantly outperforming competing methods.

Heterophilic Graph Neural Network Based on Spatial and Frequency Domain Adaptive Embedding Mechanism

Graph Neural Networks(GNNs)play a significant role in tasks related to homophilic graphs.Traditional GNNs,based on the assumption of homophily,employ low-pass filters for neighboring nodes to achieve information aggregation and embedding.However,in heterophilic graphs,nodes from different categories often establish connections,while nodes of the same category are located further apart in the graph topology.This characteristic poses challenges to traditional GNNs,leading to issues of“distant node modeling deficiency”and“failure of the homophily assumption”.In response,this paper introduces the Spatial-Frequency domain Adaptive Heterophilic Graph Neural Networks(SFA-HGNN),which integrates adaptive embedding mechanisms for both spatial and frequency domains to address the aforementioned issues.Specifically,for the first problem,we propose the“Distant Spatial Embedding Module”,aiming to select and aggregate distant nodes through high-order randomwalk transition probabilities to enhance modeling capabilities.For the second issue,we design the“Proximal Frequency Domain Embedding Module”,constructing adaptive filters to separate high and low-frequency signals of nodes,and introduce frequency-domain guided attention mechanisms to fuse the relevant information,thereby reducing the noise introduced by the failure of the homophily assumption.We deploy the SFA-HGNN on six publicly available heterophilic networks,achieving state-of-the-art results in four of them.Furthermore,we elaborate on the hyperparameter selection mechanism and validate the performance of each module through experimentation,demonstrating a positive correlation between“node structural similarity”,“node attribute vector similarity”,and“node homophily”in heterophilic networks.

面向有向图的特征提取与表征学习研究

图数据是一种用于描述不同实体之间关联关系的重要数据形式。有向图作为一种特殊形式,不仅能描述实体关联,还能明确关系的方向,提供了更精细和详实的描述。因此,有向图的特征提取和表征学习对于深入理解复杂系统具有至关重要的意义。然而,现有方法在有效提取有向图的方向信息方面仍然存在挑战,主要依赖于节点的局部信息进行特征提取,难以充分利用有向边蕴含的方向信息。为解决这一问题,提出了一种名为变分有向图自编码器(variational directed graph autoencoder,VDGAE)的无监督表示学习方法。VDGAE通过关联矩阵来建模节点与边之间的关联关系,通过计算节点与边之间的亲和力,来重构输入有向图,从而实现无监督表征学习。基于此,VDGAE能够同时为输入有向图学习节点与边的表征,充分捕获有向图的结构信息和方向信息并嵌入至节点与边的表征向量中,使得有向图能够被更准确地表征。实验结果表明,相较于11个基准方法,VDGAE在5个数据集上节点分类任务均优于基准方法,提升了11.96%的预测精度,充分验证了其有效性。

面向异构社交网络的空-频域自适应图神经网络

传统GNN基于同构性假设对近邻节点实现低通滤波功能完成邻域相似信息的聚合嵌入。但在异构图中分属不同类别的节点彼此多建立联系,而相同类别的节点在图拓扑位置上距离较远。这一特点给注重近端邻域信息聚合的传统GNN带来“远端节点信息聚合缺失”与“同构性假设失灵”的问题。因此设计融合空域与频域自适应嵌入机制的异构图神经网络(DA-HGNN)以解决上述问题。针对问题一,设计“远端空域嵌入模块”,旨在通过高阶随机游走迁移概率筛选并聚合远端相似节点,补充“消息传递的跨邻域自适应性”;针对问题二,设计“近端频域嵌入模块”,构建滤波器分离节点高频与低频信号,并设计频域导向型注意力机制对上述信息进行频域偏好的自适应融合,从而减少“同构性假设失灵”所引入的噪声。在四个公开异构图数据集中取得最优实验结果,准确率上平均提高6.41个百分点。在灵敏度分析和消融实验中阐述了超参数的选择机制和各模块的实际性能,并验证了在异构网络中“节点结构相似性”“节点属性向量相似性”以及“节点同构性”三者之间仍呈现正相关关系这一结论。在异构真实数据集中验证了欺诈检测效果,AUC指标提升4.4个百分点。

基于自适应差异化图卷积的图注意力网络表示学习算法

为解决传统图卷积网络在处理节点间复杂关系时存在的局限性,提出一种基于自适应差异化图卷积的图注意力网络表示学习算法。采用差异化图卷积网络,依据每个节点自身特征和邻居信息进行差异化采样,捕捉节点间的复杂关系;再结合二阶段关键相邻采样方式优先挖掘重要节点并保留随机性,完成关键邻居节点的采样;然后结合图注意力网络,通过局部关注和自适应学习权重分配将关键邻居节点特征聚合到自身节点上,增强节点的特征表示;最后经网络训练,进一步增强网络表示学习能力。实验结果表明,所提出的算法优化了节点聚合程度和边界清晰度,提高了节点分类的准确性和可视化效果,并且通过关注二阶邻居和使用双头注意力,在网络表示学习上也展现出了优越性能。

跨粒度子图对比学习与注意力融合的药物—基因关系预测

【目的】阐明药物和基因之间的相互联系是药物开发中的一个重要课题。目前,基于随机游走算法的图神经网络方法在解决药物与基因交互关系识别上已经取得了不错的效果,但是当前的方法,单一子图的方法往往容易忽略掉全局图的信息,不能够很好地聚合节点的信息,同时,药物和基因的节点表示采用简单的融合方式,不能够有效地利用节点表示的信息,用于交互关系的分类。针对上述问题提出了跨粒度对比学习与注意力融合的药物-基因交互关系预测方法。【方法】一方面采用跨粒度的对比学习方法,得到远距离和近距离的节点信息,同时采用对比学习的结构增加对药物和基因节点的区分。另一方面利用注意力融合机制,充分挖掘节点中隐含的信息,将远近距离信息进行注意力融合。【结果】在2个真实数据集上的实验结果表明该模型比基线模型具有更好的分类效果。