Gate-Attention and Dual-End Enhancement Mechanism for Multi-Label Text Classification

In the realm of Multi-Label Text Classification(MLTC),the dual challenges of extracting rich semantic features from text and discerning inter-label relationships have spurred innovative approaches.Many studies in semantic feature extraction have turned to external knowledge to augment the model’s grasp of textual content,often overlooking intrinsic textual cues such as label statistical features.In contrast,these endogenous insights naturally align with the classification task.In our paper,to complement this focus on intrinsic knowledge,we introduce a novel Gate-Attention mechanism.This mechanism adeptly integrates statistical features from the text itself into the semantic fabric,enhancing the model’s capacity to understand and represent the data.Additionally,to address the intricate task of mining label correlations,we propose a Dual-end enhancement mechanism.This mechanism effectively mitigates the challenges of information loss and erroneous transmission inherent in traditional long short term memory propagation.We conducted an extensive battery of experiments on the AAPD and RCV1-2 datasets.These experiments serve the dual purpose of confirming the efficacy of both the Gate-Attention mechanism and the Dual-end enhancement mechanism.Our final model unequivocally outperforms the baseline model,attesting to its robustness.These findings emphatically underscore the imperativeness of taking into account not just external knowledge but also the inherent intricacies of textual data when crafting potent MLTC models.

Multi-label dimensionality reduction and classification with extreme learning machines

In the need of some real applications, such as text categorization and image classification, the multi-label learning gradually becomes a hot research point in recent years. Much attention has been paid to the research of multi-label classification algorithms. Considering the fact that the high dimensionality of the multi-label datasets may cause the curse of dimensionality and wil hamper the classification process, a dimensionality reduction algorithm, named multi-label kernel discriminant analysis （MLKDA）, is proposed to reduce the dimensionality of multi-label datasets. MLKDA, with the kernel trick, processes the multi-label integrally and realizes the nonlinear dimensionality reduction with the idea similar with linear discriminant analysis （LDA）. In the classification process of multi-label data, the extreme learning machine （ELM） is an efficient algorithm in the premise of good accuracy. MLKDA, combined with ELM, shows a good performance in multi-label learning experiments with several datasets. The experiments on both static data and data stream show that MLKDA outperforms multi-label dimensionality reduction via dependence maximization （MDDM） and multi-label linear discriminant analysis （MLDA） in cases of balanced datasets and stronger correlation between tags, and ELM is also a good choice for multi-label classification.

ML-CLUBAS: A Multi Label Bug Classification Algorithm

In this paper, a multi label variant of CLUBAS [1] algorithm, ML-CLUBAS (Multi Label-Classification of software Bugs Using Bug Attribute Similarity) is presented. CLUBAS is a hybrid algorithm, and is designed by using text clustering, frequent term calculations and taxonomic terms mapping techniques, and is an example of classification using clustering technique. CLUBAS is a single label algorithm, where one bug cluster is exactly mapped to a single bug category. However a bug cluster can be mapped into the more than one bug category in case of cluster label matches with the more than one category term, for this purpose ML-CLUBAS a multi label variant of CLUBAS is presented in this work. The designed algorithm is evaluated using the performance parameters F-measures and accuracy, number of clusters and purity. These parameters are compared with the CLUBAS and other multi label text clustering algorithms.

Pedestrian attribute classification with multi-scale and multi-label convolutional neural networks

Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.

Multi-label learning of face demographic classification for correlation analysis

In this paper, we utilize the framework of multi-label learning for face demographic classification. We also attempt t;o explore the suitable classifiers and features for face demographic classification. Three most popular demographic information, gender, ethnicity and age are considered in experiments. Based on the results from demographic classification, we utilize statistic analysis to explore the correlation among various face demographic information. Through the analysis, we draw several conclusions on the correlation and interaction among these high-level face semantic, and the obtained results can be helpful in automatic face semantic annotation and other face analysis tasks.

基于SWPF2vec和DJ-TextRCNN的古籍文本主题分类研究

以编目分类和规则匹配为主的古籍文本主题分类方法存在工作效能低、专家知识依赖性强、分类依据单一化、古籍文本主题自动分类难等问题。对此,本文结合古籍文本内容和文字特征,尝试从古籍内容分类得到符合研究者需求的主题,推动数字人文研究范式的转型。首先,参照东汉古籍《说文解字》对文字的分析方式,以前期标注的古籍语料数据集为基础,构建全新的“字音(说)-原文(文)-结构(解)-字形(字)”四维特征数据集。其次,设计四维特征向量提取模型(speaking,word,pattern,and font to vector,SWPF2vec),并结合预训练模型实现对古籍文本细粒度的特征表示。再其次,构建融合卷积神经网络、循环神经网络和多头注意力机制的古籍文本主题分类模型(dianji-recurrent convolutional neural networks for text classification,DJ-TextRCNN)。最后,融入四维语义特征,实现对古籍文本多维度、深层次、细粒度的语义挖掘。在古籍文本主题分类任务上,DJ-TextRCNN模型在不同维度特征下的主题分类准确率均为最优,在“说文解字”四维特征下达到76.23%的准确率,初步实现了对古籍文本的精准主题分类。

基于Bert-TextCNN的开源威胁情报文本的多标签分类方法

开源威胁情报对网络安全防护十分重要,但其存在着分布广、形式多、噪声大的特点.所以如何能对收集到的海量开源威胁情报进行高效的整理和分析就成为亟需解决的问题.因此,探索了一种以Bert-TextCNN模型为基础且同时考虑标题、正文和正则判断的多标签分类方法.根据情报源发布文本的特点,设置正则判断规则,以弥补模型的欠缺;为更全面反映开源威胁情报文本所涉及的威胁主题,针对标题和正文分别设置了Bert-TextCNN多标签分类模型,并将2部分标签整理去重以得到文本的最终威胁类别.通过与只依据正文建立的Bert-TextCNN多标签分类模型进行对比,所设置的模型在性能上有所提升,且召回率提升明显,能为开源威胁情报分类工作提供有价值的参考.

基于Albert-TextCNN模型的多标签新闻文本分类

针对智能信息推送管理者的多标签新闻文本分类任务,提出了基于ALBERT-CNN模型的解决方案。利用ALBERT预训练模型和TextCNN卷积神经网络,充分进行语义理解和特征提取。通过ALBERT模型进行语义筛选,精准把握新闻文本内容和主题,再传递给TextCNN模型进行分类和标签预测。采用Sigmoid函数输出每个标签的概率,实现精准的多标签分类。实验验证382688条来自今日头条客户端的数据,ALBERT-CNN模型的F1-Score达到92.05%,召回率达到96.8%,精确率达到90%,相比于优于传统的ALBERT和ALBERT-Denses模型的F1-Score和召回率有所提升。在精确率上略低于AlBERT-Dense。该研究为提高信息推送效率和降低误导性信息的传播提供了一个新的解决方案。

Novel Apriori-Based Multi-Label Learning Algorithm by Exploiting Coupled Label Relationship

It is a key challenge to exploit the label coupling relationship in multi-label classification（MLC）problems.Most previous work focused on label pairwise relations,in which generally only global statistical information is used to analyze the coupled label relationship.In this work,firstly Bayesian and hypothesis testing methods are applied to predict the label set size of testing samples within their k nearest neighbor samples,which combines global and local statistical information,and then apriori algorithm is used to mine the label coupling relationship among multiple labels rather than pairwise labels,which can exploit the label coupling relations more accurately and comprehensively.The experimental results on text,biology and audio datasets shown that,compared with the state-of-the-art algorithm,the proposed algorithm can obtain better performance on 5 common criteria.

基于语义筛选的ALBERT-TextCNN中医文本多标签分类研究

针对中医领域中的大量未标注文本,以及传统多标签分类模型提取的文本语义信息不够完整等问题,提出一种用于中医文本的多标签分类模型语义筛选ALBERT-TextCNN。首先进行特定领域任务自训练,将哮喘领域内属于多标签分类任务但未标注文本输入ALBERT进行预训练任务;其次ALBERT多层的Transform对已标注数据分别进行动态向量化表示,基于语义筛选选取最佳编码层生成的高效文本向量;最后引入TextCNN建立多标签分类器,提取文本向量不同层次的语义信息特征。在中医数据集上验证方法的有效性,实验结果表明,该模型的多标签分类精度有所提高,更适用于中医文本的分类预测。

Multi-label local discriminative embedding

Multi-label classification problems arise frequently in text categorization, and many other related applications. Like conventional categorization problems, multi-label categorization tasks suffer from the curse of high dimensionality. Existing multi-label dimensionality reduction methods mainly suffer from two limitations. First, latent nonlinear structures are not utilized in the input space. Second, the label information is not fully exploited. This paper proposes a new method, multi-label local discriminative embedding (MLDE), which exploits latent structures to minimize intraclass distances and maximize interclass distances on the basis of label correlations. The latent structures are extracted by constructing two sets of adjacency graphs to make use of nonlinear information. Non-symmetric label correlations, which are the case in real applications, are adopted. The problem is formulated into a global objective function and a linear mapping is achieved to solve out-of-sample problems. Empirical studies across 11 Yahoo sub-tasks, Enron and Bibtex are conducted to validate the superiority of MLDE to state-of-art multi-label dimensionality reduction methods.

基于Albert与TextCNN的中文文本分类研究

互联网数据众多,为高效管理互联网的海量中文文本数据,提出基于Albert与TextCNN的中文文本分类方法(ATT)。该方法引入Albert模型解决一词多义问题,使用TF-IDF算法抽取当前文本数据中权重最高的5个词构建整个文档关键词表,将关键词表与Albert生成的词向量进行向量拼接,构成一个融合关键词信息的多义词向量。并且,在传统TextCNN基础上根据中文语言特点调整卷积核窗口大小以提取文本数据的深层局部特征。实验表明,ATT模型相较于未加入TF-IDF关键词表、未调整卷积核大小的传统模型,F1值分别提升1.88%和2.26%,为中文文本分类提供了一种新方法。

Semantic Similarity over Gene Ontology for Multi-Label Protein Subcellular Localization

As one of the essential topics in proteomics and molecular biology, protein subcellular localization has been extensively studied in previous decades. However, most of the methods are limited to the prediction of single-location proteins. In many studies, multi-location proteins are either not considered or assumed not existing. This paper proposes a novel multi-label subcellular-localization predictor based on the semantic similarity between Gene Ontology (GO) terms. Given a protein, the accession numbers of its homologs are obtained via BLAST search. Then, the homologous accession numbers of the protein are used as keys to search against the gene ontology annotation database to obtain a set of GO terms. The semantic similarity between GO terms is used to formulate semantic similarity vectors for classification. A support vector machine (SVM) classifier with a new decision scheme is proposed to classify the multi-label GO semantic similarity vectors. Experimental results show that the proposed multi-label predictor significantly outperforms the state-of-the-art predictors such as iLoc-Plant and Plant-mPLoc.

基于注意力机制和CNN的多标签文本分类模型

针对目前多标签文本分类模型存在无法充分提取文本语义与标签的相互关系,提出一种基于注意力机制和卷积神经网络(CNN)的多标签文本分类模型。通过多头注意力机制和CNN对文本进行建模表示,充分挖掘文本全局和局部的语义特征;结合标签与文本信息进行交互注意力计算,捕捉结合文本内容后标签间的相互关系;使用一种自适应融合策略进一步提取两者语义信息。实验结果表明,该模型相比于其他主流模型能有效提升多标签文本分类效果。

基于互信息解决多标签文本分类中的长尾问题

针对当前解决多标签文本分类中长尾问题的方法多以破坏原本数据分布为代价,在真实数据上的泛化性能下降,无法有效地缓解样本的长尾分布的问题,提出了基于互信息解决长尾问题的多标签文本分类方法(MLTC-LD)。首先,创建关于标签样本的关系矩阵,计算标签样本间的依赖关系;其次,考虑标签样本间关系程度的强弱构造邻居选择器,将拥有强关系的邻居信息作为主要语义特征并作为先验信息;最后,通过图注意力神经网络将先验信息引入分类器,实现了借助分布头部数据丰富类的知识来提高尾部数据贫乏类性能的目标。在三个不同的数据集上将MLTC-LD与八个基线模型进行了广泛的比较分析。实验结果表明,MLTC-LD与最优的HGLRN相比精确度分别提高了3.5%、0.3%、1.5%,证明了该方法的有效性。

基于预训练模型与BiLSTM-CNN的多标签代码坏味检测方法

为了提高多标签代码坏味检测的准确率,提出一种基于预训练模型与BiLSTM-CNN的多标签代码坏味检测方法DMSmell(deep multi-smell)。首先,利用静态分析工具获取源代码中的文本信息和结构度量信息,并采用2种检测规则对代码坏味实例进行标记;其次,利用CodeBERT预训练模型生成文本信息对应的词向量,并分别采用BiLSTM和CNN对词向量和结构度量信息进行深度特征提取;最后,结合注意力机制和多层感知机,完成多标签代码坏味的检测,并对DMSmell方法进行了性能评估。结果表明:DMSmell方法在一定程度上提高了多标签代码坏味检测的准确率,与基于分类器链的方法相比,精确匹配率提高了1.36个百分点,微查全率提高了2.45个百分点,微F1提高了1.1个百分点。这表明,将文本信息与结构度量信息相结合,并利用深度学习技术进行特征提取和分类,可以有效提高代码坏味检测的准确性,为多标签代码坏味检测的研究和应用提供重要的参考。

融合双向注意力和对比增强机制的多标签文本分类

现有多标签文本分类模型面对文本序列增长时语义信息缺失,预测特定标签时忽略已有实例中的丰富知识等问题不能很好地解决,由此提出一种融合双向注意力和对比增强机制的多标签文本分类模型。结合BERT词嵌入,利用CTransformer模型分别获取序列的全局依赖关系和局部结构信息;使用双向注意力和标签嵌入进一步生成最终文本表示和标签表示,将文本信息与标签信息进行交互,得到更为全面的综合语义信息;利用对比增强机制进行KNN实例检索,设计一个多标签对比学习目标,使模型意识到KNN分类过程,并提高推理过程中检索邻居实例的质量。分类器根据标签表示和文本表示进行文本分类。为了评估模型的性能,在三个公开英文数据集上进行测试,实验结果表明提出的模型在P@K和nDCG@K指标上均优于其他主流基线模型。

融合GAT与头尾标签的多标签文本分类模型

现有基于神经网络的多标签文本分类主要存在两方面的不足,一是现有的数据中标签-文本频率存在长尾分布,二是很少从图结构中获取全局标签之间的关联性,针对以上问题,提出了一种融合GAT与头尾标签分类器的多标签文本分类模型(GATTN),该模型利用带有注意力机制的Bi-LSTM得到文本的特征向量表示;同时把不同标签之间的关联性转换为包含了全局信息的边加权图,利用多层的图注意力网络来学习头标签之间的关联性。然后将其与文本上下文语义信息进行点积交互,得到具有文本语义和头标签信息的特征表示提高模型的泛化能力。在AAPD、RCV1-V2和EUR-Lex三个公开英文数据集上的实验结果证明,该模型针对数据长尾分布的多标签文本分类效果优于其他基线模型。

基于标签语义匹配融合的多标签文本分类方法

针对当前多标签文本分类研究中存在的文本有效信息提取不充分、标签间的相关性被忽略、文本对标签的语义关注挖掘、利用不足的问题,该文提出了一种基于标签语义匹配融合的多标签文本分类方法.首先,利用DeBERTa模型来计算细粒度为单词级的文本表示;同时,根据标签全局共现构建标签图数据,利用图注意力网络自动学习不同标签之间的关联程度,生成捕获了标签间结构信息与深层相关性的标签特征嵌入;然后,提出了一种基于标签语义匹配的嵌入融合机制建模文本对标签的语义关注,体现了两者的语义关联,并将获得的基于标签语义匹配嵌入的单词融合表示送入CNN中进行特征交互,最终实现标签预测.在AAPD与RCV1-V2这两个公开英文数据集上的实验结果表明,该文所提出的模型性能明显优于其他主流基线模型.

融合多尺度CNN与双向LSTM的唐卡问句分类模型

当前大语言模型的兴起为自然语言处理、搜索引擎、生命科学研究等领域的研究者提供了新思路,但大语言模型存在资源消耗高、推理速度慢,难以在工业场景尤其是垂直领域应用等方面的缺点。针对这一问题,提出了一种多尺度卷积神经网络(convolutional neural network,CNN)与双向长短期记忆神经网络(long short term memory,LSTM)融合的唐卡问句分类模型,本文模型将数据的全局特征与局部特征进行融合实现唐卡问句分类任务,全局特征反映数据的本质特点,局部特征关注数据中易被忽视的部分,将二者以拼接的方式融合以丰富句子的特征表示。通过在Thangka数据集与THUCNews数据集上进行实验,结果表明,本文模型相较于Bert模型在精确度上略优,在训练时间上缩短了1/20,运算推理时间缩短了1/3。在公开数据集上的实验表明,本文模型在文本分类任务上也表现出了较好的适用性和有效性。