Quantitative Comparative Study of the Performance of Lossless Compression Methods Based on a Text Data Model

Data compression plays a key role in optimizing the use of memory storage space and also reducing latency in data transmission. In this paper, we are interested in lossless compression techniques because their performance is exploited with lossy compression techniques for images and videos generally using a mixed approach. To achieve our intended objective, which is to study the performance of lossless compression methods, we first carried out a literature review, a summary of which enabled us to select the most relevant, namely the following: arithmetic coding, LZW, Tunstall’s algorithm, RLE, BWT, Huffman coding and Shannon-Fano. Secondly, we designed a purposive text dataset with a repeating pattern in order to test the behavior and effectiveness of the selected compression techniques. Thirdly, we designed the compression algorithms and developed the programs (scripts) in Matlab in order to test their performance. Finally, following the tests conducted on relevant data that we constructed according to a deliberate model, the results show that these methods presented in order of performance are very satisfactory:- LZW- Arithmetic coding- Tunstall algorithm- BWT + RLELikewise, it appears that on the one hand, the performance of certain techniques relative to others is strongly linked to the sequencing and/or recurrence of symbols that make up the message, and on the other hand, to the cumulative time of encoding and decoding.

基于TextCNN模型的电子期刊文献推荐方法研究

论文提出基于TextCNN模型的电子期刊文献推荐方法,旨在更好地精确把握文献内容的本质特征与用户文献需求的深层关系,实现电子期刊文献推荐服务的个性化和精准化。使用word2vec对文献题录信息进行向量化,使用TextCNN模型训练文献推荐模型,最后主动将符合用户需求的文献推送给科研用户。实践证明,论文设计的推荐模型能够为用户推荐电子期刊文献,效果良好。

基于PaddleOCR与Style-Text的金融票据手写体文本识别

该文提出一种基于PaddleOCR框架的金融票据手写体文本识别方法,通过引入基于生成对抗网络(GAN)的数据合成工具Style-Text,增强模型对不同背景文本的识别能力。在真实的金融票据数据集上进行的实验表明,该方法在处理复杂文本和低质量图像方面表现出显著的优势,证明其在金融票据手写体文本识别中的有效性和实用性。

Clustering Text Data Streams

Clustering text data streams is an important issue in data mining community and has a number of applications such as news group filtering, text crawling, document organization and topic detection and tracing etc. However, most methods are similarity-based approaches and only use the TF,IDF scheme to represent the semantics of text data and often lead to poor clustering quality. Recently, researchers argue that semantic smoothing model is more efficient than the existing TF,IDF scheme for improving text clustering quality. However, the existing semantic smoothing model is not suitable for dynamic text data context. In this paper, we extend the semantic smoothing model into text data streams context firstly. Based on the extended model, we then present two online clustering algorithms OCTS and OCTSM for the clustering of massive text data streams. In both algorithms, we also present a new cluster statistics structure named cluster profile which can capture the semantics of text data streams dynamically and at the same time speed up the clustering process. Some efficient implementations for our algorithms are also given. Finally, we present a series of experimental results illustrating the effectiveness of our technique.

中文科技政策文本分类:增强的TextCNN视角

近年尽管针对中文本文分类的研究成果不少,但基于深度学习对中文政策等长文本进行自动分类的研究还不多见。为此,借鉴和拓展传统的数据增强方法,提出集成新时代人民日报分词语料库(NEPD)、简单数据增强(EDA)算法、word2vec和文本卷积神经网络(TextCNN)的NEWT新型计算框架;实证部分,基于中国地方政府发布的科技政策文本进行算法校验。实验结果显示,在取词长度分别为500、750和1000词的情况下,应用NEWT算法对中文科技政策文本进行分类的效果优于RCNN、Bi-LSTM和CapsNet等传统深度学习模型,F1值的平均提升比例超过13%;同时,NEWT在较短取词长度下能够实现全文输入的近似效果,可以部分改善传统深度学习模型在中文长文本自动分类任务中的计算效率。

Identifying Scientific Project-generated Data Citation from Full-text Articles: An Investigation of TCGA Data Citation

Purpose： In the open science era, it is typical to share project-generated scientific data by depositing it in an open and accessible database. Moreover, scientific publications are preserved in a digital library archive. It is challenging to identify the data usage that is mentioned in literature and associate it with its source. Here, we investigated the data usage of a government-funded cancer genomics project, The Cancer Genome Atlas（TCGA）, via a full-text literature analysis.Design/methodology/approach： We focused on identifying articles using the TCGA dataset and constructing linkages between the articles and the specific TCGA dataset. First, we collected 5,372 TCGA-related articles from Pub Med Central（PMC）. Second, we constructed a benchmark set with 25 full-text articles that truly used the TCGA data in their studies, and we summarized the key features of the benchmark set. Third, the key features were applied to the remaining PMC full-text articles that were collected from PMC.Findings： The amount of publications that use TCGA data has increased significantly since 2011, although the TCGA project was launched in 2005. Additionally, we found that the critical areas of focus in the studies that use the TCGA data were glioblastoma multiforme, lung cancer, and breast cancer; meanwhile, data from the RNA-sequencing（RNA-seq） platform is the most preferable for use.Research limitations： The current workflow to identify articles that truly used TCGA data is labor-intensive. An automatic method is expected to improve the performance.Practical implications： This study will help cancer genomics researchers determine the latest advancements in cancer molecular therapy, and it will promote data sharing and data-intensive scientific discovery.Originality/value： Few studies have been conducted to investigate data usage by governmentfunded projects/programs since their launch. In this preliminary study, we extracted articles that use TCGA data from PMC, and we created a link between the full-text articles and the source data.

Automatic User Goals Identification Based on Anchor Text and Click-Through Data

Understanding the underlying goal behind a user＇s Web query has been proved to be helpful to improve the quality of search. This paper focuses on the problem of automatic identification of query types according to the goals. Four novel entropy-based features extracted from anchor data and click-through data are proposed, and a support vector machines （SVM） classifier is used to identify the user＇s goal based on these features. Experi- mental results show that the proposed entropy-based features are more effective than those reported in previous work. By combin- ing multiple features the goals for more than 97% of the queries studied can be correctly identified. Besides these, this paper reaches the following important conclusions： First, anchor-based features are more effective than click-through-based features; Second, the number of sites is more reliable than the number of links; Third, click-distribution- based features are more effective than session-based ones.

A Complexity Analysis and Entropy for Different Data Compression Algorithms on Text Files

In this paper, we analyze the complexity and entropy of different methods of data compression algorithms: LZW, Huffman, Fixed-length code (FLC), and Huffman after using Fixed-length code (HFLC). We test those algorithms on different files of different sizes and then conclude that: LZW is the best one in all compression scales that we tested especially on the large files, then Huffman, HFLC, and FLC, respectively. Data compression still is an important topic for research these days, and has many applications and uses needed. Therefore, we suggest continuing searching in this field and trying to combine two techniques in order to reach a best one, or use another source mapping (Hamming) like embedding a linear array into a Hypercube with other good techniques like Huffman and trying to reach good results.

A feature representation method for biomedical scientific data based on composite text description

Feature representation is one of the key issues in data clustering. The existing feature representation of scientific data is not sufficient, which to some extent affects the result of scientific data clustering. Therefore, the paper proposes a concept of composite text description(CTD) and a CTD-based feature representation method for biomedical scientific data. The method mainly uses different feature weight algorisms to represent candidate features based on two types of data sources respectively, combines and finally strengthens the two feature sets. Experiments show that comparing with traditional methods, the feature representation method is more effective than traditional methods and can significantly improve the performance of biomedcial data clustering.

美欧数据跨境流动治理政策共性内容要素分析与中国启示研究

[目的/意义]通过对美欧数据跨境流动治理政策进行深入调研和内容分析,旨在为我国数据跨境流动治理政策制定和管理实践提供启示与借鉴。[方法/过程]采用文献调研、网络调查和质性文本分析法,基于数据生命周期理论,梳理并分析美欧数据跨境流动治理政策内容,通过构建政策文本分析类目体系,提炼政策共性内容要素。[结果/结论]研究发现,美欧已将数据生命周期思想融入数据跨境流动治理政策,且二者存在诸多共性,体现出数据跨境流动政策制定的总体遵循。基于此,文章结合美欧政策内容共性要素,立足我国国情,围绕数据跨境流动中的“三难选择”问题,分别从保障个人数据权利、平衡数据跨境流动与数据安全、维护数据主权等方面提出对策建议,为我国数据跨境流动治理政策的制定与完善提供思路。

Fast Data Processing of a Polarimeter-Interferometer System on J-TEXT

A method of fast data processing has been developed to rapidly obtain evolution of the electron density profile for a multichannel polarimeter-interferometer system（POLARIS）on J-TEXT. Compared with the Abel inversion method, evolution of the density profile analyzed by this method can quickly offer important information. This method has the advantage of fast calculation speed with the order of ten milliseconds per normal shot and it is capable of processing up to 1 MHz sampled data, which is helpful for studying density sawtooth instability and the disruption between shots. In the duration of a flat-top plasma current of usual ohmic discharges on J-TEXT, shape factor u is ranged from 4 to 5. When the disruption of discharge happens, the density profile becomes peaked and the shape factor u typically decreases to 1.

基于文本数据的灾害信息风险监测与预警集成方法研究

基于灾害信息的文本数据,该文提出一种风险监测与预警集成方法,以提高灾害信息风险监测效率及监测支持预警的针对性。首先,运用八爪鱼数据采集器采集和处理灾害信息的文本数据;其次,运用BTM主题模型构建灾害信息风险监测模型,得到主题热度趋势并确定具有高风险且亟待预警的话题;然后,提出灾害信息风险预警指标体系和预警等级判定方法,计算所确定话题的风险综合指标预警加权值,并结合预警区间判定预警等级;最后,以2021年“7·20”郑州特大暴雨灾害信息的风险监测和预警为例进行实验分析,验证所提方法的可行性。研究表明,该方法可将灾害信息的文本数据贯穿于风险监测与预警两个过程,有助于为监测结果更有针对性地支持预警工作提供参考。

基于MQTT协议的轻量化文本信息分发技术研究

随着卫星通信网络的发展,基于卫星通信网络实现手机、平板电脑等移动通信设备之间的通信成为研究的热点。由于其采用无线电波进行信号传输,且需要经过多个中继站进行信号的转发和处理,导致存在网络延时大、丢包率高以及信道狭窄等问题。针对文本信息在卫星通信网络下的分发过程存在效率低下、可靠性不高的问题,设计一种基于MQTT(Message Queuing Telemetry Transport)协议的轻量化文本信息分发技术。该技术使用MQTT协议作为消息传输协议,在文本信息分发前对MQTT协议进行主题设计、发布订阅机制设计、设备连接设计以及设备心跳设计,确保设备之间的连通性;在文本信息的分发过程中,设计数据校验加密算法、文本信息轻量化处理方法和离线消息存储机制,保证文本信息分发的安全性、可靠性。实验结果表明:相较于传统的基于JSON数据的文本信息分发技术,该技术在提高文本信息分发效率的同时,可确保信息分发的安全性、完整性和稳定性。

基于BERT模型的空管危险源文本数据挖掘

由于危险源与安全隐患在民航安全管理工作中容易出现概念混淆和记录混乱的情况,根据双重预防机制管理规定,需要将两者区分开来。通过在ASIS系统上采集得到空管危险源控制清单作为研究对象,并对其进行相应的文本数据挖掘工作。根据危险源与安全隐患特点构建相应的文本分类模型:首先通过文本清洗、去停用词、Jieba分词等对空管危险源控制清单进行预处理,然后基于BERT模型生成词向量,采用BERT-Base-Chinese预训练模型进行预训练,并对模型进行微调超参数,最后结合Softmax分类器得到分类结果。

基于语义上下文感知的文本数据增强方法研究

在文本分类任务中,数据的质量和数量对分类模型的性能有着重要影响,而在现实场景中获取大规模标记数据往往是昂贵和困难的。数据增强作为一种解决数据匮乏问题的低成本方法,已在各种深度学习和机器学习任务中取得了显著效果。由于文本语言具有离散性,在语义保留的条件下进行数据增强具有一定困难。因此,提出基于语义上下文感知的数据增强方法,采用由WordNet 3.0中的词义定义(Gloss)和预训练模型BERT进行整合的Gloss选择模型,进一步识别上下文中目标词(尤其是多义词)的实际词义;然后根据下一个句子预测策略,将目标词的实际词义与被遮盖目标词的句子结合为一个句子对,使用掩码语言模型对句子对进行预测采样;最后计算语义文本相似度,并在三个基准分类数据集上对文中方法进行验证。实验结果表明,提出的方法在语义保留条件下,与选取的基线数据增强方法相比,在三个数据集的平均准确率指标上都有所提升,证明了文中方法的有效性。

我国数据素养教育研究的发展趋势及优化策略探讨--基于VOSviewer工具文本分析视角

随着网络强国、数字中国战略的全面实施,数据素养教育引发了研究群体的广泛关注,成为深化数据融合应用,释放数据红利的重要手段。本文借助文本分析技术,研究分析中国知网中以“数据素养教育”为主题的1331篇文献,围绕发文量、发文机构、资助基金、关键词、研究主题等构建我国数据素养教育研究知识图谱,阐述我国数据素养教育研究领域的发展脉络及现状,基于资源依赖理论提出我国数据素养教育目前存在的问题,并给出优化对策。

我国民族语言文献文本数字化识别问题——基于OCR及其工具

我国少数民族语言文献数量庞大,文字种类繁多,内容涵盖政治、经济、法律、历史、文学、艺术、宗教、天文、地理、医药等领域,是中华民族文化知识的重要组成部分。构建各民族文献文本数据,使之应用于自然语言处理和人工智能,能有效促进中华优秀传统知识创新性传承,促进知识社会化,是对各民族语言古文献和现代书报刊进行文字识别和文本转换数据构建的基础。国内早期OCR技术虽然解决了几种主要少数民族文字识别的问题,但因字符为非Unicode基本集编码而弃用。当前OCR技术已能较好识别蒙、藏、维、哈、朝等文种文献,但在处理我国汉文与少数民族文字混排图像文本时仍然效果不佳。因此应推进少数民族语言文献OCR识别技术创新。我国少数民族语言文献现行活态文字有十多种,其中非拉丁字系的文字有11种,OCR技术应重点解决这类少数民族语言字系的抄本、刻版和铅字印刷文本,以及汉文与民族文字混排文本的识别问题,研发开放的多功能工具和平台。在此基础上,进一步开展少数民族语言文献文本大规模数据构建,以促进我国语言科学研究和自然语言处理的创新发展。

文本分类算法及其应用场景研究综述

随着大数据时代的到来,互联网中的文本信息迎来了井喷式的增长.文本分类作为自然语言处理中最重要的技术之一,其广泛应用于多个领域,如情感分析、新闻分类、自然语言推理、主题标记、抽取式问答、虚假内容检测等.从传统机器学习分类方法理论的深入到深度学习分类方法探索的兴起,相关研究模型与思路也在不断演变,各类新的方法、数据集和评价指标层出不穷,丰富了文本分类领域的研究,取得了卓越的理论成就和应用效果.尽管如此,新技术不断发展和业务应用场景不断丰富,同时,也为文本分类研究带来了许多新的问题与挑战,如数据约束场景中不均衡数据的文本表征学习、小样本场景下的文本分类等.针对当前研究难题与挑战,本文对文本分类方法进行了系统性调研,并对当前方法在实际应用场景中面临的技术挑战和未来的研究方向进行了综合探讨.具体而言,本文主要综述了七部分内容,分别是:(1)对文本分类技术的相关基础知识进行了全面介绍,包括文本分类的常见符号定义、计算范式和文本预处理技术;(2)对基于传统机器学习的文本分类方法进行了详细总结;同时,为了方便读者针对不同的应用场景选择合适的分类模型,本文对不同分类器擅长处理的文本分类难题及方法优劣进行了总结;(3)对基于新兴深度学习的文本分类方法进行了周详梳理,根据领域内代表性技术的核心思想进行分类,在此基础上对不同类别下的主要方法进行描述,同时对其技术的优劣进行了总结;(4)为了方便读者对文本分类模型的有效性进行验证,针对文本分类技术应用最为广泛的七大场景,本文对相关数据集进行了系统性的总结;(5)本文对不同任务目标下的常用的模型评价方法进行详尽介绍,以便对模型性能进行合理的定量评估;(6)基于上述内容,本文对典型应用场景中不同种类文本分类算法进行了性能总结对比;(7)本文分别从数据约束与模型计算两个层面对当前文本分类技术所面临的挑战和未来的重要研究方向进行了总结.本文通过梳理文本分类研究发展脉络,对涉及的代表性技术进行了详细总结和对比分析,有效填补了文本分类领域前沿技术的应用综述空白.

AIGC在智慧图书馆文本挖掘与知识服务中的应用研究

阐述了生成式人工智能AIGC与LDA主题分析模型在智慧图书馆文本挖掘和知识服务中的应用,创新性地将二者相结合,旨在揭示其在智慧图书馆环境中的应用前景;分析了济南市图书馆大众点评网读者UGC数据,将济南市图书馆读者评论数据进行了文本清洗和词频统计等预处理,随后利用LDA模型对处理过的数据执行了主题建模,识别并分析了数据中的潜在主题,并通过ROSTCM6可视化软件进行了社会网络和语义网络等可视化分析,然后将潜在主题和高频词作为语料投喂训练过的大语言模型(LLM)为读者提供精准的知识服务输出;指出AIGC和LDA主题分析模型相结合在智慧图书馆知识服务中的应用场景可以实现对大规模图书资源的智能化管理和分析,将图书馆的文献信息资源、用户行为数据和用户偏好信息结合数据治理等技术手段,通过AI大语言模型进行训练后可向读者提供定制化的知识解决方案,并为图书馆的决策提供依据;通过实证研究,最后对AIGC在智慧图书馆建设中的应用场景及在文本挖掘中的挑战和未来发展方向进行了探讨。

论刑法中的数据:概念、文本与实践

数字、数据和信息是信息社会的基本概念。在刑法中,数据的定义和范畴决定着刑法对数据保护的范围与力度。在传统意义上,数据与数字紧密相连,但计算机语言的出现使数据部分脱离数字;大量信息以数据方式呈现,但信息只是数据的表现形式之一,数据还包括功能性的程序指令。刑法文本中有显性数据和隐性数据,隐性数据易于被忽视。在司法实践中出现了裁判者泛化数据概念、忽视数据价值、抹杀数据属性差异和同化数据信息等不合理现象。应当在刑法中明确数据概念的基础上,以数据的价值评价侵害数据行为的社会危害性,基于数据的国家秘密、财产、知识产权等属性认定侵害数据行为的性质,明确信息在部分罪名中的独立性,实现刑法对数据权益的保护和对侵害数据行为的打击。