Image Retrieval with Text Manipulation by Local Feature Modification

The demand for image retrieval with text manipulation exists in many fields, such as e-commerce and Internet search. Deep metric learning methods are used by most researchers to calculate the similarity between the query and the candidate image by fusing the global feature of the query image and the text feature. However, the text usually corresponds to the local feature of the query image rather than the global feature. Therefore, in this paper, we propose a framework of image retrieval with text manipulation by local feature modification(LFM-IR) which can focus on the related image regions and attributes and perform modification. A spatial attention module and a channel attention module are designed to realize the semantic mapping between image and text. We achieve excellent performance on three benchmark datasets, namely Color-Shape-Size(CSS), Massachusetts Institute of Technology(MIT) States and Fashion200K(+8.3%, +0.7% and +4.6% in R@1).

A Full Text Retrieval System in a Digital Library Environment

The volume of information being created, generated and stored is huge. Without adequate knowledge of Information Retrieval (IR) methods, the retrieval process for information would be cumbersome and frustrating. Studies have further revealed that IR methods are essential in information centres (for example, Digital Library environment) for storage and retrieval of information. Therefore, with more than one billion people accessing the Internet, and millions of queries being issued on a daily basis, modern Web search engines are facing a problem of daunting scale. The main problem associated with the existing search engines is how to avoid irrelevant information retrieval and to retrieve the relevant ones. In this study, the existing system of library retrieval was studied. Problems associated with them were analyzed in order to address this problem. The concept of existing information retrieval models was studied, and the knowledge gained was used to design a digital library information retrieval system. It was successfully implemented using a real life data. The need for a continuous evaluation of the IR methods for effective and efficient full text retrieval system was recommended.

A Weighted Multi-Layer Analytics Based Model for Emoji Recommendation

The developed system for eye and face detection using Convolutional Neural Networks(CNN)models,followed by eye classification and voice-based assistance,has shown promising potential in enhancing accessibility for individuals with visual impairments.The modular approach implemented in this research allows for a seamless flow of information and assistance between the different components of the system.This research significantly contributes to the field of accessibility technology by integrating computer vision,natural language processing,and voice technologies.By leveraging these advancements,the developed system offers a practical and efficient solution for assisting blind individuals.The modular design ensures flexibility,scalability,and ease of integration with existing assistive technologies.However,it is important to acknowledge that further research and improvements are necessary to enhance the system’s accuracy and usability.Fine-tuning the CNN models and expanding the training dataset can improve eye and face detection as well as eye classification capabilities.Additionally,incorporating real-time responses through sophisticated natural language understanding techniques and expanding the knowledge base of ChatGPT can enhance the system’s ability to provide comprehensive and accurate responses.Overall,this research paves the way for the development of more advanced and robust systems for assisting visually impaired individuals.By leveraging cutting-edge technologies and integrating them into amodular framework,this research contributes to creating a more inclusive and accessible society for individuals with visual impairments.Future work can focus on refining the system,addressing its limitations,and conducting user studies to evaluate its effectiveness and impact in real-world scenarios.

Text retrieval algorithm that decreases confusion

To overcome the problem that the confusion between texts limits the precision in text re- trieval, a new text retrieval algorithm that decrease confusion （DCTR） is proposed. The algorithm constructs the searching template to represent the user＇ s searching intention through positive and negative training. By using the prior probabilities in the template, the supported probability and anti- supported probability of each text in the text library can be estimated for discrimination. The search- ing result can be ranked according to similarities between retrieved texts and the template. The com- plexity of DCTR is close to term frequency and mversed document frequency （TF-IDF）. Its distin- guishing ability to confusable texts could be advanced and the performance of the result would be im- proved with increasing of training times.

Automatic User Goals Identification Based on Anchor Text and Click-Through Data

Understanding the underlying goal behind a user＇s Web query has been proved to be helpful to improve the quality of search. This paper focuses on the problem of automatic identification of query types according to the goals. Four novel entropy-based features extracted from anchor data and click-through data are proposed, and a support vector machines （SVM） classifier is used to identify the user＇s goal based on these features. Experi- mental results show that the proposed entropy-based features are more effective than those reported in previous work. By combin- ing multiple features the goals for more than 97% of the queries studied can be correctly identified. Besides these, this paper reaches the following important conclusions： First, anchor-based features are more effective than click-through-based features; Second, the number of sites is more reliable than the number of links; Third, click-distribution- based features are more effective than session-based ones.

A Text Categorization System with Soft Real-Time Guarantee

In order to provide predictable runtime performante for text categorization （TC） systems, an innovative system design method is proposed for soft real time TC systems. An analyzable mathematical model is established to approximately describe the nonlinear and time-varying TC systems. According to this mathematical model, the feedback control theory is adopted to prove the system＇s stableness and zero steady state error. The experiments result shows that the error of deadline satisfied ratio in the system is kept within 4 of the desired value. And the number of classifiers can be dynamically adjusted by the system itself to save the computa tion resources. The proposed methodology enables the theo retical analysis and evaluation to the TC systems, leading to a high-quality and low cost implementation approach.

E-textbook全文检索

基于电子教材的特殊应用的需求,在传统的web页面全文检索技术基础上,设计了电子教材的全文检索系统.它包含教材文档处理模块、索引服务模块和检索服务模块.根据电子教材的结构需求,定义了索引文件数据结构、文本文件数据结构、索引条目数据结构及结果排序的分数模型.通过系统的实现,为电子教材学习者提供了快速准确的检索服务,提高学习者学习效率.

基于深度学习的图像-文本匹配研究综述

图像-文本匹配任务旨在衡量图像和文本描述之间的相似性,其在桥接视觉和语言中起着至关重要的作用.近年来,图像与句子的全局对齐以及区域与单词的局部对齐研究方面取得了很大的进展.本文对当前先进的研究方法进行分类和描述.具体地,本文将现有方法划分为基于全局特征的图像-文本匹配方法、基于局部特征的图像-文本匹配方法、基于外部知识的图像-文本匹配方法、基于度量学习的图像-文本匹配方法以及多模态预训练模型,对于基于全局特征的图像-文本匹配方法,本文依据流程类型划分为两类:基于嵌入的方法和基于交互的方法;而对于基于局部特征的图像-文本匹配方法,依据其交互模式的不同,则被细分为三类:基于模态内关系建模的方法、基于模态间关系建模的方法以及基于混合交互建模的方法.随后,本文对当前图像-文本匹配任务的相关数据集进行了整理,并对现有方法的实验结果进行分析与总结.最后,对未来研究可能面临的挑战进行了展望.

基于融合矩阵的文本相似度计算实现检索结果聚类

目的/意义弥补医学文本语义表示方面的不足,实现PubMed数据库检索结果聚类。方法/过程采用Jaccard系数和TF-IDF构建融合矩阵方法,建立短语间、文档间、短语与文档内容间的相似性关系融合矩阵,训练聚类算法,将PubMed数据库检索结果集合分组,随后生成类别标签,描述每一类簇文档的含义。结果/结论基于融合矩阵的聚类效果较好,提取出描述类别的高频词能很好地区分类别含义,对检索结果文本聚类任务有效。

利用BERT和覆盖率机制改进的HiNT文本检索模型

为有效提升文本语义检索的准确度,本文针对当前文本检索模型衡量查询和文档的相关性时不能很好地解决文本歧义和一词多义等问题,提出一种基于改进的分层神经匹配模型(hierarchical neural matching model,HiNT)。该模型先对文档的各个段提取关键主题词,然后用基于变换器的双向编码器(bidirectional encoder representations from transformers,BERT)模型将其编码为多个稠密的语义向量,再利用引入覆盖率机制的局部匹配层进行处理,使模型可以根据文档的局部段级别粒度和全局文档级别粒度进行相关性计算,提高检索的准确率。本文提出的模型在MS MARCO和webtext2019zh数据集上与多个检索模型进行对比,取得了最优结果,验证了本文提出模型的有效性。

基于关键实体和文本摘要多特征融合的话题匹配算法

随着网络的快速普及,互联网新闻的数量剧增,在这种情况下,如何有效地找到更加符合特定主题的相关报道成为一个迫切需要解决的问题。针对这一问题,提出了基于关键实体和文本摘要多特征融合的话题匹配算法。首先,使用W2NER模型进行命名实体识别,通过词频、TF-IDF、词的合群性、词词相似度和词句相似度特征,提取关键的实体。其次,使用Pegasus模型进行文本摘要,通过BiLSTM融合关键实体特征与文本摘要特征,得到新闻文本的深层次语义特征。再次,使用交叉注意力机制对待匹配新闻进行特征交互,增进彼此的联系。最后,融合新闻文本的深层次语义特征和文本交互特征,共同参与文本话题匹配的判断。在来自于搜狐的真实数据上进行了不同算法的对比实验,结果表明:所提算法准确率和精确率均与其他算法效果相近,召回率和F1值均有所提升。

基于多粒度语义融合的信息检索方法

信息检索(IR)是一种通过特定的技术和方法组织、处理信息,以满足用户的信息需求的过程。近年来,基于预训练模型的稠密检索方法取得了巨大的成功;然而,这些方法只利用了文本和词语的向量表征计算查询与文档相关度,忽略了它们短语层面间的语义信息。针对该问题,提出一种名为MSIR(Multi-Scale IR)的IR方法。所提方法通过融合查询与文档中多种不同粒度的语义信息提高IR性能。首先,构建查询和文档中词语、短语和文本这3个粒度的语义单元;其次,利用预训练模型对这3个语义单元分别进行编码获得它们的语义表征;最后,利用语义表征计算查询和文档相关度。在Corvid-19、TREC2019和Robust04这3个不同大小的经典数据集上进行了对比实验。与ColBERT(ranking model based on Contextualized late interaction over BERT(Bidirectional Encoder Representation from Transformers))相比,MSIR在Robust04数据集的P@10、P@20、NDCG@10和NDCG@20指标上均实现了约8%的提升,同时在Corvid-19和TREC2019数据集上也取得了一定的改进。实验结果表明,MSIR能够成功融合多种语义粒度,提升检索精度。

基于特征增强和语义相关性匹配的图像文本检索方法

为实现图像文本检索中图像与文本的精确语义连接,提出一种基于特征增强和语义相关性匹配(FESCM)的图像文本检索方法。首先,通过特征增强表示模块,引入多头自注意力机制增强图像区域特征和文本单词特征,以减少冗余信息对图像区域和文本单词对齐的干扰;其次,通过语义相关性匹配模块,不仅利用局部匹配捕获局部显著对象之间的对应相关性,还把图像背景信息融入图像全局特征,利用全局匹配实现精确的全局语义相关性;最后,通过局部匹配分数和全局匹配分数获取图像和文本的最终匹配分数。实验结果表明,基于FESCM的图像文本检索方法在Flickr8k和Flickr30k基准数据集上的召回率总值比扩展的视觉语义嵌入方法分别提升了5.7和7.5个百分点,在MS-COCO数据集比双流层次相似度推理方法提升了3.7个百分点。因此该方法可以有效提高图像文本检索的准确度,实现图像与文本的语义连接。

一种k-NN文本分类器的改进方法

自动文本分类是提高信息利用效率和质量的有效方法。训练文本分布的不均匀会对分类的效果产生负面影响，而在实际中，很难使训练文本的分布达到均匀。针对这一问题，提出了一种改进的k-NN文本分类方法。通过在英文和中文两个文本集的实验表明，改进后的方法不仅分类的准确性有了提高，而且表现出较好的稳定性。

基于ElasticSearch的医疗数据检索系统的设计与实现

随着医疗技术的发展,医疗业务场景越加复杂,由此产生的医疗数据也越来越多,其来源复杂,结构多变,信息冗余,数据不完整。这些特性使得系统在进行检索时,无法快速、有效、精确地查询数据。为了解决这个问题,设计并实现了基于ElasticSearch的医疗数据检索系统。该系统将医疗数据进行标准化,填补缺失值,选取合适的分词算法进行分词,将处理后的数据存入ElasticSearch中,同时使用SpringBoot构建系统应用,消耗多个医疗基础业务系统产生的数据,最终形成统一的医疗数据检索系统,给用户提供便捷、精确的查询服务。

基于完全加权正负关联模式挖掘的越-英跨语言查询译后扩展

主题漂移和词不匹配是自然语言处理中一个难题,文本挖掘与信息检索的结合有助于解决该问题.鉴于此,本文提出一种基于完全加权正负关联模式挖掘的越-英跨语言查询译后扩展算法.该算法采用新的完全加权正负项集支持度和关联度计算方法以及模式评价框架,对初检用户相关反馈文档集挖掘与原查询词相关的正负关联模式,从模式中提取扩展词实现跨语言查询译后扩展.与现有基于伪相关反馈、加权关联模式挖掘的跨语言扩展算法比较,本文算法能有效地减少查询主题漂移和词不匹配问题,提高跨语言信息检索性能;本文模式挖掘方法可用于推荐系统,提高其准确性.

基于知识增强的文本语义匹配模型研究

文本语义匹配模型在信息检索、文本挖掘等领域已经获得了广泛应用。为解决现有模型主要从文本自身角度判断文本之间的语义关系而忽略对外部知识有效利用的问题,本文提出一种新的基于知识增强的文本语义匹配模型,以知识图谱实体作为外部知识,有效建模文本的外部知识信息,并自适应地过滤外部知识中存在的噪声。针对自然语言推理和释义识别两个文本语义匹配任务,与基线方法相比,本文模型在大多数指标上取得了最优效果。研究结果表明,本文模型有助于揭示知识图谱在文本语义匹配任务中的作用,为将知识图谱应用到智能信息服务领域提供了参考。

嵌入共识知识的因果图文检索方法

跨模态图像-文本检索是一项在给定一种模态(如文本)的查询条件下检索另一种模态(如图像)的任务.该任务的关键问题在于如何准确地测量图文两种模态之间的相似性,在减少视觉和语言这两种异构模态之间的视觉语义差异中起着至关重要的作用.传统的检索范式依靠深度学习提取图像和文本的特征表示,并将其映射到一个公共表示空间中进行匹配.然而,这种方法更多地依赖数据表面的相关关系,无法挖掘数据背后真实的因果关系,在高层语义信息的表示和可解释性方面面临着挑战.为此,在深度学习的基础上引入因果推断和嵌入共识知识,提出嵌入共识知识的因果图文检索方法.具体而言,将因果干预引入视觉特征提取模块,通过因果关系替换相关关系学习常识因果视觉特征,并与原始视觉特征进行连接得到最终的视觉特征表示.为解决本方法文本特征表示不足的问题,采用更强大的文本特征提取模型BERT(Bidirectional encoder representations from transformers,双向编码器表示),并且嵌入两种模态数据之间共享的共识知识对图文特征进行共识级的表示学习.在MS-COCO数据集以及MS-COCO到Flickr30k上的跨数据集实验,证明了本文方法可以在双向图文检索任务上实现召回率和平均召回率的一致性改进.

基于组合优化的遥感图文检索轻量化

针对现有基于图网络的遥感图文检索模型存在的海量参数、模型时效性低、存储空间需求大等问题,提出一种基于组合优化的遥感图文检索轻量化方法。从模型架构角度,设计基于跨阶段融合的轻量化卷积模块精简图文检索模型的参数;从数值量化角度,设计图网络混合精度训练与量化推理策略提升模型推理速度。在多个遥感检索数据集上的实验结果表明,该方法在检索精度基本不下降的条件下,总参数量、浮点运算量相比于典型方法降低60%以上。

基于要素关联图的汉越跨语言事件检索方法

汉越跨语言事件检索旨在根据输入的中文事件查询短语,检索出相关的越南语新闻事件文档。由于查询文档的新闻文本较长,中文事件查询短语与越南语的查询文档长度不一,表达差异较大,且查询文档中往往会包含大量与其描述的核心事件无关的噪声文本,现有的模型不能很好地捕获事件匹配特征,匹配效果欠佳。基于此,文中提出基于要素关联图的汉越跨语言事件检索方法。首先,预训练一个汉越双语词嵌入来解决跨语言问题;然后,抽取查询文档中的关键信息(关键词和实体)以构建要素关联图;最后,通过引入一个图编码器对构建的要素图进行编码,生成结构化的事件信息来增强传统的事件检索模型。实验结果表明文中提出的方法优于传统的基线方法。