Cross-Modal Hashing Retrieval Based on Deep Residual Network

In the era of big data rich inWe Media,the single mode retrieval system has been unable to meet people’s demand for information retrieval.This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes:A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network(CMHR-DRN).The model construction is divided into two stages:The first stage is the feature extraction of different modal data,including the use of Deep Residual Network(DRN)to extract the image features,using the method of combining TF-IDF with the full connection network to extract the text features,and the obtained image and text features used as the input of the second stage.In the second stage,the image and text features are mapped into Hash functions by supervised learning,and the image and text features are mapped to the common binary Hamming space.In the process of mapping,the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval.In training the model,adaptive moment estimation(Adam)is used to calculate the adaptive learning rate of each parameter,and the stochastic gradient descent(SGD)is calculated to obtain the minimum loss function.The whole training process is completed on Caffe deep learning framework.Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH,CMDN and CMSSH.

ViT2CMH:Vision Transformer Cross-Modal Hashing for Fine-Grained Vision-Text Retrieval

In recent years,the development of deep learning has further improved hash retrieval technology.Most of the existing hashing methods currently use Convolutional Neural Networks(CNNs)and Recurrent Neural Networks(RNNs)to process image and text information,respectively.This makes images or texts subject to local constraints,and inherent label matching cannot capture finegrained information,often leading to suboptimal results.Driven by the development of the transformer model,we propose a framework called ViT2CMH mainly based on the Vision Transformer to handle deep Cross-modal Hashing tasks rather than CNNs or RNNs.Specifically,we use a BERT network to extract text features and use the vision transformer as the image network of the model.Finally,the features are transformed into hash codes for efficient and fast retrieval.We conduct extensive experiments on Microsoft COCO(MS-COCO)and Flickr30K,comparing with baselines of some hashing methods and image-text matching methods,showing that our method has better performance.

TECMH:Transformer-Based Cross-Modal Hashing For Fine-Grained Image-Text Retrieval

In recent years,cross-modal hash retrieval has become a popular research field because of its advantages of high efficiency and low storage.Cross-modal retrieval technology can be applied to search engines,crossmodalmedical processing,etc.The existing main method is to use amulti-label matching paradigm to finish the retrieval tasks.However,such methods do not use fine-grained information in the multi-modal data,which may lead to suboptimal results.To avoid cross-modal matching turning into label matching,this paper proposes an end-to-end fine-grained cross-modal hash retrieval method,which can focus more on the fine-grained semantic information of multi-modal data.First,the method refines the image features and no longer uses multiple labels to represent text features but uses BERT for processing.Second,this method uses the inference capabilities of the transformer encoder to generate global fine-grained features.Finally,in order to better judge the effect of the fine-grained model,this paper uses the datasets in the image text matching field instead of the traditional label-matching datasets.This article experiment on Microsoft COCO(MS-COCO)and Flickr30K datasets and compare it with the previous classicalmethods.The experimental results show that this method can obtain more advanced results in the cross-modal hash retrieval field.

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Multimodal sentiment analysis aims to understand people’s emotions and opinions from diverse data.Concate-nating or multiplying various modalities is a traditional multi-modal sentiment analysis fusion method.This fusion method does not utilize the correlation information between modalities.To solve this problem,this paper proposes amodel based on amulti-head attention mechanism.First,after preprocessing the original data.Then,the feature representation is converted into a sequence of word vectors and positional encoding is introduced to better understand the semantic and sequential information in the input sequence.Next,the input coding sequence is fed into the transformer model for further processing and learning.At the transformer layer,a cross-modal attention consisting of a pair of multi-head attention modules is employed to reflect the correlation between modalities.Finally,the processed results are input into the feedforward neural network to obtain the emotional output through the classification layer.Through the above processing flow,the model can capture semantic information and contextual relationships and achieve good results in various natural language processing tasks.Our model was tested on the CMU Multimodal Opinion Sentiment and Emotion Intensity(CMU-MOSEI)and Multimodal EmotionLines Dataset(MELD),achieving an accuracy of 82.04% and F1 parameters reached 80.59% on the former dataset.

Cross-Modal Consistency with Aesthetic Similarity for Multimodal False Information Detection

With the explosive growth of false information on social media platforms, the automatic detection of multimodalfalse information has received increasing attention. Recent research has significantly contributed to multimodalinformation exchange and fusion, with many methods attempting to integrate unimodal features to generatemultimodal news representations. However, they still need to fully explore the hierarchical and complex semanticcorrelations between different modal contents, severely limiting their performance detecting multimodal falseinformation. This work proposes a two-stage detection framework for multimodal false information detection,called ASMFD, which is based on image aesthetic similarity to segment and explores the consistency andinconsistency features of images and texts. Specifically, we first use the Contrastive Language-Image Pre-training(CLIP) model to learn the relationship between text and images through label awareness and train an imageaesthetic attribute scorer using an aesthetic attribute dataset. Then, we calculate the aesthetic similarity betweenthe image and related images and use this similarity as a threshold to divide the multimodal correlation matrixinto consistency and inconsistencymatrices. Finally, the fusionmodule is designed to identify essential features fordetectingmultimodal false information. In extensive experiments on four datasets, the performance of the ASMFDis superior to state-of-the-art baseline methods.

基于线性同态hash和秘密分享的高效可验证聚合方案

针对目前联邦学习可验证聚合方案存在用户通信开销过大、无法容忍用户退出以及用户退出导致验证效率降低的问题,提出了一种基于线性同态hash和秘密分享的高效可验证聚合方案(LHSSEVA)。首先,采用线性同态hash和同态承诺实现聚合结果的可验证性,保证验证信息通信开销与模型维度无关,同时防止服务器通过伪造聚合hash欺骗用户接受错误聚合结果;然后基于椭圆曲线离散对数问题及其同态性保护输入的隐私,同时保证验证的正确性;接着通过融入秘密分享使验证过程可以容忍用户随时退出,并确保用户退出不会导致验证效率降低;最后理论分析证明了方案的正确性、可靠性和隐私性。仿真实验结果表明了方案的可行性和高效性,与VeriFL方案相比,具有更低的计算和通信开销,特别是存在用户退出时,显著提高了验证效率,具有更强的退出容忍性。

A Review of Image Steganography Based on Multiple Hashing Algorithm

Steganography is a technique for hiding secret messages while sending and receiving communications through a cover item.From ancient times to the present,the security of secret or vital information has always been a significant problem.The development of secure communication methods that keep recipient-only data transmissions secret has always been an area of interest.Therefore,several approaches,including steganography,have been developed by researchers over time to enable safe data transit.In this review,we have discussed image steganography based on Discrete Cosine Transform(DCT)algorithm,etc.We have also discussed image steganography based on multiple hashing algorithms like the Rivest–Shamir–Adleman(RSA)method,the Blowfish technique,and the hash-least significant bit(LSB)approach.In this review,a novel method of hiding information in images has been developed with minimal variance in image bits,making our method secure and effective.A cryptography mechanism was also used in this strategy.Before encoding the data and embedding it into a carry image,this review verifies that it has been encrypted.Usually,embedded text in photos conveys crucial signals about the content.This review employs hash table encryption on the message before hiding it within the picture to provide a more secure method of data transport.If the message is ever intercepted by a third party,there are several ways to stop this operation.A second level of security process implementation involves encrypting and decrypting steganography images using different hashing algorithms.

基于word-hashing的DGA僵尸网络深度检测模型

针对使用域名生成算法(DGA)僵尸网络隐蔽性强,传统检测算法特征提取复杂的问题,提出一种无需提取具体特征的深度学习模型DGA域名检测方法.首先基于word-hashing将所有域名转用二元语法字符串表示,利用词袋模型把域名映射到高维向量空间.然后利用5层深度神经网络对转换为高维向量的域名进行训练分类检测.通过深度模型,能够从训练数据中发现不同层次抽象的隐藏模式和特征,而这些模式和特征使用传统的统计方法大多是无法发现的.实验中使用了10万条DGA域名和10万条合法域名作为样本,与基于自然语言特征分类算法进行对比实验.实验结果表明该深度模型对DGA域名检测准确率达到97.23%,比基于自然语言特征分类算法得到的检测准确率高3.7%.

基于直方图量化和混沌系统的感知图像Hashing算法

研究了基于图像灰度级压缩的直方图差值量化(DQH)技术,并结合混沌系统,提出了一种新的感知图像Hash ing算法。算法首先利用混沌系统把压缩后的图像中各个灰度级的出现概率调制成一个固定长度的中间Hash序列;然后将中间Hash序列经过差值量化和二值量化得到最终的图像Hash序列。仿真结果表明,该算法对JPEG压缩、低通滤波、图像缩放和旋转等操作有良好的鲁棒性,而且混沌系统的引入使算法具有较强的安全性。

Trie Hashing结构平均路径长度分析

针对 Ｗ．Ｌｉｔｗｉｎ提出的 Ｔｒｉｅ Ｈａｓｈｉｎｇ结构的路径长度分析问题，研究并揭示 了该结构所具有的某些新的性质；建立了必要的分析前提．从而给出了 Ｔｒｉｅ Ｈａｓｈｉｎｇ 结构平均路径长度的分析方法。所得估计式仅与外部结点数目有关，理论分析与模拟 实验的结果表明，对于 Ｔｒｉｅ Ｈａｓｈｉｎｇ 结构，文中的分析方法明显优于 Ｋｌｅｉｎ 和 ｗｏｏｄ的类似结果。

动态HASHING算法及其改进

对两种动态散列算法可扩展散列和线形散列进行了研究,提出了改进的动态散列算法。改进算法避免了不必要的溢出桶,散列桶的数量线性增长,避免了因查找键分布异常而出现频繁的桶分裂及桶地址表更新的现象。

Recent development of perceptual image hashing

The easy generation, storage, transmission and reproduction of digital images have caused serious abuse and security problems. Assurance of the rightful ownership, integrity, and authenticity is a major concern to the academia as well as the industry. On the other hand, efficient search of the huge amount of images has become a great challenge. Image hashing is a technique suitable for use in image authentication and content based image retrieval （CBIR）. In this article, we review some representative image hashing techniques proposed in the recent years, with emphases on how to meet the conflicting requirements of perceptual robustness and security. Following a brief introduction to some earlier methods, we focus on a typical two-stage structure and some geometric-distortion resilient techniques. We then introduce two image hashing approaches developed in our own research, and reveal security problems in some existing methods due to the absence of secret keys in certain stage of the image feature extraction, or availability of a large quantity of images, keys, or the hash function to the adversary. More research efforts are needed in developing truly robust and secure image hashing techniques.

机群系统上基于Hashing的多目标串匹配并行算法

基于孙子定理构造均匀的Hash函数并继承Karp-Rabin模式匹配思想,利用"筛选"方法,给出一种机群系统上的多目标串匹配并行算法。通过预处理将字符串映射成惟一的一对整数值,采用比较一对整数值来取代逐个字符比较字符串的方法使得匹配过程快速且比较结果是确定的;"筛选"节省了比较时间。算法分析和实验结果表明该并行算法简明、高效和可扩展。

基于双向Hash链的无线传感网络通信节点自愈算法

无线传感网络中节点数量突增,增大了出现失效节点的概率,会影响数据传输效率,导致次级节点出现失效现象,为此提出基于双向hash链的无线传感网络通信节点自愈算法。分析无线传感网络节点流量过载现象,构建节点失效裁决模型,找出网络中失效节点;利用质心算法确定失效节点具体位置,将双向hash链和节点失效裁决模型结合起来,实现对失效节点的自愈修复。构建WSN拓扑结构,对所提方法展开仿真测试,对比结果表明所提方法的节点拓扑移动距离平均值为63.5 m,网络流量出口带宽值平均值为583 Mbyte/s,节点自愈耗时平均值为14.2 s,证明该方法具有较高的自愈效率,保证了失效节点自愈效果最优、自愈能力最强。

Fast Near-duplicate Image Detection in Riemannian Space by A Novel Hashing Scheme

There is a steep increase in data encoded as symmetric positive definite(SPD)matrix in the past decade.The set of SPD matrices forms a Riemannian manifold that constitutes a half convex cone in the vector space of matrices,which we sometimes call SPD manifold.One of the fundamental problems in the application of SPD manifold is to find the nearest neighbor of a queried SPD matrix.Hashing is a popular method that can be used for the nearest neighbor search.However,hashing cannot be directly applied to SPD manifold due to its non-Euclidean intrinsic geometry.Inspired by the idea of kernel trick,a new hashing scheme for SPD manifold by random projection and quantization in expanded data space is proposed in this paper.Experimental results in large scale nearduplicate image detection show the effectiveness and efficiency of the proposed method.

Robust Image Hashing via Random Gabor Filtering and DWT

Image hashing is a useful multimedia technology for many applications,such as image authentication,image retrieval,image copy detection and image forensics.In this paper,we propose a robust image hashing based on random Gabor filtering and discrete wavelet transform(DWT).Specifically,robust and secure image features are first extracted from the normalized image by Gabor filtering and a chaotic map called Skew tent map,and then are compressed via a single-level 2-D DWT.Image hash is finally obtained by concatenating DWT coefficients in the LL sub-band.Many experiments with open image datasets are carried out and the results illustrate that our hashing is robust,discriminative and secure.Receiver operating characteristic(ROC)curve comparisons show that our hashing is better than some popular image hashing algorithms in classification performance between robustness and discrimination.

Mechanism of Cross-modal Information Influencing Taste

Studies on the integration of cross-modal information with taste perception has been mostly limited to uni-modal level.The cross-modal sensory interaction and the neural network of information processing and its control were not fully explored and the mechanisms remain poorly understood.This mini review investigated the impact of uni-modal and multi-modal information on the taste perception,from the perspective of cognitive status,such as emotion,expectation and attention,and discussed the hypothesis that the cognitive status is the key step for visual sense to exert influence on taste.This work may help researchers better understand the mechanism of cross-modal information processing and further develop neutrally-based artificial intelligent(AI)system.

A Multi-Level Circulant Cross-Modal Transformer for Multimodal Speech Emotion Recognition

Speech emotion recognition,as an important component of humancomputer interaction technology,has received increasing attention.Recent studies have treated emotion recognition of speech signals as a multimodal task,due to its inclusion of the semantic features of two different modalities,i.e.,audio and text.However,existing methods often fail in effectively represent features and capture correlations.This paper presents a multi-level circulant cross-modal Transformer(MLCCT)formultimodal speech emotion recognition.The proposed model can be divided into three steps,feature extraction,interaction and fusion.Self-supervised embedding models are introduced for feature extraction,which give a more powerful representation of the original data than those using spectrograms or audio features such as Mel-frequency cepstral coefficients(MFCCs)and low-level descriptors(LLDs).In particular,MLCCT contains two types of feature interaction processes,where a bidirectional Long Short-term Memory(Bi-LSTM)with circulant interaction mechanism is proposed for low-level features,while a two-stream residual cross-modal Transformer block is appliedwhen high-level features are involved.Finally,we choose self-attention blocks for fusion and a fully connected layer to make predictions.To evaluate the performance of our proposed model,comprehensive experiments are conducted on three widely used benchmark datasets including IEMOCAP,MELD and CMU-MOSEI.The competitive results verify the effectiveness of our approach.

Feature Fusion Multi-View Hashing Based on Random Kernel Canonical Correlation Analysis

Hashing technology has the advantages of reducing data storage and improving the efficiency of the learning system,making it more and more widely used in image retrieval.Multi-view data describes image information more comprehensively than traditional methods using a single-view.How to use hashing to combine multi-view data for image retrieval is still a challenge.In this paper,a multi-view fusion hashing method based on RKCCA(Random Kernel Canonical Correlation Analysis)is proposed.In order to describe image content more accurately,we use deep learning dense convolutional network feature DenseNet to construct multi-view by combining GIST feature or BoW_SIFT(Bag-of-Words model+SIFT feature)feature.This algorithm uses RKCCA method to fuse multi-view features to construct association features and apply them to image retrieval.The algorithm generates binary hash code with minimal distortion error by designing quantization regularization terms.A large number of experiments on benchmark datasets show that this method is superior to other multi-view hashing methods.

Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans

Blindness provides an unparalleled opportunity to study plasticity of the nervous system in humans.Seminal work in this area examined the often dramatic modifications to the visual cortex that result when visual input is completely absent from birth or very early in life（Kupers and Ptito,2014）.More recent studies explored what happens to the visual pathways in the context of acquired blindness.This is particularly relevant as the majority of diseases that cause vision loss occur in the elderly.