Importance-aware 3D volume visualization for medical content-based image retrieval-a preliminary study

Background A medical content-based image retrieval(CBIR)system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image.CBIR is widely used in evidence-based diagnosis,teaching,and research.Although the retrieval accuracy has largely improved,there has been limited development toward visualizing important image features that indicate the similarity of retrieved images.Despite the prevalence of 3D volumetric data in medical imaging such as computed tomography(CT),current CBIR systems still rely on 2D cross-sectional views for the visualization of retrieved images.Such 2D visualization requires users to browse through the image stacks to confirm the similarity of the retrieved images and often involves mental reconstruction of 3D information,including the size,shape,and spatial relations of multiple structures.This process is time-consuming and reliant on users'experience.Methods In this study,we proposed an importance-aware 3D volume visualization method.The rendering parameters were automatically optimized to maximize the visibility of important structures that were detected and prioritized in the retrieval process.We then integrated the proposed visualization into a CBIR system,thereby complementing the 2D cross-sectional views for relevance feedback and further analyses.Results Our preliminary results demonstrate that 3D visualization can provide additional information using multimodal positron emission tomography and computed tomography(PETCT)images of a non-small cell lung cancer dataset.

A Visual Indoor Localization Method Based on Efficient Image Retrieval

The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor localization technologies generally used scene-specific 3D representations or were trained on specific datasets, making it challenging to balance accuracy and cost when applied to new scenes. Addressing this issue, this paper proposed a universal indoor visual localization method based on efficient image retrieval. Initially, a Multi-Layer Perceptron (MLP) was employed to aggregate features from intermediate layers of a convolutional neural network, obtaining a global representation of the image. This approach ensured accurate and rapid retrieval of reference images. Subsequently, a new mechanism using Random Sample Consensus (RANSAC) was designed to resolve relative pose ambiguity caused by the essential matrix decomposition based on the five-point method. Finally, the absolute pose of the queried user image was computed, thereby achieving indoor user pose estimation. The proposed indoor localization method was characterized by its simplicity, flexibility, and excellent cross-scene generalization. Experimental results demonstrated a positioning error of 0.09 m and 2.14° on the 7Scenes dataset, and 0.15 m and 6.37° on the 12Scenes dataset. These results convincingly illustrated the outstanding performance of the proposed indoor localization method.

Toward Fine-grained Image Retrieval with Adaptive Deep Learning for Cultural Heritage Image

Fine-grained image classification is a challenging research topic because of the high degree of similarity among categories and the high degree of dissimilarity for a specific category caused by different poses and scales.A cul-tural heritage image is one of thefine-grained images because each image has the same similarity in most cases.Using the classification technique,distinguishing cultural heritage architecture may be difficult.This study proposes a cultural heri-tage content retrieval method using adaptive deep learning forfine-grained image retrieval.The key contribution of this research was the creation of a retrieval mod-el that could handle incremental streams of new categories while maintaining its past performance in old categories and not losing the old categorization of a cul-tural heritage image.The goal of the proposed method is to perform a retrieval task for classes.Incremental learning for new classes was conducted to reduce the re-training process.In this step,the original class is not necessary for re-train-ing which we call an adaptive deep learning technique.Cultural heritage in the case of Thai archaeological site architecture was retrieved through machine learn-ing and image processing.We analyze the experimental results of incremental learning forfine-grained images with images of Thai archaeological site architec-ture from world heritage provinces in Thailand,which have a similar architecture.Using afine-grained image retrieval technique for this group of cultural heritage images in a database can solve the problem of a high degree of similarity among categories and a high degree of dissimilarity for a specific category.The proposed method for retrieving the correct image from a database can deliver an average accuracy of 85 percent.Adaptive deep learning forfine-grained image retrieval was used to retrieve cultural heritage content,and it outperformed state-of-the-art methods infine-grained image retrieval.

Triplet Label Based Image Retrieval Using Deep Learning in Large Database

Recent days,Image retrieval has become a tedious process as the image database has grown very larger.The introduction of Machine Learning(ML)and Deep Learning(DL)made this process more comfortable.In these,the pair-wise label similarity is used tofind the matching images from the database.But this method lacks of limited propose code and weak execution of misclassified images.In order to get-rid of the above problem,a novel triplet based label that incorporates context-spatial similarity measure is proposed.A Point Attention Based Triplet Network(PABTN)is introduced to study propose code that gives maximum discriminative ability.To improve the performance of ranking,a corre-lating resolutions for the classification,triplet labels based onfindings,a spatial-attention mechanism and Region Of Interest(ROI)and small trial information loss containing a new triplet cross-entropy loss are used.From the experimental results,it is shown that the proposed technique exhibits better results in terms of mean Reciprocal Rank(mRR)and mean Average Precision(mAP)in the CIFAR-10 and NUS-WIPE datasets.

Image Retrieval Based on Vision Transformer and Masked Learning

Deep convolutional neural networks(DCNNs)are widely used in content-based image retrieval(CBIR)because of the advantages in image feature extraction.However,the training of deep neural networks requires a large number of labeled data,which limits the application.Self-supervised learning is a more general approach in unlabeled scenarios.A method of fine-tuning feature extraction networks based on masked learning is proposed.Masked autoencoders(MAE)are used in the fine-tune vision transformer(ViT)model.In addition,the scheme of extracting image descriptors is discussed.The encoder of the MAE uses the ViT to extract global features and performs self-supervised fine-tuning by reconstructing masked area pixels.The method works well on category-level image retrieval datasets with marked improvements in instance-level datasets.For the instance-level datasets Oxford5k and Paris6k,the retrieval accuracy of the base model is improved by 7%and 17%compared to that of the original model,respectively.

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image.The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered,and dissimilar images are separated in the low embedding space.Previous works primarily focused on defining local structure loss functions like triplet loss,pairwise loss,etc.However,training via these approaches takes a long training time,and they have poor accuracy.Additionally,representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes.This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues.In the proposed work,class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters.And other instances are treated as negatives within the same cluster.Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space.The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results(85.38%recall@1 for CARS-196%and 70.13%recall@1 for CUB-200)compared to other existing methods.

OSAP‐Loss:Efficient optimization of average precision via involving samples after positive ones towards remote sensing image retrieval

In existing remote sensing image retrieval(RSIR)datasets,the number of images among different classes varies dramatically,which leads to a severe class imbalance problem.Some studies propose to train the model with the ranking‐based metric(e.g.,average precision[AP]),because AP is robust to class imbalance.However,current AP‐based methods overlook an important issue:only optimising samples ranking before each positive sample,which is limited by the definition of AP and is prone to local optimum.To achieve global optimisation of AP,a novel method,namely Optimising Samples after positive ones&AP loss(OSAP‐Loss)is proposed in this study.Specifically,a novel superior ranking function is designed to make the AP loss differentiable while providing a tighter upper bound.Then,a novel loss called Optimising Samples after Positive ones(OSP)loss is proposed to involve all positive and negative samples ranking after each positive one and to provide a more flexible optimisation strategy for each sample.Finally,a graphics processing unit memory‐free mechanism is developed to thoroughly address the non‐decomposability of AP optimisation.Extensive experimental results on RSIR as well as conventional image retrieval datasets show the superiority and competitive performance of OSAP‐Loss compared to the state‐of‐the‐art.

Refined Sparse Representation Based Similar Category Image Retrieval

Given one specific image,it would be quite significant if humanity could simply retrieve all those pictures that fall into a similar category of images.However,traditional methods are inclined to achieve high-quality retrieval by utilizing adequate learning instances,ignoring the extraction of the image’s essential information which leads to difficulty in the retrieval of similar category images just using one reference image.Aiming to solve this problem above,we proposed in this paper one refined sparse representation based similar category image retrieval model.On the one hand,saliency detection and multi-level decomposition could contribute to taking salient and spatial information into consideration more fully in the future.On the other hand,the cross mutual sparse coding model aims to extract the image’s essential feature to the maximumextent possible.At last,we set up a database concluding a large number of multi-source images.Adequate groups of comparative experiments show that our method could contribute to retrieving similar category images effectively.Moreover,adequate groups of ablation experiments show that nearly all procedures play their roles,respectively.

Secure Content Based Image Retrieval Scheme Based on Deep Hashing and Searchable Encryption

To solve the problem that the existing ciphertext domain image retrieval system is challenging to balance security,retrieval efficiency,and retrieval accuracy.This research suggests a searchable encryption and deep hashing-based secure image retrieval technique that extracts more expressive image features and constructs a secure,searchable encryption scheme.First,a deep learning framework based on residual network and transfer learn-ing model is designed to extract more representative image deep features.Secondly,the central similarity is used to quantify and construct the deep hash sequence of features.The Paillier homomorphic encryption encrypts the deep hash sequence to build a high-security and low-complexity searchable index.Finally,according to the additive homomorphic property of Paillier homomorphic encryption,a similarity measurement method suitable for com-puting in the retrieval system’s security is ensured by the encrypted domain.The experimental results,which were obtained on Web Image Database from the National University of Singapore(NUS-WIDE),Microsoft Common Objects in Context(MS COCO),and ImageNet data sets,demonstrate the system’s robust security and precise retrieval,the proposed scheme can achieve efficient image retrieval without revealing user privacy.The retrieval accuracy is improved by at least 37%compared to traditional hashing schemes.At the same time,the retrieval time is saved by at least 9.7%compared to the latest deep hashing schemes.

A Content-Based Medical Image Retrieval Method Using Relative Difference-Based Similarity Measure

Content-based medical image retrieval(CBMIR)is a technique for retrieving medical images based on automatically derived image features.There are many applications of CBMIR,such as teaching,research,diagnosis and electronic patient records.Several methods are applied to enhance the retrieval performance of CBMIR systems.Developing new and effective similarity measure and features fusion methods are two of the most powerful and effective strategies for improving these systems.This study proposes the relative difference-based similarity measure(RDBSM)for CBMIR.The new measure was first used in the similarity calculation stage for the CBMIR using an unweighted fusion method of traditional color and texture features.Furthermore,the study also proposes a weighted fusion method for medical image features extracted using pre-trained convolutional neural networks(CNNs)models.Our proposed RDBSM has outperformed the standard well-known similarity and distance measures using two popular medical image datasets,Kvasir and PH2,in terms of recall and precision retrieval measures.The effectiveness and quality of our proposed similarity measure are also proved using a significant test and statistical confidence bound.

Image Retrieval with Text Manipulation by Local Feature Modification

The demand for image retrieval with text manipulation exists in many fields, such as e-commerce and Internet search. Deep metric learning methods are used by most researchers to calculate the similarity between the query and the candidate image by fusing the global feature of the query image and the text feature. However, the text usually corresponds to the local feature of the query image rather than the global feature. Therefore, in this paper, we propose a framework of image retrieval with text manipulation by local feature modification(LFM-IR) which can focus on the related image regions and attributes and perform modification. A spatial attention module and a channel attention module are designed to realize the semantic mapping between image and text. We achieve excellent performance on three benchmark datasets, namely Color-Shape-Size(CSS), Massachusetts Institute of Technology(MIT) States and Fashion200K(+8.3%, +0.7% and +4.6% in R@1).

Fusion of Hash-Based Hard and Soft Biometrics for Enhancing Face Image Database Search and Retrieval

The utilization of digital picture search and retrieval has grown substantially in numerous fields for different purposes during the last decade,owing to the continuing advances in image processing and computer vision approaches.In multiple real-life applications,for example,social media,content-based face picture retrieval is a well-invested technique for large-scale databases,where there is a significant necessity for reliable retrieval capabilities enabling quick search in a vast number of pictures.Humans widely employ faces for recognizing and identifying people.Thus,face recognition through formal or personal pictures is increasingly used in various real-life applications,such as helping crime investigators retrieve matching images from face image databases to identify victims and criminals.However,such face image retrieval becomes more challenging in large-scale databases,where traditional vision-based face analysis requires ample additional storage space than the raw face images already occupied to store extracted lengthy feature vectors and takes much longer to process and match thousands of face images.This work mainly contributes to enhancing face image retrieval performance in large-scale databases using hash codes inferred by locality-sensitive hashing(LSH)for facial hard and soft biometrics as(Hard BioHash)and(Soft BioHash),respectively,to be used as a search input for retrieving the top-k matching faces.Moreover,we propose the multi-biometric score-level fusion of both face hard and soft BioHashes(Hard-Soft BioHash Fusion)for further augmented face image retrieval.The experimental outcomes applied on the Labeled Faces in the Wild(LFW)dataset and the related attributes dataset(LFW-attributes),demonstrate that the retrieval performance of the suggested fusion approach(Hard-Soft BioHash Fusion)significantly improved the retrieval performance compared to solely using Hard BioHash or Soft BioHash in isolation,where the suggested method provides an augmented accuracy of 87%when executed on 1000 specimens and 77%on 5743 samples.These results remarkably outperform the results of the Hard BioHash method by(50%on the 1000 samples and 30%on the 5743 samples),and the Soft BioHash method by(78%on the 1000 samples and 63%on the 5743 samples).

An Angle Structure Descriptor for Image Retrieval

This paper presents an efficient image feature representation method, namely angle structure descriptor(ASD), which is built based on the angle structures of images. According to the diversity in directions, angle structures are defined in local blocks. Combining color information in HSV color space, we use angle structures to detect images. The internal correlations between neighboring pixels in angle structures are explored to form a feature vector. With angle structures as bridges, ASD extracts image features by integrating multiple information as a whole, such as color, texture, shape and spatial layout information. In addition, the proposed algorithm is efficient for image retrieval without any clustering implementation or model training. Experimental results demonstrate that ASD outperforms the other related algorithms.

Deep image retrieval using artificial neural network interpolation and indexing based on similarity measurement

In content-based image retrieval(CBIR),primitive image signatures are critical because they represent the visual characteristics.Image signatures,which are algorithmically descriptive and accurately recognized visual components,are used to appropriately index and retrieve comparable results.To differentiate an image in the category of qualifying contender,feature vectors must have image information's like colour,objects,shape,spatial viewpoints.Previous methods such as sketch-based image retrieval by salient contour(SBIR)and greedy learning of deep Boltzmann machine(GDBM)used spatial information to distinguish between image categories.This requires interest points and also feature analysis emerged image detection problems.Thus,a proposed model to overcome this issue and predict the repeating pattern as well as series of pixels that conclude similarity has been necessary.In this study,a technique called CBIR-similarity measure via artificial neural network interpolation(CBIR-SMANN)has been presented.By collecting datasets,the images are resized then subject to Gaussian filtering in the pre-processing stage,then by permitting them to the Hessian detector,the interesting points are gathered.Based on Skewness,mean,kurtosis and standard deviation features were extracted then given to ANN for interpolation.Interpolated results are stored in a database for retrieval.In the testing stage,the query image was inputted that is subjected to pre-processing,and feature extraction was then fed to the similarity measurement function.Thus,ANN helps to get similar images from the database.CBIR-SMANN have been implemented in the python tool and then evaluated for its performance.Results show that CBIR-SMANN exhibited a high recall value of 78%with a minimum retrieval time of 980 ms.This showed the supremacy of the proposed model was comparatively greater than the previous ones.

An Efficient Content-Based Image Retrieval System Using kNN and Fuzzy Mathematical Algorithm

The implementation of content-based image retrieval(CBIR)mainly depends on two key technologies:image feature extraction and image feature matching.In this paper,we extract the color features based on Global Color Histogram(GCH)and texture features based on Gray Level Co-occurrence Matrix(GLCM).In order to obtain the effective and representative features of the image,we adopt the fuzzy mathematical algorithm in the process of color feature extraction and texture feature extraction respectively.And we combine the fuzzy color feature vector with the fuzzy texture feature vector to form the comprehensive fuzzy feature vector of the image according to a certain way.Image feature matching mainly depends on the similarity between two image feature vectors.In this paper,we propose a novel similarity measure method based on k-Nearest Neighbors(kNN)and fuzzy mathematical algorithm(SBkNNF).Finding out the k nearest neighborhood images of the query image from the image data set according to an appropriate similarity measure method.Using the k similarity values between the query image and its k neighborhood images to constitute the new k-dimensional fuzzy feature vector corresponding to the query image.And using the k similarity values between the retrieved image and the k neighborhood images of the query image to constitute the new k-dimensional fuzzy feature vector corresponding to the retrieved image.Calculating the similarity between the two kdimensional fuzzy feature vector according to a certain fuzzy similarity algorithm to measure the similarity between the query image and the retrieved image.Extensive experiments are carried out on three data sets:WANG data set,Corel-5k data set and Corel-10k data set.The experimental results show that the outperforming retrieval performance of our proposed CBIR system with the other CBIR systems.

Integrating Color and Spatial Feature for Content-Based Image Retrieval

In this paper, we present a novel and efficient scheme for extracting, indexing and retrieving color images. Our motivation was to reduce the space overhead of partition\|based approaches taking advantage of the fact that only a relatively low number of distinct values of a particular visual feature is present in most images. To extract color feature and build indices into our image database we take into consideration factors such as human color perception and perceptual range, and the image is partitioned into a set of regions by using a simple classifying scheme. The compact color feature vector and the spatial color histogram, which are extracted from the seqmented image region, are used for representing the color and spatial information in the image. We have also developed the region\|based distance measures to compare the similarity of two images. Extensive tests on a large image collection were conducted to demonstrate the effectiveness of the proposed approach.

Content-based image retrieval applied to BI-RADS tissue classification in screening mammography

AIM:To present a content-based image retrieval(CBIR) system that supports the classification of breast tissue density and can be used in the processing chain to adapt parameters for lesion segmentation and classification.METHODS:Breast density is characterized by image texture using singular value decomposition(SVD) and histograms.Pattern similarity is computed by a support vector machine(SVM) to separate the four BI-RADS tissue categories.The crucial number of remaining singular values is varied(SVD),and linear,radial,and polynomial kernels are investigated(SVM).The system is supported by a large reference database for training and evaluation.Experiments are based on 5-fold cross validation.RESULTS:Adopted from DDSM,MIAS,LLNL,and RWTH datasets,the reference database is composed of over 10000 various mammograms with unified and reliable ground truth.An average precision of 82.14% is obtained using 25 singular values(SVD),polynomial kernel and the one-against-one(SVM).CONCLUSION:Breast density characterization using SVD allied with SVM for image retrieval enable the development of a CBIR system that can effectively aid radiologists in their diagnosis.

Color-based Image Retrieval Using Sub-range Cumulative Histogram

１．ＩｎｔｒｏｄｕｃｔｉｏｎＲｅｃｅｎｔｌｙ，ａｌａｒｇｅｎｕｍｂｅｒｏｆｄｉｇｉｔａｌｉｍａｇｅｄａｔａｂａｓｅｓａｒｅｃｒｅａｔｉｎｇａｎｄｇｅｎｅｒａｔｉｎｇａｔａｖｅｒｙｆａｓｔｒａｔｅｄｕｅｔｏｔｈｅｐｒｏｇｒｅｓｏｆｎｅｗｔｅｃｈｎｏｌ...

An Encrypted Image Retrieval Method Based on SimHash in Cloud Computing

With the massive growth of images data and the rise of cloud computing that can provide cheap storage space and convenient access,more and more users store data in cloud server.However,how to quickly query the expected data with privacy-preserving is still a challenging in the encryption image data retrieval.Towards this goal,this paper proposes a ciphertext image retrieval method based on SimHash in cloud computing.Firstly,we extract local feature of images,and then cluster the features by K-means.Based on it,the visual word codebook is introduced to represent feature information of images,which hashes the codebook to the corresponding fingerprint.Finally,the image feature vector is generated by SimHash searchable encryption feature algorithm for similarity retrieval.Extensive experiments on two public datasets validate the effectiveness of our method.Besides,the proposed method outperforms one popular searchable encryption,and the results are competitive to the state-of-the-art.

Phishing Detection with Image Retrieval Based on Improved Texton Correlation Descriptor

Anti-detection is becoming as an emerging challenge for anti-phishing.This paper solves the threats of anti-detection from the threshold setting condition.Enough webpages are considered to complicate threshold setting condition when the threshold is settled.According to the common visual behavior which is easily attracted by the salient region of webpages,image retrieval methods based on texton correlation descriptor(TCD)are improved to obtain enough webpages which have similarity in the salient region for the images of webpages.There are two steps for improving TCD which has advantage of recognizing the salient region of images:(1)This paper proposed Weighted Euclidean Distance based on neighborhood location(NLW-Euclidean distance)and double cross windows,and combine them to solve the problems in TCD;(2)Space structure is introduced to map the image set to Euclid space so that similarity relation among images can be used to complicate threshold setting conditions.Experimental results show that the proposed method can improve the effectiveness of anti-phishing and make the system more stable,and significantly reduce the possibilities of being hacked to be used as mining systems for blockchain.