Solving Geometry Problems via Feature Learning and Contrastive Learning of Multimodal Data

This paper presents an end-to-end deep learning method to solve geometry problems via feature learning and contrastive learning of multimodal data.A key challenge in solving geometry problems using deep learning is to automatically adapt to the task of understanding single-modal and multimodal problems.Existing methods either focus on single-modal ormultimodal problems,and they cannot fit each other.A general geometry problem solver shouldobviouslybe able toprocess variousmodalproblems at the same time.Inthispaper,a shared feature-learning model of multimodal data is adopted to learn the unified feature representation of text and image,which can solve the heterogeneity issue between multimodal geometry problems.A contrastive learning model of multimodal data enhances the semantic relevance betweenmultimodal features and maps them into a unified semantic space,which can effectively adapt to both single-modal and multimodal downstream tasks.Based on the feature extraction and fusion of multimodal data,a proposed geometry problem solver uses relation extraction,theorem reasoning,and problem solving to present solutions in a readable way.Experimental results show the effectiveness of the method.

Breast non-mass-like lesions on contrast-enhanced ultrasonography: Feature analysis, breast image reporting and data system classification assessment

BACKGROUND Breast non-mass-like lesions(NMLs)account for 9.2%of all breast lesions.The specificity of the ultrasound diagnosis of NMLs is low,and it cannot be objectively classified according to the 5th Edition of the Breast Imaging Reporting and Data System(BI-RADS).Contrast-enhanced ultrasound(CEUS)can help to differentiate and classify breast lesions but there are few studies on NMLs alone.AIM To analyze the features of benign and malignant breast NMLs in grayscale ultrasonography(US),color Doppler flow imaging(CDFI)and CEUS,and to explore the efficacy of the combined diagnosis of NMLs and the effect of CEUS on the BI-RADS classification of NMLs.METHODS A total of 51 breast NMLs verified by pathology were analyzed in our hospital from January 2017 to April 2019.All lesions were examined by US,CDFI and CEUS,and their features from those examinations were analyzed.With pathology as the gold standard,binary logic regression was used to analyze the independent risk factors for malignant breast NMLs,and a regression equation was established to calculate the efficiency of combined diagnosis.Based on the regression equation,the combined diagnostic efficiency of US combined with CEUS(US+CEUS)was determined.The initial BI-RADS-US classification of NMLs was adjusted according to the independent risk factors identified by CEUS,and the diagnostic efficiency of CEUS combined with BI-RADS(CEUS+BI-RADS)was calculated based on the results.ROC curves were drawn to compare the diagnostic values of the three methods,including US,US+CEUS,and CEUS+BI-RADS,for benign and malignant NMLs.RESULTS Microcalcification,enhancement time,enhancement intensity,lesion scope,and peripheral blood vessels were significantly different between benign and malignant NMLs.Among these features,microcalcification,higher enhancement,and lesion scope were identified as independent risk factors for malignant breast NMLs.When US,US+CEUS,and CEUS+BI-RADS were used to identify the benign and malignant breast NMLs,their sensitivity rates were 82.6%,91.3%,and 87.0%,respectively;their specificity rates were 71.4%,89.2%,and 92.9%,respectively;their positive predictive values were 70.4%,87.5%,and 90.9%,respectively;their negative predictive values were 83.3%,92.6%,and 89.7%,respectively;their accuracy rates were 76.5%,90.2%,and 90.2%,respectively;and their corresponding areas under ROC curves were 0.752,0.877 and 0.903,respectively.Z tests showed that the area under the ROC curve of US was statistically smaller than that of US+CEUS and CEUS+BI-RADS,and there was no statistical difference between US+CEUS and CEUS+BI-RADS.CONCLUSION US combined with CEUS can improve diagnostic efficiency for NMLs.The adjustment of the BI-RADS classification according to the features of contrastenhanced US of NMLs enables the diagnostic results to be simple and intuitive,facilitates the management of NMLs,and effectively reduces the incidence of unnecessary biopsy.

A Fusion of Residual Blocks and Stack Auto Encoder Features for Stomach Cancer Classification

Diagnosing gastrointestinal cancer by classical means is a hazardous procedure.Years have witnessed several computerized solutions for stomach disease detection and classification.However,the existing techniques faced challenges,such as irrelevant feature extraction,high similarity among different disease symptoms,and the least-important features from a single source.This paper designed a new deep learning-based architecture based on the fusion of two models,Residual blocks and Auto Encoder.First,the Hyper-Kvasir dataset was employed to evaluate the proposed work.The research selected a pre-trained convolutional neural network(CNN)model and improved it with several residual blocks.This process aims to improve the learning capability of deep models and lessen the number of parameters.Besides,this article designed an Auto-Encoder-based network consisting of five convolutional layers in the encoder stage and five in the decoder phase.The research selected the global average pooling and convolutional layers for the feature extraction optimized by a hybrid Marine Predator optimization and Slime Mould optimization algorithm.These features of both models are fused using a novel fusion technique that is later classified using the Artificial Neural Network classifier.The experiment worked on the HyperKvasir dataset,which consists of 23 stomach-infected classes.At last,the proposed method obtained an improved accuracy of 93.90%on this dataset.Comparison is also conducted with some recent techniques and shows that the proposed method’s accuracy is improved.

Deep-Net:Fine-Tuned Deep Neural Network Multi-Features Fusion for Brain Tumor Recognition

Manual diagnosis of brain tumors usingmagnetic resonance images(MRI)is a hectic process and time-consuming.Also,it always requires an expert person for the diagnosis.Therefore,many computer-controlled methods for diagnosing and classifying brain tumors have been introduced in the literature.This paper proposes a novel multimodal brain tumor classification framework based on two-way deep learning feature extraction and a hybrid feature optimization algorithm.NasNet-Mobile,a pre-trained deep learning model,has been fine-tuned and twoway trained on original and enhancedMRI images.The haze-convolutional neural network(haze-CNN)approach is developed and employed on the original images for contrast enhancement.Next,transfer learning(TL)is utilized for training two-way fine-tuned models and extracting feature vectors from the global average pooling layer.Then,using a multiset canonical correlation analysis(CCA)method,features of both deep learning models are fused into a single feature matrix—this technique aims to enhance the information in terms of features for better classification.Although the information was increased,computational time also jumped.This issue is resolved using a hybrid feature optimization algorithm that chooses the best classification features.The experiments were done on two publicly available datasets—BraTs2018 and BraTs2019—and yielded accuracy rates of 94.8%and 95.7%,respectively.The proposedmethod is comparedwith several recent studies andoutperformed inaccuracy.In addition,we analyze the performance of each middle step of the proposed approach and find the selection technique strengthens the proposed framework.

Similarity Measures of Satellite Images Using an Adaptive Feature Contrast Model

Similarity measurement is one of key operations to retrieve “desired” images from an image database. As a famous psychological similarity measure approach, the Feature Contrast (FC) model is defined as a linear combination of both common and distinct features. In this paper, an adaptive feature contrast (AdaFC) model is proposed to measure similarity between satellite images for image retrieval. In the AdaFC, an adaptive function is used to model a variable role of distinct features in the similarity measurement. Specifically, given some distinct features in a satellite image, e.g., a COAST image, they might play a significant role when the image is compared with an image including different semantics, e.g., a SEA image, and might be trivial when it is compared with a third image including same semantics, e.g., another COAST image. Experimental results on satellite images show that the proposed model can consistently improve similarity retrieval effectiveness of satellite images including multiple geo-objects, for example COAST images.

Macroscopic Contrast of English,Chinese and the Shui Language Syntactic Features and Thought Patterns

This paper is to explore the syntactic features and thought patterns among English,Chinese and the Shui language,discussing this topic from macroscopic,giving a brief contrast and comparison on macroscopic of English,Chinese and the Shui language.

Gastric Tract Disease Recognition Using Optimized Deep Learning Features

Artificial intelligence aids for healthcare have received a great deal of attention.Approximately one million patients with gastrointestinal diseases have been diagnosed via wireless capsule endoscopy(WCE).Early diagnosis facilitates appropriate treatment and saves lives.Deep learning-based techniques have been used to identify gastrointestinal ulcers,bleeding sites,and polyps.However,small lesions may be misclassified.We developed a deep learning-based best-feature method to classify various stomach diseases evident in WCE images.Initially,we use hybrid contrast enhancement to distinguish diseased from normal regions.Then,a pretrained model is fine-tuned,and further training is done via transfer learning.Deep features are extracted from the last two layers and fused using a vector length-based approach.We improve the genetic algorithm using a fitness function and kurtosis to select optimal features that are graded by a classifier.We evaluate a database containing 24,000 WCE images of ulcers,bleeding sites,polyps,and healthy tissue.The cubic support vector machine classifier was optimal;the average accuracy was 99%.

基于非监督对比学习的火星地形特征提取方法

火星表面地形智能识别对火星车自主探测具有重要意义,火星地形图像的特征提取方法目前主要分为传统的浅层视觉特征提取和基于监督学习的深层特征提取2类。找回丢失图像信息、获取大量带标签数据是要解决的关键问题。为此,提出一种基于非监督对比学习的火星地形特征识别方法,通过建立图像字典数据集,用“问询”和“编码”2组神经网络分别将单个图像与“字典”数据集中其他图像进行对比,用相似度泛函作为损失函数对网络进行训练,从而实现对火星地形图像的特征识别。所提方法还具有对训练数据集之外的新类型地形图像识别能力,后续识别分类优越性突出。仿真结果表明:所提方法识别准确率为85.4%,对新类型地形图像的识别准确率为84.5%。

基于特征对比的循环生成对抗网络图像风格转换研究

无监督图像到图像转换任务是在非配对训练数据的情况下学习源域图像到目标域图像的转换.但是,图像风格转换任务依然面临着图像内容丢失、模型坍塌等现象.为了解决上述问题,提出了一种局部特征对比来保持图像内容,通过特征提取器获得多层图像深层特征,使得图像编码器学习到高级语义信息,获得信息更加丰富的图像特征.同时,增加局部特征对比损失来引导特征提取器学习到有利于图像内容生成的特征.实验结果表明,在大多数情况下,所提方法在FID和KID分数方面优于之前的方法,图像生成质量有一定的提升.

结合自我特征和对比学习的推荐模型

针对图神经网络推荐中图卷积在消息传递过程的嵌入表示过平滑和噪声问题,提出一种结合自我特征和对比学习的推荐模型(SfCLRec)。采用预训练-正式训练架构训练模型,首先预训练用户和项目的嵌入表示,通过融合节点自我特征维持节点本身的特征唯一性,并引入层级对比学习任务减少来自高阶邻居节点中的噪声;其次,在正式训练阶段根据评分机制重新构建协同图邻接矩阵;最后,根据最终嵌入得到预测评分。实验结果表明,相较于LightGCN、SimGCL(Simple Graph Contrastive Learning)等现有图神经网络推荐模型,SfCLRec在3个公开数据集ML-latest-small、Last.FM和Yelp中均取得了较好的召回率和归一化折损累计增益(NDCG),验证了SfCLRec的有效性。

融合对比学习和特征选择的入侵检测模型

入侵检测系统可以主动识别恶意流量,是保护网络安全的重要工具.针对网络流量中存在的冗余特征以及现有的入侵检测算法在特征选择过程中存在的不足,提出一种融合对比学习和特征选择的入侵检测模型(contrastive learning and feature selection,CL-FS).利用皮尔逊相关系数(Pearson correlation coefficient,PCCs)对预处理后的网络流量进行相关性分析,过滤掉相似特征;使用自编码器(autoencoder,AE)进行深度特征提取,在提取阶段融入对比学习,减少类间相似性,将提取的新特征和过滤后的特征融合,得到表征能力更强的特征集;利用改进的鸽群算法进行包裹特征选择,根据贝叶斯分类器的性能选择最优特征子集,提高分类精度.在NSL-KDD,UNSW-NB15这2个数据集的实验结果表明,CL-FS模型可以提升分类精度并减少处理时间,在2个数据集上的2分类实验准确率分别为90.45%和88.52%,分类处理时间大约减少为原来的一半.

基于粗到细的多尺度单幅图像去雾方法

为了解决现有图像去雾算法易出现细节纹理丢失、颜色失真或对非均匀浓雾处理不彻底的问题,提出一种基于粗到细的多尺度单幅图像去雾方法。首先,主干网络使用残差特征注意力模块对有雾图像进行特征提取;其次,将不同尺度的输入图像进行卷积预处理,通过多尺度特征融合模块将预处理的浅层特征与主干网络融合;再次,将不同粒度的非对称特征进行有效融合;最后,将浅层信息与深层信息自适应混合输出,通过对比正则损失构建正负样本信息,使得去雾图像更接近无雾图像。实验结果表明,与已有代表性的去雾方法相比,提出的方法能对合成数据集与真实数据集进行有效去雾,在细节保留与色彩还原上优于对比方法。

基于特征波段偏振成像的差异增强伪装目标检测

光谱偏振探测技术利用目标多维度信息,提高伪装目标检测的精度和可靠性。现有的光谱偏振成像系统产生的高维度数据难以实时解算,复杂场景下光谱偏振探测的性能不佳。为此,提出一种基于特征波段偏振成像系统的伪装目标检测算法。针对目标场景筛选特征波段,定制751 nm窄带滤光片结合快照式偏振阵列相机,构建特征波段偏振图像采集系统,实时获取目标图像。提出差异增强和交织序列融合检测算法,设计偏振参数图像,增强特征波段偏振图像的目标对比度;融合差异增强和交织序列映射结果,对目标图像背景噪声进行抑制,进一步突出目标特征;通过阈值分割提取伪装目标。实验结果表明:所提的伪装目标检测算法在不同场景下的综合评价指标F均在0.90以上,检测速度达到20帧/s,实现了复杂场景下伪装目标的快速精准探测。

基于数据与特征增强的自监督图表示学习方法

图表示学习在处理图数据结构中起着非常重要的作用,但它面临着严重依赖于标记信息的挑战。为了克服这一挑战,提出了一种新的自监督图表示学习框架,通过使用对比学习方法,融合原始图的结构与属性以及频谱的高低频信息,在保留节点信息的基础上进行增强。同时,利用残差融合机制和无偏特征增强方法,在保证特征有效性的同时进一步减少增强样本的偏差。此外,在对比部分估计负样本为真的概率,并使用权重来度量负样本的硬度和相似度。通过在3个公开数据集上实验证明,在节点分类的下游任务中表现不仅优于当前最先进的无监督方法,而且还在多数任务中超过了以往的有监督方法。

常规超声联合超声造影对胸膜下肺结核、细菌性肺炎的检查影像特征分析

目的分析常规超声联合超声造影对胸膜下肺结核、细菌性肺炎的检查影像特征。方法选择于本院确诊的胸膜下肺结核或细菌性肺炎患者115例,其中确诊为胸膜下肺结核的患者有78例,将其作为肺结核组;而确诊为细菌性肺炎的患者有37例,将其作为肺炎组。所有患者均行常规超声、超声造影检查,分析采用常规超声、超声造影检查的影像学表现。结果肺结核组78例患者中52例呈楔形,占比66.67%;21例呈类圆形,占比26.92%;5例为不规则形,占比6.41%。肺炎组37例患者中23例为楔形,占比62.16%;14例为类圆形,占比37.84%,将两组患者形态进行对比,无明显统计学差异(P>0.05);肺结核组中出现碎片征的患者有24例,占比30.77%,在肺炎组中有30例,占比81.08%,肺炎组患者中碎片征的占比相较于肺结核组升高(P<0.05);边缘较为规整,后方出现彗星尾征的高回声为瀑布征,肺结核组患者中瀑布征的占比为62.82%,相较于肺炎组患者中的占比16.22%上升(P<0.05);肺炎组患者中有34例患者的病灶为离心型增强模式,占比91.89%,而其余3例则为局部向整体的增强模式;肺结核组患者中有55例为离心型增强模式,占比70.51%,13例患者为向心型增强模式,占比16.67%;10例患者表现为离心型、向心型均存在的增强模式,占比12.82%,将两组患者增强模式占比进行对比,差异有统计学意义(P<0.05);肺结核组患者中共有64例出现坏死区,占比82.05%,肺炎组患者中共有11例出现坏死区,占比29.73%,经比较,肺结核组患者的坏死区占比相较于肺炎组升高(P<0.05);肺结核组患者中有33例呈现为病灶内有许多间断的筛孔样坏死,占比51.56%,而24例则呈现为形态较为规整的大片状坏死,占比37.50%,7例则呈现为病灶内几乎完全坏死,占比10.94%;肺炎组患者中有2例为形态较为规整的大片状坏死,占比18.18%,9例呈现为形态较为规整的小片状坏死,占比81.82%,经对比,两组间有统计学差异(P<0.05);两组患者超声造影下正常肺组织开始增强时间、病灶开始增强时间等指标进行对比,均无明显差异(均P>0.05)。结论肺结核病灶经常规超声检查为“瀑布征”,超声造影检查显示坏死区面积较大,且多为大片状坏死;而常规超声检查细菌性肺炎病灶为“碎片征”,坏死区面积较小,且形态较为规整,以上特点可作为鉴别两种疾病的标志。

基于快速局部对比度和目标特征的星图弱小目标检测算法

为了解决局部对比度方法在用于星图空间目标检测时存在运算量大和去除背景噪声困难的问题,提出了一种基于快速局部对比度和目标特征的方法来检测目标。在对比度计算前、对比度计算中和对比度计算后3个环节,分别提高了算法实时性、对复杂背景的抑制和去除噪声。首先,通过中值滤波去除高频噪声;然后,通过快速局部极大值滤波确定目标区域,通过局部对比度计算抑制背景,突出目标成像特征;最后,根据目标成像特征,设置目标能量分布、目标能量集中和目标能量传递3个特征函数,通过设置特征阈值去除噪声,提取真实目标。实验结果表明,本文所提方法在检测率和时间消耗上均具有优越性,对于信噪比为1.5的目标有95%的检测率,平均耗时仅为某些对比方法的1/30~1/6。本文所提方法更适用于星图复杂背景条件下的目标快速检测,满足星图空间目标检测算法鲁棒性强、实时性高的要求。

基于对比学习的多特征融合戴口罩人脸识别

随着计算机视觉技术应用的发展和智能终端的普及,口罩遮挡人脸识别已成为人物身份信息识别的重要部分。口罩的大面积遮挡对人脸特征的学习带来极大挑战。针对戴口罩人脸特征学习困难这一问题,提出了一种基于对比学习的多特征融合口罩遮挡人脸识别算法,该算法改进了传统的基于三元组关系的人脸特征向量学习损失函数,提出了基于多实例关系的损失函数,充分挖掘戴口罩人脸和完整人脸多个正负样本之间的同模态内和跨模态间的关联关系,学习人脸中具有高区分度的能力的特征,同时结合人脸眉眼等局部特征和轮廓等全局特征,学习口罩遮挡人脸的有效特征向量表示。在真实的戴口罩人脸数据集和生成的戴口罩人脸数据上与基准算法进行了比较,实验结果表明所提算法相比传统的基于三元组损失函数和特征融合算法具有更高的识别准确率。

基于对比学习的视觉增强多模态命名实体识别

多模态命名实体识别(MNER)的目的是在给定的图像-文本对中检测实体范围并将其分类为相应的实体类型。尽管现存的MNER方法取得了成功,但它们都集中在使用图像编码器提取视觉特征后,不做增强或过滤处理,直接送入跨模态交互机制。此外,由于文本和图像的表示来自不同的编码器,很难弥合两种模态之间的语义鸿沟,因此,提出了一个基于对比学习的视觉增强多模态命名实体识别模型(MCLAug)。首先,使用ResNet收集图像特征,在此基础上提出金字塔双向融合策略,将低层次高分辨率和高层次强语义的图像信息结合起来,以增强视觉特征。其次,利用CLIP模型中的多模态对比学习思想,计算并最小化对比损失,使两种模态的表示更加一致。最后,利用跨模态注意力机制和门控融合机制获得融合后的图像和文本表示,并通过CRF解码器来执行MNER任务。在两个公开数据集上进行了对比实验并进行消融研究和案例研究,结果证明了所提模型的有效性。

高帧率超声造影定性特征联合定量参数鉴别前列腺良恶性疾病的应用价值

目的:探讨高帧率超声造影(high frame rate contrast-enhanced ultrasound,H-CEUS)定性特征联合定量参数对前列腺良恶性疾病的鉴别诊断价值。方法:选取2022年02月至2023年01月在我院就诊疑似前列腺癌(prostate cancer,PCa)并进行前列腺穿刺活检的患者60例(共67个病灶),根据病理结果分为良性组和恶性组,穿刺前行经直肠常规超声及H-CEUS,记录前列腺基本情况、造影定性特征并绘制时间强度曲线获得定量分析参数,比较两组间差异;以病理结果为“金标准”绘制受试者工作特征(receiver operating characteristic,ROC)曲线,应用Z检验比较H-CEUS定性特征、定量参数单独及联合应用对于前列腺病变良恶性的诊断效能。结果:与良性组相比,恶性组H-CEUS定性特征为供血动脉形态不规则(1/33 vs 11/34)及走形异常(3/33 vs 20/34)、快进(9/33 vs 29/34)、高增强(4/33 vs 25/34)、造影剂分布不均匀(9/33 vs 13/34)的比例较大,差异具有统计学意义(χ2=30.41、18.37、22.96、25.72、8.06,P<0.001、<0.001、<0.001、<0.001、=0.005);定量参数PCa较良性组造影到达时间早[(16.93±3.69)s vs(21.54±3.86)s],峰值强度[(48.8±5.58)dB vs(45.77±4.42)dB]、强度差[4.87(0.87,8.03)vs-0.44(-2.22,2.35)]及强度比[(1.15±0.24)vs(1.01±0.97)]的值较良性大,差异具有统计学意义(t/U=4.24、-2.324、151、-2.535,P<0.001、=0.025、=0.004、=0.015)。ROC曲线示H-CEUS定性及定量联合应用的AUC=0.938,截断值为0.44时诊断效能最佳,约登指数、敏感度、特异度、准确度、阳性预测值及阴性预测值为0.750、89.29%、85.71%、87.75%、89.3%、85.7%。根据净重新分类指数NRI值,联合应用对定性特征及定量参数均为正改善(P<0.05)。结论:H-CEUS应用于前列腺有助于观察造影灌注细节、分析成像特征,对于前列腺良恶性疾病具有较好的鉴别诊断能力,将造影灌注定性特征与定量参数结合的诊断效能优于单独应用。

对比特征增强的高架库小目标检测方法

针对高架库区场景下安全帽检测中目标特征信息少、分类精度低等问题,提出小目标对比特征增强网络。首先提出快速空间金字塔池化跨层融合模块,减少空间维度上的目标信息丢失。然后提出小目标对比特征增强模块,使用双路并行空洞卷积获取不同感受野,利用通道注意力获取特征图在通道维度上更为精准的特征信息,采用浅层特征图减去深层特征图的方法削弱浅层特征图中大目标信息,以增强小目标特征信息表达。加入高效通道注意力解耦检测头,通过将检测头解耦为分类和回归分支,分别学习目标的语义信息和位置信息。实验结果表明,在TT100K数据集上,所提方法的mAP@0.5比基准网络YOLOv5提高了6.4个百分点,比YOLOv7提高了1.9个百分点。在自建高架库数据集上,所提方法的mAP@0.5相比基准网络提高了4.9个百分点,其中安全帽的mAP@0.5相比基准网络提高了6.9个百分点。