基于Vision Transformer的虹膜——人脸多特征融合识别研究

为了提高生物特征识别系统的准确性和鲁棒性,本文研究基于计算机视觉的虹膜—人脸多特征融合识别方法。本文对面部图像中虹膜区域进行提取以及预处理,采用对比度增强和归一化操作,加强了特征提取的一致性,提升了图像质量。为了获取丰富的深度特征,本文使用Vision Transformer模型对预处理后的虹膜和面部图像进行特征提取。利用多头注意力机制将虹膜和面部的多模态特征信息进行融合,再利用全连接层进行分类识别。试验结果表明,该方法识别性能优秀,识别准确性显著提升。

基于Vision Transformer的小麦病害图像识别算法

小麦白粉病、赤霉病和锈病是危害小麦产量的三大病害。为提高小麦病害图像的识别准确率,构建一种基于Vision Transformer的小麦病害图像识别算法。首先,通过田间拍摄的方式收集包含小麦白粉病、赤霉病和锈病3种病害在内的小麦病害图像,并对原始图像进行预处理,建立小麦病害图像识别数据集;然后,基于改进的Vision Transformer构建小麦病害图像识别算法,分析不同迁移学习方式和数据增强对模型识别效果的影响。试验可知,全参数迁移学习和数据增强能明显提高Vision Transformer模型的收敛速度和识别精度。最后,在相同时间条件下,对比Vision Transformer、AlexNet和VGG16算法在相同数据集上的表现。试验结果表明,Vision Transformer模型对3种小麦病害图像的平均识别准确率为96.81%,相较于AlexNet和VGG16模型识别准确率分别提高6.68%和4.94%。

基于Vision Transformer与迁移学习的裤装廓形识别与分类

针对裤装廓形识别与分类模型的分类不准确问题,文章采用带有自注意力机制的Vision Transformer模型实现裤装廓形图像的分类,对于图片背景等无关信息对廓形识别的干扰,添加自注意力机制,增强有用特征通道。为防止因裤型样本数据集较少产生过拟合问题,可通过迁移学习方法对阔腿裤、喇叭裤、紧身裤、哈伦裤4种裤装廓形进行训练和验证,将改进的Vision Transformer模型与传统CNN模型进行对比实验,验证模型效果。实验结果表明:使用Vision Transformer模型在4种裤装廓形分类上的分类准确率达到97.72%,与ResNet-50和MobileNetV2模型相比均有提升,可为服装廓形的图像分类识别提供有力支撑,在实际服装领域中有较高的使用价值。

细粒度图像分类上Vision Transformer的发展综述

细粒度图像分类(fine-grained image classification,FGIC)一直是计算机视觉领域中的重要问题。与传统图像分类任务相比,FGIC的挑战在于类间对象极其相似,使任务难度进一步增加。随着深度学习的发展,Vision Transformer(ViT)模型在视觉领域掀起热潮,并被引入到FGIC任务中。介绍了FGIC任务所面临的挑战,分析了ViT模型及其特性。主要根据模型结构全面综述了基于ViT的FGIC算法,包括特征提取、特征关系构建、特征注意和特征增强四方面内容,对每种算法进行了总结,并分析了它们的优缺点。通过对不同ViT模型在相同公用数据集上进行模型性能比较,以验证它们在FGIC任务上的有效性。最后指出了目前研究的不足,并提出未来研究方向,以进一步探索ViT在FGIC中的潜力。

Collaborative positioning for swarms:A brief survey of vision,LiDAR and wireless sensors based methods

As positioning sensors,edge computation power,and communication technologies continue to develop,a moving agent can now sense its surroundings and communicate with other agents.By receiving spatial information from both its environment and other agents,an agent can use various methods and sensor types to localize itself.With its high flexibility and robustness,collaborative positioning has become a widely used method in both military and civilian applications.This paper introduces the basic fundamental concepts and applications of collaborative positioning,and reviews recent progress in the field based on camera,LiDAR(Light Detection and Ranging),wireless sensor,and their integration.The paper compares the current methods with respect to their sensor type,summarizes their main paradigms,and analyzes their evaluation experiments.Finally,the paper discusses the main challenges and open issues that require further research.

Frequency and associated factors of accommodation and non-strabismic binocular vision dysfunction among medical university students

AIM:To investigate the frequency and associated factors of accommodation and non-strabismic binocular vision dysfunction among medical university students.METHODS:Totally 158 student volunteers underwent routine vision examination in the optometry clinic of Guangxi Medical University.Their data were used to identify the different types of accommodation and nonstrabismic binocular vision dysfunction and to determine their frequency.Correlation analysis and logistic regression were used to examine the factors associated with these abnormalities.RESULTS:The results showed that 36.71%of the subjects had accommodation and non-strabismic binocular vision issues,with 8.86%being attributed to accommodation dysfunction and 27.85%to binocular abnormalities.Convergence insufficiency(CI)was the most common abnormality,accounting for 13.29%.Those with these abnormalities experienced higher levels of eyestrain(χ2=69.518,P<0.001).The linear correlations were observed between the difference of binocular spherical equivalent(SE)and the index of horizontal esotropia at a distance(r=0.231,P=0.004)and the asthenopia survey scale(ASS)score(r=0.346,P<0.001).Furthermore,the right eye's SE was inversely correlated with the convergence of positive and negative fusion images at close range(r=-0.321,P<0.001),the convergence of negative fusion images at close range(r=-0.294,P<0.001),the vergence facility(VF;r=-0.234,P=0.003),and the set of negative fusion images at far range(r=-0.237,P=0.003).Logistic regression analysis indicated that gender,age,and the difference in right and binocular SE did not influence the emergence of these abnormalities.CONCLUSION:Binocular vision abnormalities are more prevalent than accommodation dysfunction,with CI being the most frequent type.Greater binocular refractive disparity leads to more severe eyestrain symptoms.

基于Vision Transformer和迁移学习的家庭领域哭声识别

针对SVM等传统机器学习算法准确率低和当前使用CNN处理家庭领域哭声识别在不同婴儿间出现泛化能力差的问题,提出了一种基于Vision Transformer和迁移学习的婴儿哭声音频分类算法。首先,为实现数据集样本的扩增,采用了包括梅尔频谱转换和数据增强的数据预处理技术,进而达到了增强模型鲁棒性的目的。而后,在微调后的Vision Transformer模型上进行迁移学习训练,同时,训练过程中利用了LookAhead优化器来不断调整模型参数以避免过拟合,最终实验实现了对婴儿哭声音频的自动分类。实验结果表明,本实验模型相比其他深度学习模型具有更高的精确率和更快的收敛速度,同时还能有效地学习到婴儿哭声中更具区分性的特征。可以在新生儿监护、听力筛查和异常检测等领域中发挥重要作用。

基于Vision Transformer的阿尔茨海默病分类研究

为了有效地提升对阿尔茨海默病(AD)的磁共振成像(MRI)图像分类准确率,提出一种LC(Layer-Cut)-ViT方法。该方法通过引入Vision Transformer(ViT)的自注意力机制对MRI图像进行层切分,使模型能更好地理解图像的全局信息,同时突出切片间的特征关系。此外,通过配准、颅骨分离算法提取MRI图像的脑部组织部分,进一步提升模型的性能。实验结果显示,所提方法对阿尔茨海默病的MRI图像具有较好的分类能力。

Dual-Path Vision Transformer用于急性缺血性脑卒中辅助诊断

急性缺血性脑卒中是由于脑组织血液供应障碍导致的脑功能障碍,数字减影脑血管造影(DSA)是诊断脑血管疾病的金标准。基于患者的正面和侧面DSA图像,对急性缺血性脑卒中的治疗效果进行分级评估,构建基于Vision Transformer的双路径图像分类智能模型DPVF。为了提高辅助诊断速度,基于EdgeViT的轻量化设计思想进行了模型的构建;为了使模型保持轻量化的同时具有较高的精度,提出空间-通道自注意力模块,促进Transformer模型捕获更全面的特征信息,提高模型的表达能力;此外,对于DPVF的两分支的特征融合,构建交叉注意力模块对两分支输出进行交叉融合,促使模型提取更丰富的特征,从而提高模型表现。实验结果显示DPVF在测试集上的准确率达98.5%,满足实际需求。

基于Vision Transformer和迁移学习的垃圾图像分类研究

为解决垃圾图像分类中分类准确率低及小样本类别性能差的问题,以生活垃圾图像为研究对象,以正确识别生活垃圾类别为研究目标,利用Vision Transformer模型为分类网络架构,使用迁移学习机制实现该模型在华为云垃圾分类数据集上的训练及分类推理。实验结果表明,基于注意力机制的分类模型相较于基于卷积结构的ResNet、DenseNet分类模型具有更高的分类准确率,可达96%,同时测试集的混淆矩阵表明Vision Transformer分类模型在样本不均衡数据集中对于小样本类别也具有较高的准确率,具有实际部署、推理的应用价值。

基于改进Vision Transformer的蝴蝶品种分类

蝴蝶作为一种品类繁多且相似度极高的生物,具有重要的生态环境感知功能。不同品类蝴蝶对环境变化的敏感程度各不相同,因此在农学与生物学研究方向上对蝴蝶的研究具有十分重要的意义。近年来,计算机视觉技术的飞速发展为快速识别蝴蝶品类提供了强有力的技术支持。然而,传统的Vision Transformer模型存在着一些问题,例如缺乏卷积所具有的归纳偏置、局部信息提取能力不足、容易过拟合以及在小数据集上训练缓慢等。针对这些问题,提出了一种基于Vision Transformer改进的蝴蝶分类算法。引入VanillaNet卷积结构,并通过全局注意力机制改进了Class token的更新方式。实验结果显示,在100类蝴蝶数据集上,改进后的Vision Transformer模型的Top-1准确率达到了94.87%,比改进前提升了28.9%。在使用改进的Class token后,算法的Top-1准确率进一步提升至96.64%,相比改进前提升了30.44%。与原网络模型相比,改进后的模型更适用于蝴蝶品种分类任务。

基于改进Vision Transformer的道岔故障智能诊断

道岔故障种类繁多,特征复杂,存在检测难、分类难等问题,导致故障排查效率低下,对铁路运输安全构成威胁。Vision Transformer模型在图像分类方面具有较高准确度,但是其处理的是图像块,而不是传统的像素级特征,在某些情况下可能会影响曲线局部信息的获取。针对上述情况,提出一种基于改进Vision Transformer模型的故障曲线分类算法。首先,对典型道岔故障及原因进行梳理分类,指出几种典型的道岔故障;其次,对使用道岔动作电流数据生成的图像尺寸进行调整并根据故障图像特点进行数据增强,使用ResNet网络取代原Vision Transformer模型中的故障图像分块机制进行特征提取,同时采用相对位置编码增强模型的适应性和泛化能力;最后,利用模型的多头自注意力机制,综合全局与局部信息进行分类,并得到分类权重。经过实验验证,本文道岔故障分类识别总体准确率达99.77%,各分类识别的平均精确率达99.78%,与原模型相比,在训练集和验证集上的识别精度分别提升了5.4%和2.4%。为了更好地理解模型的性能,采用Grad-CAM方法将迭代过程可视化,剖析了模型关注区域的变化过程,并在测试集上与VGG-16、DenseNet121等经典分类模型进行性能对比;通过ROC曲线评估分类效果,显示改进的模型取得更优结果。研究结果为道岔故障识别分类提供了新的理论支持,并为未来的研究提供了新的思路和方法。

基于Vision Transformer-LSTM(ViTL)的多时序遥感影像农作物分类方法

针对当前遥感农作物分类研究中深度学习模型对光谱时间和空间信息特征采样不足,农作物提取仍然存在边界模糊、漏提、误提的问题,提出了一种名为视觉Transformer-长短期记忆递归神经网络(Vision Transformer-long short term memory,ViTL)的深度学习模型,ViTL模型集成了双路Vision-Transformer特征提取、时空特征融合和长短期记忆递归神经网络(LSTM)时序分类等3个关键模块,双路Vision-Transformer特征提取模块用于捕获图像的时空特征相关性,一路提取空间分类特征,一路提取时间变化特征;时空特征融合模块用于将多时特征信息进行交叉融合;LSTM时序分类模块捕捉多时序的依赖关系并进行输出分类。综合利用基于多时序卫星影像的遥感技术理论和方法,对黑龙江省齐齐哈尔市讷河市作物信息进行提取,研究结果表明,ViTL模型表现出色,其总体准确率(Overall Accuracy,OA)、平均交并比(Mean Intersection over Union,MIoU)和F1分数分别达到0.8676、0.6987和0.8175,与其他广泛使用的深度学习方法相比,包括三维卷积神经网络(3-D CNN)、二维卷积神经网络(2-D CNN)和长短期记忆递归神经网络(LSTM),ViTL模型的F1分数提高了9%~12%,显示出显著的优越性。ViTL模型克服了面对多时序遥感影像的农作物分类任务中的时间和空间信息特征采样不足问题,为准确、高效地农作物分类提供了新思路。

Highly Efficient Back‑End‑of‑Line Compatible Flexible Si‑Based Optical Memristive Crossbar Array for Edge Neuromorphic Physiological Signal Processing and Bionic Machine Vision

The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices,opening numerous opportunities across countless domains,including personalized healthcare and advanced robotics.Leveraging 3D integration,edge devices can achieve unprecedented miniaturization while simultaneously boosting processing power and minimizing energy consumption.Here,we demonstrate a back-end-of-line compatible optoelectronic synapse with a transfer learning method on health care applications,including electroencephalogram(EEG)-based seizure prediction,electromyography(EMG)-based gesture recognition,and electrocardiogram(ECG)-based arrhythmia detection.With experiments on three biomedical datasets,we observe the classification accuracy improvement for the pretrained model with 2.93%on EEG,4.90%on ECG,and 7.92%on EMG,respectively.The optical programming property of the device enables an ultralow power(2.8×10^(-13) J)fine-tuning process and offers solutions for patient-specific issues in edge computing scenarios.Moreover,the device exhibits impressive light-sensitive characteristics that enable a range of light-triggered synaptic functions,making it promising for neuromorphic vision application.To display the benefits of these intricate synaptic properties,a 5×5 optoelectronic synapse array is developed,effectively simulating human visual perception and memory functions.The proposed flexible optoelectronic synapse holds immense potential for advancing the fields of neuromorphic physiological signal processing and artificial visual systems in wearable applications.

FPGA and computer-vision-based atom tracking technology for scanning probe microscopy

Atom tracking technology enhanced with innovative algorithms has been implemented in this study,utilizing a comprehensive suite of controllers and software independently developed domestically.Leveraging an on-board field-programmable gate array(FPGA)with a core frequency of 100 MHz,our system facilitates reading and writing operations across 16 channels,performing discrete incremental proportional-integral-derivative(PID)calculations within 3.4 microseconds.Building upon this foundation,gradient and extremum algorithms are further integrated,incorporating circular and spiral scanning modes with a horizontal movement accuracy of 0.38 pm.This integration enhances the real-time performance and significantly increases the accuracy of atom tracking.Atom tracking achieves an equivalent precision of at least 142 pm on a highly oriented pyrolytic graphite(HOPG)surface under room temperature atmospheric conditions.Through applying computer vision and image processing algorithms,atom tracking can be used when scanning a large area.The techniques primarily consist of two algorithms:the region of interest(ROI)-based feature matching algorithm,which achieves 97.92%accuracy,and the feature description-based matching algorithm,with an impressive 99.99%accuracy.Both implementation approaches have been tested for scanner drift measurements,and these technologies are scalable and applicable in various domains of scanning probe microscopy with broad application prospects in the field of nanoengineering.

Artificial hawk-eye camera for foveated, tetrachromatic, and dynamic vision

With the rapid development of drones and autonomous vehicles, miniaturized and lightweight vision sensors that can track targets are of great interests. Limited by the flat structure, conventional image sensors apply a large number of lenses to achieve corresponding functions, increasing the overall volume and weight of the system.

Dynamic Vision-Based Machinery Fault Diagnosis With Cross-Modality Feature Alignment

Intelligent machinery fault diagnosis methods have been popularly and successfully developed in the past decades,and the vibration acceleration data collected by contact accelerometers have been widely investigated.In many industrial scenarios,contactless sensors are more preferred.The event camera is an emerging bio-inspired technology for vision sensing,which asynchronously records per-pixel brightness change polarity with high temporal resolution and low latency.It offers a promising tool for contactless machine vibration sensing and fault diagnosis.However,the dynamic vision-based methods suffer from variations of practical factors such as camera position,machine operating condition,etc.Furthermore,as a new sensing technology,the labeled dynamic vision data are limited,which generally cannot cover a wide range of machine fault modes.Aiming at these challenges,a novel dynamic vision-based machinery fault diagnosis method is proposed in this paper.It is motivated to explore the abundant vibration acceleration data for enhancing the dynamic vision-based model performance.A crossmodality feature alignment method is thus proposed with deep adversarial neural networks to achieve fault diagnosis knowledge transfer.An event erasing method is further proposed for improving model robustness against variations.The proposed method can effectively identify unseen fault mode with dynamic vision data.Experiments on two rotating machine monitoring datasets are carried out for validations,and the results suggest the proposed method is promising for generalized contactless machinery fault diagnosis.

Development and validation of a novel questionnaire regarding vision screening among preschool teachers in Malaysia

AIM:To develop and evaluate the validity and reliability of a knowledge,attitude,and practice questionnaire related to vision screening(KAP-VST)among preschool teachers in Malaysia.METHODS:The questionnaire was developed through a literature review and discussions with experts.Content and face validation were conducted by a panel of experts(n=10)and preschool teachers(n=10),respectively.A pilot study was conducted for construct validation(n=161)and test-retest reliability(n=60)of the newly developed questionnaire.RESULTS:Based on the content and face validation,71 items were generated,and 68 items were selected after exploratory factor analysis.The content validity index for items(I-CVI)score ranged from 0.8-1.0,and the content validity index for scale(S-CVI)/Ave was 0.99.Internal consistency was KR^(2)0=0.93 for knowledge,Cronbach’s alpha=0.758 for attitude,and Cronbach’s alpha=0.856 for practice.CONCLUSION:The KAP-VST is a valid and reliable instrument for assessing knowledge,attitude,and practice in relation to vision screening among preschool teachers in Malaysia.

Exploring Deep Learning Methods for Computer Vision Applications across Multiple Sectors:Challenges and Future Trends

Computer vision(CV)was developed for computers and other systems to act or make recommendations based on visual inputs,such as digital photos,movies,and other media.Deep learning(DL)methods are more successful than other traditional machine learning(ML)methods inCV.DL techniques can produce state-of-the-art results for difficult CV problems like picture categorization,object detection,and face recognition.In this review,a structured discussion on the history,methods,and applications of DL methods to CV problems is presented.The sector-wise presentation of applications in this papermay be particularly useful for researchers in niche fields who have limited or introductory knowledge of DL methods and CV.This review will provide readers with context and examples of how these techniques can be applied to specific areas.A curated list of popular datasets and a brief description of them are also included for the benefit of readers.

基于Vision Transformer和卷积注入的车辆重识别

针对车辆重识别中提取特征鲁棒性不高的问题,本文提出基于Vision Transformer的车辆重识别方法。首先,利用注意力机制提出目标导向映射模块,并结合辅助信息嵌入模块,抑制由不同视角、相机拍摄及无效背景引入的噪声。其次,以Vision Transformer远距离建模能力为基础提出通道感知模块,通过并行设计模型能够同时获取图像块之间和图像通道之间的特征,在关注图像块之间关联的基础上,进一步构建通道之间的关联。最后,利用卷积神经网络的局部归纳偏置,将全局特征向量输入到卷积注入模块中进行细化,并与全局特征联合优化,以构建鲁棒性的车辆特征。为了验证提出方法的有效性,在Ve⁃Ri776、VehicleID和VeRi-Wild数据集上分别进行了实验验证。实验结果证明,本文的方法取得了良好的效果。