基于Vision Transformer的虹膜——人脸多特征融合识别研究

为了提高生物特征识别系统的准确性和鲁棒性,本文研究基于计算机视觉的虹膜—人脸多特征融合识别方法。本文对面部图像中虹膜区域进行提取以及预处理,采用对比度增强和归一化操作,加强了特征提取的一致性,提升了图像质量。为了获取丰富的深度特征,本文使用Vision Transformer模型对预处理后的虹膜和面部图像进行特征提取。利用多头注意力机制将虹膜和面部的多模态特征信息进行融合,再利用全连接层进行分类识别。试验结果表明,该方法识别性能优秀,识别准确性显著提升。

Dual-Path Vision Transformer用于急性缺血性脑卒中辅助诊断

急性缺血性脑卒中是由于脑组织血液供应障碍导致的脑功能障碍,数字减影脑血管造影(DSA)是诊断脑血管疾病的金标准。基于患者的正面和侧面DSA图像,对急性缺血性脑卒中的治疗效果进行分级评估,构建基于Vision Transformer的双路径图像分类智能模型DPVF。为了提高辅助诊断速度,基于EdgeViT的轻量化设计思想进行了模型的构建;为了使模型保持轻量化的同时具有较高的精度,提出空间-通道自注意力模块,促进Transformer模型捕获更全面的特征信息,提高模型的表达能力;此外,对于DPVF的两分支的特征融合,构建交叉注意力模块对两分支输出进行交叉融合,促使模型提取更丰富的特征,从而提高模型表现。实验结果显示DPVF在测试集上的准确率达98.5%,满足实际需求。

基于Vision Transformer的小麦病害图像识别算法

小麦白粉病、赤霉病和锈病是危害小麦产量的三大病害。为提高小麦病害图像的识别准确率,构建一种基于Vision Transformer的小麦病害图像识别算法。首先,通过田间拍摄的方式收集包含小麦白粉病、赤霉病和锈病3种病害在内的小麦病害图像,并对原始图像进行预处理,建立小麦病害图像识别数据集;然后,基于改进的Vision Transformer构建小麦病害图像识别算法,分析不同迁移学习方式和数据增强对模型识别效果的影响。试验可知,全参数迁移学习和数据增强能明显提高Vision Transformer模型的收敛速度和识别精度。最后,在相同时间条件下,对比Vision Transformer、AlexNet和VGG16算法在相同数据集上的表现。试验结果表明,Vision Transformer模型对3种小麦病害图像的平均识别准确率为96.81%,相较于AlexNet和VGG16模型识别准确率分别提高6.68%和4.94%。

细粒度图像分类上Vision Transformer的发展综述

细粒度图像分类(fine-grained image classification,FGIC)一直是计算机视觉领域中的重要问题。与传统图像分类任务相比,FGIC的挑战在于类间对象极其相似,使任务难度进一步增加。随着深度学习的发展,Vision Transformer(ViT)模型在视觉领域掀起热潮,并被引入到FGIC任务中。介绍了FGIC任务所面临的挑战,分析了ViT模型及其特性。主要根据模型结构全面综述了基于ViT的FGIC算法,包括特征提取、特征关系构建、特征注意和特征增强四方面内容,对每种算法进行了总结,并分析了它们的优缺点。通过对不同ViT模型在相同公用数据集上进行模型性能比较,以验证它们在FGIC任务上的有效性。最后指出了目前研究的不足,并提出未来研究方向,以进一步探索ViT在FGIC中的潜力。

基于Vision Transformer的阿尔茨海默病分类研究

为了有效地提升对阿尔茨海默病(AD)的磁共振成像(MRI)图像分类准确率,提出一种LC(Layer-Cut)-ViT方法。该方法通过引入Vision Transformer(ViT)的自注意力机制对MRI图像进行层切分,使模型能更好地理解图像的全局信息,同时突出切片间的特征关系。此外,通过配准、颅骨分离算法提取MRI图像的脑部组织部分,进一步提升模型的性能。实验结果显示,所提方法对阿尔茨海默病的MRI图像具有较好的分类能力。

基于Vision Transformer和迁移学习的家庭领域哭声识别

针对SVM等传统机器学习算法准确率低和当前使用CNN处理家庭领域哭声识别在不同婴儿间出现泛化能力差的问题,提出了一种基于Vision Transformer和迁移学习的婴儿哭声音频分类算法。首先,为实现数据集样本的扩增,采用了包括梅尔频谱转换和数据增强的数据预处理技术,进而达到了增强模型鲁棒性的目的。而后,在微调后的Vision Transformer模型上进行迁移学习训练,同时,训练过程中利用了LookAhead优化器来不断调整模型参数以避免过拟合,最终实验实现了对婴儿哭声音频的自动分类。实验结果表明,本实验模型相比其他深度学习模型具有更高的精确率和更快的收敛速度,同时还能有效地学习到婴儿哭声中更具区分性的特征。可以在新生儿监护、听力筛查和异常检测等领域中发挥重要作用。

Artificial hawk-eye camera for foveated, tetrachromatic, and dynamic vision

With the rapid development of drones and autonomous vehicles, miniaturized and lightweight vision sensors that can track targets are of great interests. Limited by the flat structure, conventional image sensors apply a large number of lenses to achieve corresponding functions, increasing the overall volume and weight of the system.

基于Vision Transformer和卷积注入的车辆重识别

针对车辆重识别中提取特征鲁棒性不高的问题,本文提出基于Vision Transformer的车辆重识别方法。首先,利用注意力机制提出目标导向映射模块,并结合辅助信息嵌入模块,抑制由不同视角、相机拍摄及无效背景引入的噪声。其次,以Vision Transformer远距离建模能力为基础提出通道感知模块,通过并行设计模型能够同时获取图像块之间和图像通道之间的特征,在关注图像块之间关联的基础上,进一步构建通道之间的关联。最后,利用卷积神经网络的局部归纳偏置,将全局特征向量输入到卷积注入模块中进行细化,并与全局特征联合优化,以构建鲁棒性的车辆特征。为了验证提出方法的有效性,在Ve⁃Ri776、VehicleID和VeRi-Wild数据集上分别进行了实验验证。实验结果证明,本文的方法取得了良好的效果。

FPGA and computer-vision-based atom tracking technology for scanning probe microscopy

Atom tracking technology enhanced with innovative algorithms has been implemented in this study,utilizing a comprehensive suite of controllers and software independently developed domestically.Leveraging an on-board field-programmable gate array(FPGA)with a core frequency of 100 MHz,our system facilitates reading and writing operations across 16 channels,performing discrete incremental proportional-integral-derivative(PID)calculations within 3.4 microseconds.Building upon this foundation,gradient and extremum algorithms are further integrated,incorporating circular and spiral scanning modes with a horizontal movement accuracy of 0.38 pm.This integration enhances the real-time performance and significantly increases the accuracy of atom tracking.Atom tracking achieves an equivalent precision of at least 142 pm on a highly oriented pyrolytic graphite(HOPG)surface under room temperature atmospheric conditions.Through applying computer vision and image processing algorithms,atom tracking can be used when scanning a large area.The techniques primarily consist of two algorithms:the region of interest(ROI)-based feature matching algorithm,which achieves 97.92%accuracy,and the feature description-based matching algorithm,with an impressive 99.99%accuracy.Both implementation approaches have been tested for scanner drift measurements,and these technologies are scalable and applicable in various domains of scanning probe microscopy with broad application prospects in the field of nanoengineering.

基于改进Vision Transformer的蝴蝶品种分类

蝴蝶作为一种品类繁多且相似度极高的生物,具有重要的生态环境感知功能。不同品类蝴蝶对环境变化的敏感程度各不相同,因此在农学与生物学研究方向上对蝴蝶的研究具有十分重要的意义。近年来,计算机视觉技术的飞速发展为快速识别蝴蝶品类提供了强有力的技术支持。然而,传统的Vision Transformer模型存在着一些问题,例如缺乏卷积所具有的归纳偏置、局部信息提取能力不足、容易过拟合以及在小数据集上训练缓慢等。针对这些问题,提出了一种基于Vision Transformer改进的蝴蝶分类算法。引入VanillaNet卷积结构,并通过全局注意力机制改进了Class token的更新方式。实验结果显示,在100类蝴蝶数据集上,改进后的Vision Transformer模型的Top-1准确率达到了94.87%,比改进前提升了28.9%。在使用改进的Class token后,算法的Top-1准确率进一步提升至96.64%,相比改进前提升了30.44%。与原网络模型相比,改进后的模型更适用于蝴蝶品种分类任务。

Development and validation of a novel questionnaire regarding vision screening among preschool teachers in Malaysia

AIM:To develop and evaluate the validity and reliability of a knowledge,attitude,and practice questionnaire related to vision screening(KAP-VST)among preschool teachers in Malaysia.METHODS:The questionnaire was developed through a literature review and discussions with experts.Content and face validation were conducted by a panel of experts(n=10)and preschool teachers(n=10),respectively.A pilot study was conducted for construct validation(n=161)and test-retest reliability(n=60)of the newly developed questionnaire.RESULTS:Based on the content and face validation,71 items were generated,and 68 items were selected after exploratory factor analysis.The content validity index for items(I-CVI)score ranged from 0.8-1.0,and the content validity index for scale(S-CVI)/Ave was 0.99.Internal consistency was KR^(2)0=0.93 for knowledge,Cronbach’s alpha=0.758 for attitude,and Cronbach’s alpha=0.856 for practice.CONCLUSION:The KAP-VST is a valid and reliable instrument for assessing knowledge,attitude,and practice in relation to vision screening among preschool teachers in Malaysia.

Highly Efficient Back‑End‑of‑Line Compatible Flexible Si‑Based Optical Memristive Crossbar Array for Edge Neuromorphic Physiological Signal Processing and Bionic Machine Vision

The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices,opening numerous opportunities across countless domains,including personalized healthcare and advanced robotics.Leveraging 3D integration,edge devices can achieve unprecedented miniaturization while simultaneously boosting processing power and minimizing energy consumption.Here,we demonstrate a back-end-of-line compatible optoelectronic synapse with a transfer learning method on health care applications,including electroencephalogram(EEG)-based seizure prediction,electromyography(EMG)-based gesture recognition,and electrocardiogram(ECG)-based arrhythmia detection.With experiments on three biomedical datasets,we observe the classification accuracy improvement for the pretrained model with 2.93%on EEG,4.90%on ECG,and 7.92%on EMG,respectively.The optical programming property of the device enables an ultralow power(2.8×10^(-13) J)fine-tuning process and offers solutions for patient-specific issues in edge computing scenarios.Moreover,the device exhibits impressive light-sensitive characteristics that enable a range of light-triggered synaptic functions,making it promising for neuromorphic vision application.To display the benefits of these intricate synaptic properties,a 5×5 optoelectronic synapse array is developed,effectively simulating human visual perception and memory functions.The proposed flexible optoelectronic synapse holds immense potential for advancing the fields of neuromorphic physiological signal processing and artificial visual systems in wearable applications.

Collaborative positioning for swarms:A brief survey of vision,LiDAR and wireless sensors based methods

As positioning sensors,edge computation power,and communication technologies continue to develop,a moving agent can now sense its surroundings and communicate with other agents.By receiving spatial information from both its environment and other agents,an agent can use various methods and sensor types to localize itself.With its high flexibility and robustness,collaborative positioning has become a widely used method in both military and civilian applications.This paper introduces the basic fundamental concepts and applications of collaborative positioning,and reviews recent progress in the field based on camera,LiDAR(Light Detection and Ranging),wireless sensor,and their integration.The paper compares the current methods with respect to their sensor type,summarizes their main paradigms,and analyzes their evaluation experiments.Finally,the paper discusses the main challenges and open issues that require further research.

Exploring Deep Learning Methods for Computer Vision Applications across Multiple Sectors:Challenges and Future Trends

Computer vision(CV)was developed for computers and other systems to act or make recommendations based on visual inputs,such as digital photos,movies,and other media.Deep learning(DL)methods are more successful than other traditional machine learning(ML)methods inCV.DL techniques can produce state-of-the-art results for difficult CV problems like picture categorization,object detection,and face recognition.In this review,a structured discussion on the history,methods,and applications of DL methods to CV problems is presented.The sector-wise presentation of applications in this papermay be particularly useful for researchers in niche fields who have limited or introductory knowledge of DL methods and CV.This review will provide readers with context and examples of how these techniques can be applied to specific areas.A curated list of popular datasets and a brief description of them are also included for the benefit of readers.

Vision based intelligent traffic light management system using Faster R‐CNN

Transportation systems primarily depend on vehicular flow on roads. Developed coun-tries have shifted towards automated signal control, which manages and updates signal synchronisation automatically. In contrast, traffic in underdeveloped countries is mainly governed by manual traffic light systems. These existing manual systems lead to numerous issues, wasting substantial resources such as time, energy, and fuel, as they cannot make real‐time decisions. In this work, we propose an algorithm to determine traffic signal durations based on real‐time vehicle density, obtained from live closed circuit television camera feeds adjacent to traffic signals. The algorithm automates the traffic light system, making decisions based on vehicle density and employing Faster R‐CNN for vehicle detection. Additionally, we have created a local dataset from live streams of Punjab Safe City cameras in collaboration with the local police authority. The proposed algorithm achieves a class accuracy of 96.6% and a vehicle detection accuracy of 95.7%. Across both day and night modes, our proposed method maintains an average precision, recall, F1 score, and vehicle detection accuracy of 0.94, 0.98, 0.96 and 0.95, respectively. Our proposed work surpasses all evaluation metrics compared to state‐of‐the‐art methodologies.

Dynamic Vision-Based Machinery Fault Diagnosis With Cross-Modality Feature Alignment

Intelligent machinery fault diagnosis methods have been popularly and successfully developed in the past decades,and the vibration acceleration data collected by contact accelerometers have been widely investigated.In many industrial scenarios,contactless sensors are more preferred.The event camera is an emerging bio-inspired technology for vision sensing,which asynchronously records per-pixel brightness change polarity with high temporal resolution and low latency.It offers a promising tool for contactless machine vibration sensing and fault diagnosis.However,the dynamic vision-based methods suffer from variations of practical factors such as camera position,machine operating condition,etc.Furthermore,as a new sensing technology,the labeled dynamic vision data are limited,which generally cannot cover a wide range of machine fault modes.Aiming at these challenges,a novel dynamic vision-based machinery fault diagnosis method is proposed in this paper.It is motivated to explore the abundant vibration acceleration data for enhancing the dynamic vision-based model performance.A crossmodality feature alignment method is thus proposed with deep adversarial neural networks to achieve fault diagnosis knowledge transfer.An event erasing method is further proposed for improving model robustness against variations.The proposed method can effectively identify unseen fault mode with dynamic vision data.Experiments on two rotating machine monitoring datasets are carried out for validations,and the results suggest the proposed method is promising for generalized contactless machinery fault diagnosis.

面向Vision Transformer模型的剪枝技术研究

本文针对Vision Transformer(ViT)模型开展剪枝技术研究,探索了多头自注意力机制中的QKV(Query、Key、Value)权重和全连接层(Fully Connected,FC)权重的剪枝问题。针对ViT模型本文提出了3组剪枝方案:只对QKV剪枝、只对FC剪枝以及对QKV和FC同时进行剪枝,以探究不同剪枝策略对ViT模型准确率和模型参数压缩率的影响。本文开展的研究工作为深度学习模型的压缩和优化提供了重要参考,对于实际应用中的模型精简和性能优化具有指导意义。

Clinical usefulness of the baby vision test in young children and its correlation with the Snellen chart

AIM:To investigate the efficacy of a new visual acuity(VA)screening method,the baby vision test for young children.METHODS:A total 105 eyes of 65 children aged 2-8y were included in the study.Acuity testing was conducted using a standardized recognition acuity chart(Snellen visual chart:at 3 m)and the baby vision model assessment.The baby vision device includes a screen,a near infrared camera and a computer.Children were seated at a measured distance of 33-40 cm from a display for testing.VA was estimated according to the highest resolution the children could follow.Decimal VA data were converted to logarithm of the minimum angle of resolution(logMAR)for statistical analysis.The VA results for each child were recorded and analyzed for consistency.RESULTS:The mean VA measured using the Snellen visual chart was 0.62±0.32,and that assessed using the baby vision test was 0.66±0.27.The 95%limit of agreement was-0.609 to 0.695,with 95.2%(100/105)plots within the 95%limits of agreement.VA values of the baby vision test were significantly correlated with those of the Snellen chart(R=0.274,P=0.005).CONCLUSION:The baby vision test can be used as a relatively reliable method for estimating VA in young children.This new acuity assessment might be a valid predictor of optotype-measured acuity later in preverbal children.

Frequency and associated factors of accommodation and non-strabismic binocular vision dysfunction among medical university students

AIM:To investigate the frequency and associated factors of accommodation and non-strabismic binocular vision dysfunction among medical university students.METHODS:Totally 158 student volunteers underwent routine vision examination in the optometry clinic of Guangxi Medical University.Their data were used to identify the different types of accommodation and nonstrabismic binocular vision dysfunction and to determine their frequency.Correlation analysis and logistic regression were used to examine the factors associated with these abnormalities.RESULTS:The results showed that 36.71%of the subjects had accommodation and non-strabismic binocular vision issues,with 8.86%being attributed to accommodation dysfunction and 27.85%to binocular abnormalities.Convergence insufficiency(CI)was the most common abnormality,accounting for 13.29%.Those with these abnormalities experienced higher levels of eyestrain(χ2=69.518,P<0.001).The linear correlations were observed between the difference of binocular spherical equivalent(SE)and the index of horizontal esotropia at a distance(r=0.231,P=0.004)and the asthenopia survey scale(ASS)score(r=0.346,P<0.001).Furthermore,the right eye's SE was inversely correlated with the convergence of positive and negative fusion images at close range(r=-0.321,P<0.001),the convergence of negative fusion images at close range(r=-0.294,P<0.001),the vergence facility(VF;r=-0.234,P=0.003),and the set of negative fusion images at far range(r=-0.237,P=0.003).Logistic regression analysis indicated that gender,age,and the difference in right and binocular SE did not influence the emergence of these abnormalities.CONCLUSION:Binocular vision abnormalities are more prevalent than accommodation dysfunction,with CI being the most frequent type.Greater binocular refractive disparity leads to more severe eyestrain symptoms.

Automatic diagnosis of diabetic retinopathy using vision transformer based on wide-field optical coherence tomography angiography

Diabetic retinopathy(DR)is one of the major causes of visual impairment in adults with diabetes.Optical coherence tomography angiography(OCTA)is nowadays widely used as the golden criterion for diagnosing DR.Recently,wide-field OCTA(WF-OCTA)provided more abundant information including that of the peripheral retinal degenerative changes and it can contribute in accurately diagnosing DR.The need for an automatic DR diagnostic system based on WF-OCTA pictures attracts more and more attention due to the large diabetic population and the prevalence of retinopathy cases.In this study,automatic diagnosis of DR using vision transformer was performed using WF-OCTA images(12 mm×12 mm single-scan)centered on the fovea as the dataset.WF-OCTA images were automatically classified into four classes:No DR,mild nonproliferative diabetic retinopathy(NPDR),moderate to severe NPDR,and proliferative diabetic retinopathy(PDR).The proposed method for detecting DR on the test set achieves accuracy of 99.55%,sensitivity of 99.49%,and specificity of 99.57%.The accuracy of the method for DR staging reaches up to 99.20%,which has been proven to be higher than that attained by classical convolutional neural network models.Results show that the automatic diagnosis of DR based on vision transformer and WF-OCTA pictures is more effective for detecting and staging DR.