Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input t...Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input to augment the RGB images.Depth-based methods attempt to convert estimated depth maps to pseudo-LiDAR and then use LiDAR-based object detectors or focus on the perspective of image and depth fusion learning.However,they demonstrate limited performance and efficiency as a result of depth inaccuracy and complex fusion mode with convolutions.Different from these approaches,our proposed depth-guided vision transformer with a normalizing flows(NF-DVT)network uses normalizing flows to build priors in depth maps to achieve more accurate depth information.Then we develop a novel Swin-Transformer-based backbone with a fusion module to process RGB image patches and depth map patches with two separate branches and fuse them using cross-attention to exchange information with each other.Furthermore,with the help of pixel-wise relative depth values in depth maps,we develop new relative position embeddings in the cross-attention mechanism to capture more accurate sequence ordering of input tokens.Our method is the first Swin-Transformer-based backbone architecture for monocular 3D object detection.The experimental results on the KITTI and the challenging Waymo Open datasets show the effectiveness of our proposed method and superior performance over previous counterparts.展开更多
Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by ut...Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.展开更多
Cloud detection from satellite and drone imagery is crucial for applications such as weather forecasting and environmentalmonitoring.Addressing the limitations of conventional convolutional neural networks,we propose ...Cloud detection from satellite and drone imagery is crucial for applications such as weather forecasting and environmentalmonitoring.Addressing the limitations of conventional convolutional neural networks,we propose an innovative transformer-based method.This method leverages transformers,which are adept at processing data sequences,to enhance cloud detection accuracy.Additionally,we introduce a Cyclic Refinement Architecture that improves the resolution and quality of feature extraction,thereby aiding in the retention of critical details often lost during cloud detection.Our extensive experimental validation shows that our approach significantly outperforms established models,excelling in high-resolution feature extraction and precise cloud segmentation.By integrating Positional Visual Transformers(PVT)with this architecture,our method advances high-resolution feature delineation and segmentation accuracy.Ultimately,our research offers a novel perspective for surmounting traditional challenges in cloud detection and contributes to the advancement of precise and dependable image analysis across various domains.展开更多
目的:为提高小肠病变分类识别的准确性,提出一种基于Swin Transformer网络与Adapt-RandAugment数据增强方法的小肠胶囊内镜图像分类方法。方法:基于RandAugment数据增强子策略和增强小肠胶囊内镜图像时不丢失特征、不失真的原则提出Adap...目的:为提高小肠病变分类识别的准确性,提出一种基于Swin Transformer网络与Adapt-RandAugment数据增强方法的小肠胶囊内镜图像分类方法。方法:基于RandAugment数据增强子策略和增强小肠胶囊内镜图像时不丢失特征、不失真的原则提出Adapt-RandAugment数据增强方法。在公开的小肠胶囊内镜图像Kvasir-Capsule数据集中,基于Swin Transformer网络,采用Adapt-RandAugment数据增强方法进行训练,以卷积神经网络ResNet152、DenseNet161为基准,验证Swin Transformer网络和Adapt-RandAugment数据增强方法组合对小肠胶囊内镜图像分类识别的性能。结果:提出的方法宏平均精度(macro average precision,MAC-PRE)、宏平均召回率(macro average recall,MAC-REC)、宏F1分数(macro average F1 score,MAC-F1-S)分别为0.3832、0.3148、0.2905,微平均精度(micro average precision,MIC-PRE)、微平均召回率(micro average recall,MIC-REC)、微平均F1分数(micro average F1 score,MIC-F1-S)均为0.7553,马修斯相关系数(Matthews correlation coefficient,MCC)为0.4523,均优于ResNet152和DenseNet161网络。结论:基于Swin Transformer网络与Adapt-RandAugment数据增强方法的小肠胶囊内镜图像分类方法具有较好的小肠胶囊内镜图像分类识别效果和较高的识别准确率。展开更多
The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the...The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the prices of power transformer materials manifest as nonsmooth and nonlinear sequences.Hence,estimating the acquisition costs of power grid projects is difficult,hindering the normal operation of power engineering construction.To more accurately predict the price of power transformer materials,this study proposes a method based on complementary ensemble empirical mode decomposition(CEEMD)and gated recurrent unit(GRU)network.First,the CEEMD decomposed the price series into multiple intrinsic mode functions(IMFs).Multiple IMFs were clustered to obtain several aggregated sequences based on the sample entropy of each IMF.Then,an empirical wavelet transform(EWT)was applied to the aggregation sequence with a large sample entropy,and the multiple subsequences obtained from the decomposition were predicted by the GRU model.The GRU model was used to directly predict the aggregation sequences with a small sample entropy.In this study,we used authentic historical pricing data for power transformer materials to validate the proposed approach.The empirical findings demonstrated the efficacy of our method across both datasets,with mean absolute percentage errors(MAPEs)of less than 1%and 3%.This approach holds a significant reference value for future research in the field of power transformer material price prediction.展开更多
RFID-based human activity recognition(HAR)attracts attention due to its convenience,noninvasiveness,and privacy protection.Existing RFID-based HAR methods use modeling,CNN,or LSTM to extract features effectively.Still...RFID-based human activity recognition(HAR)attracts attention due to its convenience,noninvasiveness,and privacy protection.Existing RFID-based HAR methods use modeling,CNN,or LSTM to extract features effectively.Still,they have shortcomings:1)requiring complex hand-crafted data cleaning processes and 2)only addressing single-person activity recognition based on specific RF signals.To solve these problems,this paper proposes a novel device-free method based on Time-streaming Multiscale Transformer called TransTM.This model leverages the Transformer's powerful data fitting capabilities to take raw RFID RSSI data as input without pre-processing.Concretely,we propose a multiscale convolutional hybrid Transformer to capture behavioral features that recognizes singlehuman activities and human-to-human interactions.Compared with existing CNN-and LSTM-based methods,the Transformer-based method has more data fitting power,generalization,and scalability.Furthermore,using RF signals,our method achieves an excellent classification effect on human behaviorbased classification tasks.Experimental results on the actual RFID datasets show that this model achieves a high average recognition accuracy(99.1%).The dataset we collected for detecting RFID-based indoor human activities will be published.展开更多
A consumption of 46.9 million tons of processed tomatoes was reported in 2022 which is merely 20%of the total consumption.An increase of 3.3%in consumption is predicted from 2024 to 2032.Tomatoes are also rich in iron...A consumption of 46.9 million tons of processed tomatoes was reported in 2022 which is merely 20%of the total consumption.An increase of 3.3%in consumption is predicted from 2024 to 2032.Tomatoes are also rich in iron,potassium,antioxidant lycopene,vitamins A,C and K which are important for preventing cancer,and maintaining blood pressure and glucose levels.Thus,tomatoes are globally important due to their widespread usage and nutritional value.To face the high demand for tomatoes,it is mandatory to investigate the causes of crop loss and minimize them.Diseases are one of the major causes that adversely affect crop yield and degrade the quality of the tomato fruit.This leads to financial losses and affects the livelihood of farmers.Therefore,automatic disease detection at any stage of the tomato plant is a critical issue.Deep learning models introduced in the literature show promising results,but the models are difficult to implement on handheld devices such as mobile phones due to high computational costs and a large number of parameters.Also,most of the models proposed so far work efficiently for images with plain backgrounds where a clear demarcation exists between the background and leaf region.Moreover,the existing techniques lack in recognizing multiple diseases on the same leaf.To address these concerns,we introduce a customized deep learning-based convolution vision transformer model.Themodel achieves an accuracy of 93.51%for classifying tomato leaf images with plain as well as complex backgrounds into 13 categories.It requires a space storage of merely 5.8 MB which is 98.93%,98.33%,and 92.64%less than stateof-the-art visual geometry group,vision transformers,and convolution vision transformermodels,respectively.Its training time of 44 min is 51.12%,74.12%,and 57.7%lower than the above-mentioned models.Thus,it can be deployed on(Internet of Things)IoT-enabled devices,drones,or mobile devices to assist farmers in the real-time monitoring of tomato crops.The periodicmonitoring promotes timely action to prevent the spread of diseases and reduce crop loss.展开更多
Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have becom...Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.展开更多
This paper addresses the problem of predicting population density leveraging cellular station data.As wireless communication devices are commonly used,cellular station data has become integral for estimating populatio...This paper addresses the problem of predicting population density leveraging cellular station data.As wireless communication devices are commonly used,cellular station data has become integral for estimating population figures and studying their movement,thereby implying significant contributions to urban planning.However,existing research grapples with issues pertinent to preprocessing base station data and the modeling of population prediction.To address this,we propose methodologies for preprocessing cellular station data to eliminate any irregular or redundant data.The preprocessing reveals a distinct cyclical characteristic and high-frequency variation in population shift.Further,we devise a multi-view enhancement model grounded on the Transformer(MVformer),targeting the improvement of the accuracy of extended time-series population predictions.Comparative experiments,conducted on the above-mentioned population dataset using four alternate Transformer-based models,indicate that our proposedMVformer model enhances prediction accuracy by approximately 30%for both univariate and multivariate time-series prediction assignments.The performance of this model in tasks pertaining to population prediction exhibits commendable results.展开更多
基金supported in part by the Major Project for New Generation of AI (2018AAA0100400)the National Natural Science Foundation of China (61836014,U21B2042,62072457,62006231)the InnoHK Program。
文摘Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input to augment the RGB images.Depth-based methods attempt to convert estimated depth maps to pseudo-LiDAR and then use LiDAR-based object detectors or focus on the perspective of image and depth fusion learning.However,they demonstrate limited performance and efficiency as a result of depth inaccuracy and complex fusion mode with convolutions.Different from these approaches,our proposed depth-guided vision transformer with a normalizing flows(NF-DVT)network uses normalizing flows to build priors in depth maps to achieve more accurate depth information.Then we develop a novel Swin-Transformer-based backbone with a fusion module to process RGB image patches and depth map patches with two separate branches and fuse them using cross-attention to exchange information with each other.Furthermore,with the help of pixel-wise relative depth values in depth maps,we develop new relative position embeddings in the cross-attention mechanism to capture more accurate sequence ordering of input tokens.Our method is the first Swin-Transformer-based backbone architecture for monocular 3D object detection.The experimental results on the KITTI and the challenging Waymo Open datasets show the effectiveness of our proposed method and superior performance over previous counterparts.
基金supported in part by the National Natural Science Foundation of China under Grants 61502162,61702175,and 61772184in part by the Fund of the State Key Laboratory of Geo-information Engineering under Grant SKLGIE2016-M-4-2+4 种基金in part by the Hunan Natural Science Foundation of China under Grant 2018JJ2059in part by the Key R&D Project of Hunan Province of China under Grant 2018GK2014in part by the Open Fund of the State Key Laboratory of Integrated Services Networks under Grant ISN17-14Chinese Scholarship Council(CSC)through College of Computer Science and Electronic Engineering,Changsha,410082Hunan University with Grant CSC No.2018GXZ020784.
文摘Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.
基金funded by the Chongqing Normal University Startup Foundation for PhD(22XLB021)supported by the Open Research Project of the State Key Laboratory of Industrial Control Technology,Zhejiang University,China(No.ICT2023B40).
文摘Cloud detection from satellite and drone imagery is crucial for applications such as weather forecasting and environmentalmonitoring.Addressing the limitations of conventional convolutional neural networks,we propose an innovative transformer-based method.This method leverages transformers,which are adept at processing data sequences,to enhance cloud detection accuracy.Additionally,we introduce a Cyclic Refinement Architecture that improves the resolution and quality of feature extraction,thereby aiding in the retention of critical details often lost during cloud detection.Our extensive experimental validation shows that our approach significantly outperforms established models,excelling in high-resolution feature extraction and precise cloud segmentation.By integrating Positional Visual Transformers(PVT)with this architecture,our method advances high-resolution feature delineation and segmentation accuracy.Ultimately,our research offers a novel perspective for surmounting traditional challenges in cloud detection and contributes to the advancement of precise and dependable image analysis across various domains.
文摘目的:为提高小肠病变分类识别的准确性,提出一种基于Swin Transformer网络与Adapt-RandAugment数据增强方法的小肠胶囊内镜图像分类方法。方法:基于RandAugment数据增强子策略和增强小肠胶囊内镜图像时不丢失特征、不失真的原则提出Adapt-RandAugment数据增强方法。在公开的小肠胶囊内镜图像Kvasir-Capsule数据集中,基于Swin Transformer网络,采用Adapt-RandAugment数据增强方法进行训练,以卷积神经网络ResNet152、DenseNet161为基准,验证Swin Transformer网络和Adapt-RandAugment数据增强方法组合对小肠胶囊内镜图像分类识别的性能。结果:提出的方法宏平均精度(macro average precision,MAC-PRE)、宏平均召回率(macro average recall,MAC-REC)、宏F1分数(macro average F1 score,MAC-F1-S)分别为0.3832、0.3148、0.2905,微平均精度(micro average precision,MIC-PRE)、微平均召回率(micro average recall,MIC-REC)、微平均F1分数(micro average F1 score,MIC-F1-S)均为0.7553,马修斯相关系数(Matthews correlation coefficient,MCC)为0.4523,均优于ResNet152和DenseNet161网络。结论:基于Swin Transformer网络与Adapt-RandAugment数据增强方法的小肠胶囊内镜图像分类方法具有较好的小肠胶囊内镜图像分类识别效果和较高的识别准确率。
基金supported by China Southern Power Grid Science and Technology Innovation Research Project(000000KK52220052).
文摘The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the prices of power transformer materials manifest as nonsmooth and nonlinear sequences.Hence,estimating the acquisition costs of power grid projects is difficult,hindering the normal operation of power engineering construction.To more accurately predict the price of power transformer materials,this study proposes a method based on complementary ensemble empirical mode decomposition(CEEMD)and gated recurrent unit(GRU)network.First,the CEEMD decomposed the price series into multiple intrinsic mode functions(IMFs).Multiple IMFs were clustered to obtain several aggregated sequences based on the sample entropy of each IMF.Then,an empirical wavelet transform(EWT)was applied to the aggregation sequence with a large sample entropy,and the multiple subsequences obtained from the decomposition were predicted by the GRU model.The GRU model was used to directly predict the aggregation sequences with a small sample entropy.In this study,we used authentic historical pricing data for power transformer materials to validate the proposed approach.The empirical findings demonstrated the efficacy of our method across both datasets,with mean absolute percentage errors(MAPEs)of less than 1%and 3%.This approach holds a significant reference value for future research in the field of power transformer material price prediction.
基金the Strategic Priority Research Program of Chinese Academy of Sciences(Grant No.XDC02040300)for this study.
文摘RFID-based human activity recognition(HAR)attracts attention due to its convenience,noninvasiveness,and privacy protection.Existing RFID-based HAR methods use modeling,CNN,or LSTM to extract features effectively.Still,they have shortcomings:1)requiring complex hand-crafted data cleaning processes and 2)only addressing single-person activity recognition based on specific RF signals.To solve these problems,this paper proposes a novel device-free method based on Time-streaming Multiscale Transformer called TransTM.This model leverages the Transformer's powerful data fitting capabilities to take raw RFID RSSI data as input without pre-processing.Concretely,we propose a multiscale convolutional hybrid Transformer to capture behavioral features that recognizes singlehuman activities and human-to-human interactions.Compared with existing CNN-and LSTM-based methods,the Transformer-based method has more data fitting power,generalization,and scalability.Furthermore,using RF signals,our method achieves an excellent classification effect on human behaviorbased classification tasks.Experimental results on the actual RFID datasets show that this model achieves a high average recognition accuracy(99.1%).The dataset we collected for detecting RFID-based indoor human activities will be published.
基金the Department of Informatics,Modeling,Electronics and Systems(DIMES)University of Calabria(Grant/Award Number:SIMPATICO_ZUMPANO).
文摘A consumption of 46.9 million tons of processed tomatoes was reported in 2022 which is merely 20%of the total consumption.An increase of 3.3%in consumption is predicted from 2024 to 2032.Tomatoes are also rich in iron,potassium,antioxidant lycopene,vitamins A,C and K which are important for preventing cancer,and maintaining blood pressure and glucose levels.Thus,tomatoes are globally important due to their widespread usage and nutritional value.To face the high demand for tomatoes,it is mandatory to investigate the causes of crop loss and minimize them.Diseases are one of the major causes that adversely affect crop yield and degrade the quality of the tomato fruit.This leads to financial losses and affects the livelihood of farmers.Therefore,automatic disease detection at any stage of the tomato plant is a critical issue.Deep learning models introduced in the literature show promising results,but the models are difficult to implement on handheld devices such as mobile phones due to high computational costs and a large number of parameters.Also,most of the models proposed so far work efficiently for images with plain backgrounds where a clear demarcation exists between the background and leaf region.Moreover,the existing techniques lack in recognizing multiple diseases on the same leaf.To address these concerns,we introduce a customized deep learning-based convolution vision transformer model.Themodel achieves an accuracy of 93.51%for classifying tomato leaf images with plain as well as complex backgrounds into 13 categories.It requires a space storage of merely 5.8 MB which is 98.93%,98.33%,and 92.64%less than stateof-the-art visual geometry group,vision transformers,and convolution vision transformermodels,respectively.Its training time of 44 min is 51.12%,74.12%,and 57.7%lower than the above-mentioned models.Thus,it can be deployed on(Internet of Things)IoT-enabled devices,drones,or mobile devices to assist farmers in the real-time monitoring of tomato crops.The periodicmonitoring promotes timely action to prevent the spread of diseases and reduce crop loss.
基金supported by the National Natural Science Foundation of China under Grant 62177029the Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX21_0740),China.
文摘Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.
基金Guangdong Basic and Applied Basic Research Foundation under Grant No.2024A1515012485in part by the Shenzhen Fundamental Research Program under Grant JCYJ20220810112354002.
文摘This paper addresses the problem of predicting population density leveraging cellular station data.As wireless communication devices are commonly used,cellular station data has become integral for estimating population figures and studying their movement,thereby implying significant contributions to urban planning.However,existing research grapples with issues pertinent to preprocessing base station data and the modeling of population prediction.To address this,we propose methodologies for preprocessing cellular station data to eliminate any irregular or redundant data.The preprocessing reveals a distinct cyclical characteristic and high-frequency variation in population shift.Further,we devise a multi-view enhancement model grounded on the Transformer(MVformer),targeting the improvement of the accuracy of extended time-series population predictions.Comparative experiments,conducted on the above-mentioned population dataset using four alternate Transformer-based models,indicate that our proposedMVformer model enhances prediction accuracy by approximately 30%for both univariate and multivariate time-series prediction assignments.The performance of this model in tasks pertaining to population prediction exhibits commendable results.
