Enhancing Dense Small Object Detection in UAV Images Based on Hybrid Transformer

Transformer-based models have facilitated significant advances in object detection.However,their extensive computational consumption and suboptimal detection of dense small objects curtail their applicability in unmanned aerial vehicle(UAV)imagery.Addressing these limitations,we propose a hybrid transformer-based detector,H-DETR,and enhance it for dense small objects,leading to an accurate and efficient model.Firstly,we introduce a hybrid transformer encoder,which integrates a convolutional neural network-based cross-scale fusion module with the original encoder to handle multi-scale feature sequences more efficiently.Furthermore,we propose two novel strategies to enhance detection performance without incurring additional inference computation.Query filter is designed to cope with the dense clustering inherent in drone-captured images by counteracting similar queries with a training-aware non-maximum suppression.Adversarial denoising learning is a novel enhancement method inspired by adversarial learning,which improves the detection of numerous small targets by counteracting the effects of artificial spatial and semantic noise.Extensive experiments on the VisDrone and UAVDT datasets substantiate the effectiveness of our approach,achieving a significant improvement in accuracy with a reduction in computational complexity.Our method achieves 31.9%and 21.1%AP on the VisDrone and UAVDT datasets,respectively,and has a faster inference speed,making it a competitive model in UAV image object detection.

Local saliency consistency-based label inference for weakly supervised salient object detection using scribble annotations

Recently,weak supervision has received growing attention in the field of salient object detection due to the convenience of labelling.However,there is a large performance gap between weakly supervised and fully supervised salient object detectors because the scribble annotation can only provide very limited foreground/background information.Therefore,an intuitive idea is to infer annotations that cover more complete object and background regions for training.To this end,a label inference strategy is proposed based on the assumption that pixels with similar colours and close positions should have consistent labels.Specifically,k-means clustering algorithm was first performed on both colours and coordinates of original annotations,and then assigned the same labels to points having similar colours with colour cluster centres and near coordinate cluster centres.Next,the same annotations for pixels with similar colours within each kernel neighbourhood was set further.Extensive experiments on six benchmarks demonstrate that our method can significantly improve the performance and achieve the state-of-the-art results.

MSC-YOLO:Improved YOLOv7 Based on Multi-Scale Spatial Context for Small Object Detection in UAV-View

Accurately identifying small objects in high-resolution aerial images presents a complex and crucial task in thefield of small object detection on unmanned aerial vehicles(UAVs).This task is challenging due to variations inUAV flight altitude,differences in object scales,as well as factors like flight speed and motion blur.To enhancethe detection efficacy of small targets in drone aerial imagery,we propose an enhanced You Only Look Onceversion 7(YOLOv7)algorithm based on multi-scale spatial context.We build the MSC-YOLO model,whichincorporates an additional prediction head,denoted as P2,to improve adaptability for small objects.We replaceconventional downsampling with a Spatial-to-Depth Convolutional Combination(CSPDC)module to mitigatethe loss of intricate feature details related to small objects.Furthermore,we propose a Spatial Context Pyramidwith Multi-Scale Attention(SCPMA)module,which captures spatial and channel-dependent features of smalltargets acrossmultiple scales.This module enhances the perception of spatial contextual features and the utilizationof multiscale feature information.On the Visdrone2023 and UAVDT datasets,MSC-YOLO achieves remarkableresults,outperforming the baseline method YOLOv7 by 3.0%in terms ofmean average precision(mAP).The MSCYOLOalgorithm proposed in this paper has demonstrated satisfactory performance in detecting small targets inUAV aerial photography,providing strong support for practical applications.

CAW-YOLO:Cross-Layer Fusion and Weighted Receptive Field-Based YOLO for Small Object Detection in Remote Sensing

In recent years,there has been extensive research on object detection methods applied to optical remote sensing images utilizing convolutional neural networks.Despite these efforts,the detection of small objects in remote sensing remains a formidable challenge.The deep network structure will bring about the loss of object features,resulting in the loss of object features and the near elimination of some subtle features associated with small objects in deep layers.Additionally,the features of small objects are susceptible to interference from background features contained within the image,leading to a decline in detection accuracy.Moreover,the sensitivity of small objects to the bounding box perturbation further increases the detection difficulty.In this paper,we introduce a novel approach,Cross-Layer Fusion and Weighted Receptive Field-based YOLO(CAW-YOLO),specifically designed for small object detection in remote sensing.To address feature loss in deep layers,we have devised a cross-layer attention fusion module.Background noise is effectively filtered through the incorporation of Bi-Level Routing Attention(BRA).To enhance the model’s capacity to perceive multi-scale objects,particularly small-scale objects,we introduce a weightedmulti-receptive field atrous spatial pyramid poolingmodule.Furthermore,wemitigate the sensitivity arising from bounding box perturbation by incorporating the joint Normalized Wasserstein Distance(NWD)and Efficient Intersection over Union(EIoU)losses.The efficacy of the proposedmodel in detecting small objects in remote sensing has been validated through experiments conducted on three publicly available datasets.The experimental results unequivocally demonstrate the model’s pronounced advantages in small object detection for remote sensing,surpassing the performance of current mainstream models.

A Novel Tensor Decomposition-Based Efficient Detector for Low-Altitude Aerial Objects With Knowledge Distillation Scheme

Unmanned aerial vehicles(UAVs) have gained significant attention in practical applications, especially the low-altitude aerial(LAA) object detection imposes stringent requirements on recognition accuracy and computational resources. In this paper, the LAA images-oriented tensor decomposition and knowledge distillation-based network(TDKD-Net) is proposed,where the TT-format TD(tensor decomposition) and equalweighted response-based KD(knowledge distillation) methods are designed to minimize redundant parameters while ensuring comparable performance. Moreover, some robust network structures are developed, including the small object detection head and the dual-domain attention mechanism, which enable the model to leverage the learned knowledge from small-scale targets and selectively focus on salient features. Considering the imbalance of bounding box regression samples and the inaccuracy of regression geometric factors, the focal and efficient IoU(intersection of union) loss with optimal transport assignment(F-EIoU-OTA)mechanism is proposed to improve the detection accuracy. The proposed TDKD-Net is comprehensively evaluated through extensive experiments, and the results have demonstrated the effectiveness and superiority of the developed methods in comparison to other advanced detection algorithms, which also present high generalization and strong robustness. As a resource-efficient precise network, the complex detection of small and occluded LAA objects is also well addressed by TDKD-Net, which provides useful insights on handling imbalanced issues and realizing domain adaptation.

Few-Shot Object Detection Based on the Transformer and High-Resolution Network

Now object detection based on deep learning tries different strategies.It uses fewer data training networks to achieve the effect of large dataset training.However,the existing methods usually do not achieve the balance between network parameters and training data.It makes the information provided by a small amount of picture data insufficient to optimize model parameters,resulting in unsatisfactory detection results.To improve the accuracy of few shot object detection,this paper proposes a network based on the transformer and high-resolution feature extraction(THR).High-resolution feature extractionmaintains the resolution representation of the image.Channels and spatial attention are used to make the network focus on features that are more useful to the object.In addition,the recently popular transformer is used to fuse the features of the existing object.This compensates for the previous network failure by making full use of existing object features.Experiments on the Pascal VOC and MS-COCO datasets prove that the THR network has achieved better results than previous mainstream few shot object detection.

UGC-YOLO:Underwater Environment Object Detection Based on YOLO with a Global Context Block

With the continuous development and utilization of marine resources,the underwater target detection has gradually become a popular research topic in the field of underwater robot operations and target detection.However,it is difficult to combine the environmental semantic information and the semantic information of targets at different scales by detection algorithms due to the complex underwater environment.In this paper,a cascade model based on the UGC-YOLO network structure with high detection accuracy is proposed.The YOLOv3 convolutional neural network is employed as the baseline structure.By fusing the global semantic information between two residual stages in the parallel structure of the feature extraction network,the perception of underwater targets is improved and the detection rate of hard-to-detect underwater objects is raised.Furthermore,the deformable convolution is applied to capture longrange semantic dependencies and PPM pooling is introduced in the highest layer network for aggregating semantic information.Finally,a multi-scale weighted fusion approach is presented for learning semantic information at different scales.Experiments are conducted on an underwater test dataset and the results have demonstrated that our proposed algorithm could detect aquatic targets in complex degraded underwater images.Compared with the baseline network algorithm,the Common Objects in Context(COCO)evaluation metric has been improved by 4.34%.

Entropy Based Feature Fusion Using Deep Learning for Waste Object Detection and Classification Model

Object Detection is the task of localization and classification of objects in a video or image.In recent times,because of its widespread applications,it has obtained more importance.In the modern world,waste pollution is one significant environmental problem.The prominence of recycling is known very well for both ecological and economic reasons,and the industry needs higher efficiency.Waste object detection utilizing deep learning(DL)involves training a machine-learning method to classify and detect various types of waste in videos or images.This technology is utilized for several purposes recycling and sorting waste,enhancing waste management and reducing environmental pollution.Recent studies of automatic waste detection are difficult to compare because of the need for benchmarks and broadly accepted standards concerning the employed data andmetrics.Therefore,this study designs an Entropy-based Feature Fusion using Deep Learning forWasteObject Detection and Classification(EFFDL-WODC)algorithm.The presented EFFDL-WODC system inherits the concepts of feature fusion and DL techniques for the effectual recognition and classification of various kinds of waste objects.In the presented EFFDL-WODC system,two major procedures can be contained,such as waste object detection and waste object classification.For object detection,the EFFDL-WODC technique uses a YOLOv7 object detector with a fusionbased backbone network.In addition,entropy feature fusion-based models such as VGG-16,SqueezeNet,and NASNetmodels are used.Finally,the EFFDL-WODC technique uses a graph convolutional network(GCN)model performed for the classification of detected waste objects.The performance validation of the EFFDL-WODC approach was validated on the benchmark database.The comprehensive comparative results demonstrated the improved performance of the EFFDL-WODC technique over recent approaches.

Night Vision Object Tracking System Using Correlation Aware LSTM-Based Modified Yolo Algorithm

Improved picture quality is critical to the effectiveness of object recog-nition and tracking.The consistency of those photos is impacted by night-video systems because the contrast between high-profile items and different atmospheric conditions,such as mist,fog,dust etc.The pictures then shift in intensity,colour,polarity and consistency.A general challenge for computer vision analyses lies in the horrid appearance of night images in arbitrary illumination and ambient envir-onments.In recent years,target recognition techniques focused on deep learning and machine learning have become standard algorithms for object detection with the exponential growth of computer performance capabilities.However,the iden-tification of objects in the night world also poses further problems because of the distorted backdrop and dim light.The Correlation aware LSTM based YOLO(You Look Only Once)classifier method for exact object recognition and deter-mining its properties under night vision was a major inspiration for this work.In order to create virtual target sets similar to daily environments,we employ night images as inputs;and to obtain high enhanced image using histogram based enhancement and iterative wienerfilter for removing the noise in the image.The process of the feature extraction and feature selection was done for electing the potential features using the Adaptive internal linear embedding(AILE)and uplift linear discriminant analysis(ULDA).The region of interest mask can be segmen-ted using the Recurrent-Phase Level set Segmentation.Finally,we use deep con-volution feature fusion and region of interest pooling to integrate the presently extremely sophisticated quicker Long short term memory based(LSTM)with YOLO method for object tracking system.A range of experimentalfindings demonstrate that our technique achieves high average accuracy with a precision of 99.7%for object detection of SSAN datasets that is considerably more than that of the other standard object detection mechanism.Our approach may therefore satisfy the true demands of night scene target detection applications.We very much believe that our method will help future research.

Few-shot object detection based on positive-sample improvement

Traditional object detectors based on deep learning rely on plenty of labeled samples,which are expensive to obtain.Few-shot object detection(FSOD)attempts to solve this problem,learning detection objects from a few labeled samples,but the performance is often unsatisfactory due to the scarcity of samples.We believe that the main reasons that restrict the performance of few-shot detectors are:(1)the positive samples is scarce,and(2)the quality of positive samples is low.Therefore,we put forward a novel few-shot object detector based on YOLOv4,starting from both improving the quantity and quality of positive samples.First,we design a hybrid multivariate positive sample augmentation(HMPSA)module to amplify the quantity of positive samples and increase positive sample diversity while suppressing negative samples.Then,we design a selective non-local fusion attention(SNFA)module to help the detector better learn the target features and improve the feature quality of positive samples.Finally,we optimize the loss function to make it more suitable for the task of FSOD.Experimental results on PASCAL VOC and MS COCO demonstrate that our designed few-shot object detector has competitive performance with other state-of-the-art detectors.

All-optical object identification and threedimensional reconstruction based on optical computing metasurface

Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurface,as a two-dimensional artificial design component,has displayed the supernormal character of controlling phase,amplitude,polarization,and frequency distributions of the light beam,capable of performing mathematical operations on the input light field.Here,we propose and demonstrate an all-optical object identification technique based on optical computing metasurface,and apply it to 3D reconstruction.Unlike traditional mechanisms,this scheme reduces memory consumption in the processing of the contour surface extraction.The identification and reconstruction of experimental results from high-contrast and low-contrast objects agree well with the real objects.The exploration of the all-optical object identification and 3D reconstruction techniques provides potential applications of high efficiencies,low consumption,and compact systems.

DSAFF-Net:A Backbone Network Based on Mask R-CNN for Small Object Detection

Recently,object detection based on convolutional neural networks(CNNs)has developed rapidly.The backbone networks for basic feature extraction are an important component of the whole detection task.Therefore,we present a new feature extraction strategy in this paper,which name is DSAFF-Net.In this strategy,we design:1)a sandwich attention feature fusion module(SAFF module).Its purpose is to enhance the semantic information of shallow features and resolution of deep features,which is beneficial to small object detection after feature fusion.2)to add a new stage called D-block to alleviate the disadvantages of decreasing spatial resolution when the pooling layer increases the receptive field.The method proposed in the new stage replaces the original method of obtaining the P6 feature map and uses the result as the input of the regional proposal network(RPN).In the experimental phase,we use the new strategy to extract features.The experiment takes the public dataset of Microsoft Common Objects in Context(MS COCO)object detection and the dataset of Corona Virus Disease 2019(COVID-19)image classification as the experimental object respectively.The results show that the average recognition accuracy of COVID-19 in the classification dataset is improved to 98.163%,and small object detection in object detection tasks is improved by 4.0%.

Automatic Examination of Condition of Used Books with YOLO-Based Object Detection Framework

As the demand for used books has grown in recent years,various online/offline market platforms have emerged to support the trade in used books.The price of used books can depend on various factors,such as the state of preservation(i.e.,condition),the value of possession,and so on.Therefore,some online platforms provide a reference document to evaluate the condition of used books,but it is still not trivial for individual sellers to determine the price.The lack of a standard quantitative method to assess the condition of the used book would confuse both sellers and consumers,thereby decreasing the user experience of the online secondhand marketplace.Therefore,this paper discusses the automatic examination of the condition of used books based on deep learning approaches.In this work,we present a book damage detection system based on various You Only Look Once(YOLO)object detection models.Using YOLOv5,YOLOR,and YOLOX,we also introduce various training configurations that can be applied to improve performance.Specifically,a combination of different augmentation strategies including flip,rotation,crop,mosaic,and mixup was used for comparison.To train and validate our system,a book damage dataset composed of a total of 620 book images with 3,989 annotations,containing six types of damages(i.e.,Wear,Spot,Notch,Barcode,Tag,and Ripped),collected from the library books is presented.We evaluated each model trained with different configurations to figure out their detection accuracy as well as training efficiency.The experimental results showed that YOLOX trained with its best training configuration yielded the best performance in terms of detection accuracy,by achieving 60.0%(mAP@.5:.95)and 72.9%(mAP@.5)for book damage detection.However,YOLOX performed worst in terms of training efficiency,indicating that there is a trade-off between accuracy and efficiency.Based on the findings from the study,we discuss the feasibility and limitations of our system and future research directions.

Smart object recommendation based on topic learning and joint features in the social internet of things

With the extensive integration of the Internet,social networks and the internet of things,the social internet of things has increasingly become a significant research issue.In the social internet of things application scenario,one of the greatest challenges is how to accurately recommend or match smart objects for users with massive resources.Although a variety of recommendation algorithms have been employed in this field,they ignore the massive text resources in the social internet of things,which can effectively improve the effect of recommendation.In this paper,a smart object recommendation approach named object recommendation based on topic learning and joint features is proposed.The proposed approach extracts and calculates topics and service relevant features of texts related to smart objects and introduces the“thing-thing”relationship information in the internet of things to improve the effect of recommendation.Experiments show that the proposed approach enables higher accuracy compared to the existing recommendation methods.

Visual SLAM Based on Object Detection Network:A Review

Visual simultaneous localization and mapping(SLAM)is crucial in robotics and autonomous driving.However,traditional visual SLAM faces challenges in dynamic environments.To address this issue,researchers have proposed semantic SLAM,which combines object detection,semantic segmentation,instance segmentation,and visual SLAM.Despite the growing body of literature on semantic SLAM,there is currently a lack of comprehensive research on the integration of object detection and visual SLAM.Therefore,this study aims to gather information from multiple databases and review relevant literature using specific keywords.It focuses on visual SLAM based on object detection,covering different aspects.Firstly,it discusses the current research status and challenges in this field,highlighting methods for incorporating semantic information from object detection networks into mileage measurement,closed-loop detection,and map construction.It also compares the characteristics and performance of various visual SLAM object detection algorithms.Lastly,it provides an outlook on future research directions and emerging trends in visual SLAM.Research has shown that visual SLAM based on object detection has significant improvements compared to traditional SLAM in dynamic point removal,data association,point cloud segmentation,and other technologies.It can improve the robustness and accuracy of the entire SLAM system and can run in real time.With the continuous optimization of algorithms and the improvement of hardware level,object visual SLAM has great potential for development.

Robust Deep Transfer Learning Based Object Detection and Tracking Approach

At present days,object detection and tracking concepts have gained more importance among researchers and business people.Presently,deep learning(DL)approaches have been used for object tracking as it increases the perfor-mance and speed of the tracking process.This paper presents a novel robust DL based object detection and tracking algorithm using Automated Image Anno-tation with ResNet based Faster regional convolutional neural network(R-CNN)named(AIA-FRCNN)model.The AIA-RFRCNN method performs image anno-tation using a Discriminative Correlation Filter(DCF)with Channel and Spatial Reliability tracker(CSR)called DCF-CSRT model.The AIA-RFRCNN model makes use of Faster RCNN as an object detector and tracker,which involves region proposal network(RPN)and Fast R-CNN.The RPN is a full convolution network that concurrently predicts the bounding box and score of different objects.The RPN is a trained model used for the generation of the high-quality region proposals,which are utilized by Fast R-CNN for detection process.Besides,Residual Network(ResNet 101)model is used as a shared convolutional neural network(CNN)for the generation of feature maps.The performance of the ResNet 101 model is further improved by the use of Adam optimizer,which tunes the hyperparameters namely learning rate,batch size,momentum,and weight decay.Finally,softmax layer is applied to classify the images.The performance of the AIA-RFRCNN method has been assessed using a benchmark dataset and a detailed comparative analysis of the results takes place.The outcome of the experiments indicated the superior characteristics of the AIA-RFRCNN model under diverse aspects.

Visual Object Tracking Based on Modified LeNet-5 and RCCF

The field of object tracking has recently made significant progress.Particularly,the performance results in both deep learning and correlation filters,based trackers achieved effective tracking performance.Moreover,there are still some difficulties with object tracking for example illumination and deformation(DEF).The precision and accuracy of tracking algorithms suffer from the effects of such occurrences.For this situation,finding a solution is important.This research proposes a new tracking algorithm to handle this problem.The features are extracted by using Modified LeNet-5,and the precision and accuracy are improved by developing the Real-Time Cross-modality Correlation Filtering method(RCCF).In Modified LeNet-5,the visual tracking performance is improved by adjusting the number and size of the convolution kernels in the pooling and convolution layers.The high-level,middle-level,and handcraft features are extracted from the modified LeNet-5 network.The handcraft features are used to determine the specific location of the target because the handcraft features contain more spatial information regarding the visual object.The LeNet features are more suitable for a target appearance change in object tracking.Extensive experiments were conducted by the Object Tracking Benchmarking(OTB)databases like OTB50 and OTB100.The experimental results reveal that the proposed tracker outperforms other state-of-the-art trackers under different problems.The experimental simulation is carried out in python.The overall success rate and precision of the proposed algorithm are 93.8%and 92.5%.The average running frame rate reaches 42 frames per second,which can meet the real-time requirements.

An Ontology Based Multilayer Perceptron for Object Detection

In object detection,spatial knowledge assisted systems are effective.Object detection is a main and challenging issue to analyze object-related information.Several existing object detection techniques were developed to consider the object detection problem as a classification problem to perform feature selection and classification.But these techniques still face,less computational efficiency and high time consumption.This paper resolves the above limitations using the Fuzzy Tversky index Ontology-based Multi-Layer Perception method which improves the accuracy of object detection with minimum time.The proposed method uses a multilayer forfinding the similarity score.A fuzzy membership function is used to validate the score for predicting the burned and non-burned zone.Experimental assessment is performed with different factors such as classification rate,time complexity,error rate,space complexity,and precision by using the forestfire dataset.The results show that this novel technique can help to improve the classification rate and reduce the time and space complexity as well as error rate than the conventional methods.

Object Tracking Algorithm Based on Multi-Time-Space Perception and Instance-Specific Proposals

Aiming at the problem that a single correlation filter model is sensitive to complex scenes such as background interference and occlusion,a tracking algorithm based on multi-time-space perception and instance-specific proposals is proposed to optimize the mathematical model of the correlation filter(CF).Firstly,according to the consistency of the changes between the object frames and the filter frames,the mask matrix is introduced into the objective function of the filter,so as to extract the spatio-temporal information of the object with background awareness.Secondly,the object function of multi-feature fusion is constructed for the object location,which is optimized by the Lagrange method and solved by closed iteration.In the process of filter optimization,the constraints term of time-space perception is designed to enhance the learning ability of the CF to optimize the final track-ing results.Finally,when the tracking results fluctuate,the boundary suppres-sion factor is introduced into the instance-specific proposals to reduce the risk of model drift effectively.The accuracy and success rate of the proposed algorithm are verified by simulation analysis on two popular benchmarks,the object tracking benchmark 2015(OTB2015)and the temple color 128(TC-128).Extensive experimental results illustrate that the optimized appearance model of the proposed algorithm is effective.The distance precision rate and overlap success rate of the proposed algorithm are 0.756 and 0.656 on the OTB2015 benchmark,which are better than the results of other competing algorithms.The results of this study can solve the problem of real-time object tracking in the real traffic environment and provide a specific reference for the detection of traffic abnormalities.

Application of 3D Scanned Big Data of Large-scale Cultural Heritage Objects Based on Noise-robust Transparent Visualization

Three-dimensional(3D) scanning technology has undergone remarkable developments in recent years.Data acquired by 3D scanning have the form of 3D point clouds.The 3D scanned point clouds have data sizes that can be considered big data.They also contain measurement noise inherent in measurement data.These properties of 3D scanned point clouds make many traditional CG/visualization techniques difficult.This paper reviewed our recent achievements in developing varieties of high-quality visualizations suitable for the visual analysis of 3D scanned point clouds.We demonstrated the effectiveness of the method by applying the visualizations to various cultural heritage objects.The main visualization targets used in this paper are the floats in the Gion Festival in Kyoto(the float parade is on the UNESCO Intangible Cultural Heritage List) and Borobudur Temple in Indonesia(a UNESCO World Heritage Site).