Multi-Stream Temporally Enhanced Network for Video Salient Object Detection

Video salient object detection(VSOD)aims at locating the most attractive objects in a video by exploring the spatial and temporal features.VSOD poses a challenging task in computer vision,as it involves processing complex spatial data that is also influenced by temporal dynamics.Despite the progress made in existing VSOD models,they still struggle in scenes of great background diversity within and between frames.Additionally,they encounter difficulties related to accumulated noise and high time consumption during the extraction of temporal features over a long-term duration.We propose a multi-stream temporal enhanced network(MSTENet)to address these problems.It investigates saliency cues collaboration in the spatial domain with a multi-stream structure to deal with the great background diversity challenge.A straightforward,yet efficient approach for temporal feature extraction is developed to avoid the accumulative noises and reduce time consumption.The distinction between MSTENet and other VSOD methods stems from its incorporation of both foreground supervision and background supervision,facilitating enhanced extraction of collaborative saliency cues.Another notable differentiation is the innovative integration of spatial and temporal features,wherein the temporal module is integrated into the multi-stream structure,enabling comprehensive spatial-temporal interactions within an end-to-end framework.Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance on five benchmark datasets while maintaining a real-time speed of 27 fps(Titan XP).Our code and models are available at https://github.com/RuJiaLe/MSTENet.

SwinVid:Enhancing Video Object Detection Using Swin Transformer

What causes object detection in video to be less accurate than it is in still images?Because some video frames have degraded in appearance from fast movement,out-of-focus camera shots,and changes in posture.These reasons have made video object detection(VID)a growing area of research in recent years.Video object detection can be used for various healthcare applications,such as detecting and tracking tumors in medical imaging,monitoring the movement of patients in hospitals and long-term care facilities,and analyzing videos of surgeries to improve technique and training.Additionally,it can be used in telemedicine to help diagnose and monitor patients remotely.Existing VID techniques are based on recurrent neural networks or optical flow for feature aggregation to produce reliable features which can be used for detection.Some of those methods aggregate features on the full-sequence level or from nearby frames.To create feature maps,existing VID techniques frequently use Convolutional Neural Networks(CNNs)as the backbone network.On the other hand,Vision Transformers have outperformed CNNs in various vision tasks,including object detection in still images and image classification.We propose in this research to use Swin-Transformer,a state-of-the-art Vision Transformer,as an alternative to CNN-based backbone networks for object detection in videos.The proposed architecture enhances the accuracy of existing VID methods.The ImageNet VID and EPIC KITCHENS datasets are used to evaluate the suggested methodology.We have demonstrated that our proposed method is efficient by achieving 84.3%mean average precision(mAP)on ImageNet VID using less memory in comparison to other leading VID techniques.The source code is available on the website https://github.com/amaharek/SwinVid.

Motion connectivity-based initial video object extraction

In order to obtain the initial video objects from the video sequences, an improved initial video object extraction algorithm based on motion connectivity is proposed. Moving objects in video sequences are highly connected and structured, which makes motion connectivity an advanced feature for segmentation. Accordingly, after sharp noise elimination, the cumulated difference image, which exhibits the coherent motion of the moving object, is adaptively thresholded. Then the maximal connected region is labeled, post-processed and output as the final segmenting mask. Hence the initial video object is effectively extracted. Comparative experimental results show that the proposed algorithm extracts the initial video object automatically, promptly and properly, thereby achieving satisfactory subjective and objective performance.

Real-time moving object detection for video monitoring systems

Moving object detection is one of the challenging problems in video monitoring systems, especially when the illumination changes and shadow exists. Amethod for real-time moving object detection is described. Anew background model is proposed to handle the illumination varition problem. With optical flow technology and background subtraction, a moving object is extracted quickly and accurately. An effective shadow elimination algorithm based on color features is used to refine the moving obj ects. Experimental results demonstrate that the proposed method can update the background exactly and quickly along with the varition of illumination, and the shadow can be eliminated effectively. The proposed algorithm is a real-time one which the foundation for further object recognition and understanding of video mum＇toting systems.

Scribble-Supervised Video Object Segmentation

Recently,video object segmentation has received great attention in the computer vision community.Most of the existing methods heavily rely on the pixel-wise human annotations,which are expensive and time-consuming to obtain.To tackle this problem,we make an early attempt to achieve video object segmentation with scribble-level supervision,which can alleviate large amounts of human labor for collecting the manual annotation.However,using conventional network architectures and learning objective functions under this scenario cannot work well as the supervision information is highly sparse and incomplete.To address this issue,this paper introduces two novel elements to learn the video object segmentation model.The first one is the scribble attention module,which captures more accurate context information and learns an effective attention map to enhance the contrast between foreground and background.The other one is the scribble-supervised loss,which can optimize the unlabeled pixels and dynamically correct inaccurate segmented areas during the training stage.To evaluate the proposed method,we implement experiments on two video object segmentation benchmark datasets,You Tube-video object segmentation(VOS),and densely annotated video segmentation(DAVIS)-2017.We first generate the scribble annotations from the original per-pixel annotations.Then,we train our model and compare its test performance with the baseline models and other existing works.Extensive experiments demonstrate that the proposed method can work effectively and approach to the methods requiring the dense per-pixel annotations.

Objective Performance Evaluation of Video Segmentation Algorithms with Ground-Truth

While the development of particular video segmentation algorithms has attracted considerable research interest, relatively little effort has been devoted to provide a methodology for evaluating their performance. In this paper, we propose a methodology to objectively evaluate video segmentation algorithm with ground-truth, which is based on computing the deviation of segmentation results from the reference segmentation. Four different metrics based on classification pixels, edges, relative foreground area and relative position respectively are combined to address the spatial accuracy. Temporal coherency is evaluated by utilizing the difference of spatial accuracy between successive frames. The experimental results show the feasibility of our approach. Moreover, it is computationally more efficient than previous methods. It can be applied to provide an offline ranking among different segmentation algorithms and to optimally set the parameters for a given algorithm.

Algorithm Research on Moving Object Detection of Surveillance Video Sequence

In video surveillance, there are many interference factors such as target changes, complex scenes, and target deformation in the moving object tracking. In order to resolve this issue, based on the comparative analysis of several common moving object detection methods, a moving object detection and recognition algorithm combined frame difference with background subtraction is presented in this paper. In the algorithm, we first calculate the average of the values of the gray of the continuous multi-frame image in the dynamic image, and then get background image obtained by the statistical average of the continuous image sequence, that is, the continuous interception of the N-frame images are summed, and find the average. In this case, weight of object information has been increasing, and also restrains the static background. Eventually the motion detection image contains both the target contour and more target information of the target contour point from the background image, so as to achieve separating the moving target from the image. The simulation results show the effectiveness of the proposed algorithm.

Realtime Object Detection Through M-ResNet in Video Surveillance System

Object detection plays a vital role in the video surveillance systems.To enhance security,surveillance cameras are now installed in public areas such as traffic signals,roadways,retail malls,train stations,and banks.However,monitor-ing the video continually at a quicker pace is a challenging job.As a consequence,security cameras are useless and need human monitoring.The primary difficulty with video surveillance is identifying abnormalities such as thefts,accidents,crimes,or other unlawful actions.The anomalous action does not occur at a high-er rate than usual occurrences.To detect the object in a video,first we analyze the images pixel by pixel.In digital image processing,segmentation is the process of segregating the individual image parts into pixels.The performance of segmenta-tion is affected by irregular illumination and/or low illumination.These factors highly affect the real-time object detection process in the video surveillance sys-tem.In this paper,a modified ResNet model(M-Resnet)is proposed to enhance the image which is affected by insufficient light.Experimental results provide the comparison of existing method output and modification architecture of the ResNet model shows the considerable amount improvement in detection objects in the video stream.The proposed model shows better results in the metrics like preci-sion,recall,pixel accuracy,etc.,andfinds a reasonable improvement in the object detection.

Real-time detection of moving objects in video sequences

An approach to detection of moving objects in video sequences, with application to video surveillance is presented. The algorithm combines two kinds of change points, which are detected from the region-based frame difference and adjusted background subtraction. An adaptive threshold technique is employed to automatically choose the threshold value to segment the moving objects from the still background. And experiment results show that the algorithm is effective and efficient in practical situations. Furthermore, the algorithm is robust to the effects of the changing of lighting condition and can be applied for video surveillance system.

Initial Object Segmentation for Video Object Plane Generation Using Cellular Neural Networks

MPEG 4 is a basic tool for interactivity and manipulation of video sequences. Video object segmentation is a key issue in defining the content of any video sequence, which is often divided into two steps: initial object segmentation and object tracking. In this paper, an initial object segmentation method for video object plane(VOP) generation using color information is proposed. Based on 3 by 3 linear templates, a cellular neural network (CNN) is used to implemented object segmentation. The Experimental results are presented to verify the efficiency and robustness of this approach.

Research on video motion object segmentation for content-based application

With the development of the modern information society, more and more multimedia information is available. So the technology of multimedia processing is becoming the important task for the irrelevant area of scientist. Among of the multimedia, the visual informarion is more attractive due to its direct, vivid characteristic, but at the same rime the huge amount of video data causes many challenges if the video storage, processing and transmission.

An Efficient Method for Underwater Video Summarization and Object Detection Using YoLoV3

Currently,worldwide industries and communities are concerned with building,expanding,and exploring the assets and resources found in the oceans and seas.More precisely,to analyze a stock,archaeology,and surveillance,sev-eral cameras are installed underseas to collect videos.However,on the other hand,these large size videos require a lot of time and memory for their processing to extract relevant information.Hence,to automate this manual procedure of video assessment,an accurate and efficient automated system is a greater necessity.From this perspective,we intend to present a complete framework solution for the task of video summarization and object detection in underwater videos.We employed a perceived motion energy(PME)method tofirst extract the keyframes followed by an object detection model approach namely YoloV3 to perform object detection in underwater videos.The issues of blurriness and low contrast in underwater images are also taken into account in the presented approach by applying the image enhancement method.Furthermore,the suggested framework of underwater video summarization and object detection has been evaluated on a publicly available brackish dataset.It is observed that the proposed framework shows good performance and hence ultimately assists several marine researchers or scientists related to thefield of underwater archaeology,stock assessment,and surveillance.

AUTOMATIC SEGMENTATION OF VIDEO OBJECT PLANES IN MPEG-4 BASED ON SPATIO-TEMPORAL INFORMATION

Segmentation of semantic Video Object Planes (VOP's) from video sequence is a key to the standard MPEG-4 with content-based video coding. In this paper, the approach of automatic Segmentation of VOP's Based on Spatio-Temporal Information (SBSTI) is proposed.The proceeding results demonstrate the good performance of the algorithm.

Visible-infrared person re-identification using query related cluster

Visible-infrared person re-identification(VIPR), is a cross-modal retrieval task that searches a target from a gallery captured by cameras of different spectrums.The severe challenge for VIPR is the large intra-class variation caused by the modal discrepancy between visible and infrared images.For that, this paper proposes a query related cluster(QRC) method for VIPR.Firstly, this paper uses an attention mechanism to calculate the similarity relation between a visible query and infrared images with the same identity in the gallery.Secondly, those infrared images with the same query images are aggregated by using the similarity relation to form a dynamic clustering center corresponding to the query image.Thirdly, QRC loss function is designed to enlarge the similarity between the query image and its dynamic cluster center to achieve query related clustering, so as to compact the intra-class variations.Consequently, in the proposed QRC method, each query has its own dynamic clustering center, which can well characterize intra-class variations in VIPR.Experimental results demonstrate that the proposed QRC method is superior to many state-of-the-art approaches, acquiring a 90.77% rank-1 identification rate on the RegDB dataset.

New Fragile Watermarking Technique to Identify Inserted Video Objects Using H.264 and Color Features

The transmission of video content over a network raises various issues relating to copyright authenticity,ethics,legality,and privacy.The protection of copyrighted video content is a significant issue in the video industry,and it is essential to find effective solutions to prevent tampering and modification of digital video content during its transmission through digital media.However,there are stillmany unresolved challenges.This paper aims to address those challenges by proposing a new technique for detectingmoving objects in digital videos,which can help prove the credibility of video content by detecting any fake objects inserted by hackers.The proposed technique involves using two methods,the H.264 and the extraction color features methods,to embed and extract watermarks in video frames.The study tested the performance of the system against various attacks and found it to be robust.The evaluation was done using different metrics such as Peak-Signal-to-Noise Ratio(PSNR),Mean Squared Error(MSE),Structural Similarity Index Measure(SSIM),Bit Correction Ratio(BCR),and Normalized Correlation.The accuracy of identifying moving objects was high,ranging from 96.3%to 98.7%.The system was also able to embed a fragile watermark with a success rate of over 93.65%and had an average capacity of hiding of 78.67.The reconstructed video frames had high quality with a PSNR of at least 65.45 dB and SSIMof over 0.97,making them imperceptible to the human eye.The system also had an acceptable average time difference(T=1.227/s)compared with other state-of-the-art methods.

VIDEO OBJECT SEGMENTATION BY 2-D MESH-BASED MOTION ANALYSIS

Video object extraction is a key technology in content-based video coding.A novel video object extracting algorithm by two Dimensional (2-D) mesh-based motion analysis is proposed in this paper.Firstly,a 2-D mesh fitting the original frame image is obtained via feature detection algorithm. Then,higher order statistics motion analysis is applied on the 2-D mesh representation to get an initial motion detection mask.After post-processing,the final segmenting mask is quickly obtained.And hence the video object is effectively extracted.Experimental results show that the proposed algorithm combines the merits of mesh-based segmenting algorithms and pixel-based segmenting algorithms,and hereby achieves satisfactory subjective and objective performance while dramatically increasing the segmenting speed.

Detection of Objects in Motion—A Survey of Video Surveillance

Video surveillance system is the most important issue in homeland security field. It is used as a security system because of its ability to track and to detect a particular person. To overcome the lack of the conventional video surveillance system that is based on human perception, we introduce a novel cognitive video surveillance system (CVS) that is based on mobile agents. CVS offers important attributes such as suspect objects detection and smart camera cooperation for people tracking. According to many studies, an agent-based approach is appropriate for distributed systems, since mobile agents can transfer copies of themselves to other servers in the system.

Rebound of Region of Interest (RROI), a New Kernel-Based Algorithm for Video Object Tracking Applications

This paper presents a new kernel-based algorithm for video object tracking called rebound of region of interest (RROI). The novel algorithm uses a rectangle-shaped section as region of interest (ROI) to represent and track specific objects in videos. The proposed algorithm is constituted by two stages. The first stage seeks to determine the direction of the object’s motion by analyzing the changing regions around the object being tracked between two consecutive frames. Once the direction of the object’s motion has been predicted, it is initialized an iterative process that seeks to minimize a function of dissimilarity in order to find the location of the object being tracked in the next frame. The main advantage of the proposed algorithm is that, unlike existing kernel-based methods, it is immune to highly cluttered conditions. The results obtained by the proposed algorithm show that the tracking process was successfully carried out for a set of color videos with different challenging conditions such as occlusion, illumination changes, cluttered conditions, and object scale changes.

From Sex Objects to Heroines A Tough Road for Female Characters in Video Games

Female character in video games is the one of the most controversial topic in game studies and nowadays women＇s anthropology. From the beginning of role in games＇ plot to sexualized graphical representation, woman in virtual world were （and sometimes still are） a point of discussion about characters sexualisation and role of female sex in virtual industry. The main aim of this paper is to present analysis of female representation in video games and how in last 30 years it has changed. In other words, how female characters were ennobled from sex object to main protagonist status. In presented research, there were used mostly the examples of popular video games with extended plot and world which can be explored by protagonists. Video game historiography, textual analysis （which helped to treat video game character as a protagonist）, and thematic analysis of video games were used as a main research method. The main conclusion of this paper is that female characters can overcome all gender or sexual stereotypes and even eventually became an icon of popular culture

Review of Image-Based Person Re-Identification in Deep Learning

Person Re-identification(re-ID)is a hot research topic in the field of computer vision now,which can be regarded as a sub-problem of image retrieval.The goal of person re-ID is to give a monitoring pedestrian image and retrieve other images of the pedestrian across the device.At present,person re-ID is mainly divided into two categories.One is the traditional methods,which relies heavily on manual features.The other is to use deep learning technology to solve.Because traditional methods mainly rely on manual feature,they cannot adapt well to a complex environment with a large amount of data.In recent years,with the development of deep learning technology,a large number of person re-ID methods based on deep learning have been proposed,which greatly improves the accuracy of person re-ID.