MSSTNet:Multi-scale facial videos pulse extraction network based on separable spatiotemporal convolution and dimension separable attention

Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.

Identification of Key Links in Electric Power Operation Based-Spatiotemporal Mixing Convolution Neural Network

As the scale of the power system continues to expand,the environment for power operations becomes more and more complex.Existing risk management and control methods for power operations can only set the same risk detection standard and conduct the risk detection for any scenario indiscriminately.Therefore,more reliable and accurate security control methods are urgently needed.In order to improve the accuracy and reliability of the operation risk management and control method,this paper proposes a method for identifying the key links in the whole process of electric power operation based on the spatiotemporal hybrid convolutional neural network.To provide early warning and control of targeted risks,first,the video stream is framed adaptively according to the pixel changes in the video stream.Then,the optimized MobileNet is used to extract the feature map of the video stream,which contains both time-series and static spatial scene information.The feature maps are combined and non-linearly mapped to realize the identification of dynamic operating scenes.Finally,training samples and test samples are produced by using the whole process image of a power company in Xinjiang as a case study,and the proposed algorithm is compared with the unimproved MobileNet.The experimental results demonstrated that the method proposed in this paper can accurately identify the type and start and end time of each operation link in the whole process of electric power operation,and has good real-time performance.The average accuracy of the algorithm can reach 87.8%,and the frame rate is 61 frames/s,which is of great significance for improving the reliability and accuracy of security control methods.

Prediction of Spatiotemporal Evolution of Urban Traffic Emissions Based on Taxi Trajectories

With the rapid increase of the amount of vehicles in urban areas,the pollution of vehicle emissions is becoming more and more serious.Precise prediction of the spatiotemporal evolution of urban traffic emissions plays a great role in urban planning and policy making.Most existing methods usually focus on estimating vehicle emissions at historical or current moments which cannot well meet the demands of future planning.Recent work has started to pay attention to the evolution of vehicle emissions at future moments using multiple attributes related to emissions,however,they are not effective and efficient enough in the combination and utilization of different inputs.To address this issue,we propose a joint framework to predict the future evolution of vehicle emissions based on the GPS trajectories of taxis with a multi-channel spatiotemporal network and the motor vehicle emission simulator(MOVES)model.Specifically,we first estimate the spatial distribution matrices with GPS trajectories through map-matching algorithms.These matrices can reflect the attributes related to the traffic status of road networks such as volume,speed and acceleration.Then,our multi-channel spatiotemporal network is used to efficiently combine three key attributes(volume,speed and acceleration)through the feature sharing mechanism and generate a precise prediction of them in the future period.Finally,we adopt an MOVES model to estimate vehicle emissions by integrating several traffic factors including the predicted traffic states,road networks and the statistical information of urban vehicles.We evaluate our model on the Xi′an taxi GPS trajectories dataset.Experiments show that our proposed network can effectively predict the temporal evolution of vehicle emissions.