Deepfake Video Detection Based on Improved CapsNet and Temporal–Spatial Features

Rapid development of deepfake technology led to the spread of forged audios and videos across network platforms,presenting risks for numerous countries,societies,and individuals,and posing a serious threat to cyberspace security.To address the problem of insufficient extraction of spatial features and the fact that temporal features are not considered in the deepfake video detection,we propose a detection method based on improved CapsNet and temporal–spatial features(iCapsNet–TSF).First,the dynamic routing algorithm of CapsNet is improved using weight initialization and updating.Then,the optical flow algorithm is used to extract interframe temporal features of the videos to form a dataset of temporal–spatial features.Finally,the iCapsNet model is employed to fully learn the temporal–spatial features of facial videos,and the results are fused.Experimental results show that the detection accuracy of iCapsNet–TSF reaches 94.07%,98.83%,and 98.50%on the Celeb-DF,FaceSwap,and Deepfakes datasets,respectively,displaying a better performance than most existing mainstream algorithms.The iCapsNet–TSF method combines the capsule network and the optical flow algorithm,providing a novel strategy for the deepfake detection,which is of great significance to the prevention of deepfake attacks and the preservation of cyberspace security.

Re-Distributing Facial Features for Engagement Prediction with ModernTCN

Automatically detecting learners’engagement levels helps to develop more effective online teaching and assessment programs,allowing teachers to provide timely feedback and make personalized adjustments based on students’needs to enhance teaching effectiveness.Traditional approaches mainly rely on single-frame multimodal facial spatial information,neglecting temporal emotional and behavioural features,with accuracy affected by significant pose variations.Additionally,convolutional padding can erode feature maps,affecting feature extraction’s representational capacity.To address these issues,we propose a hybrid neural network architecture,the redistributing facial features and temporal convolutional network(RefEIP).This network consists of three key components:first,utilizing the spatial attention mechanism large kernel attention(LKA)to automatically capture local patches and mitigate the effects of pose variations;second,employing the feature organization and weight distribution(FOWD)module to redistribute feature weights and eliminate the impact of white features and enhancing representation in facial feature maps.Finally,we analyse the temporal changes in video frames through the modern temporal convolutional network(ModernTCN)module to detect engagement levels.We constructed a near-infrared engagement video dataset(NEVD)to better validate the efficiency of the RefEIP network.Through extensive experiments and in-depth studies,we evaluated these methods on the NEVD and the Database for Affect in Situations of Elicitation(DAiSEE),achieving an accuracy of 90.8%on NEVD and 61.2%on DAiSEE in the fourclass classification task,indicating significant advantages in addressing engagement video analysis problems.

How do temporal and spectral features .matter in crop classification in Heilongjiang Province, China？

How to fully use spectral and temporal information for efficient identification of crops becomes a crucial issue since each crop has its specific seasonal dynamics. A thorough understanding on the relative usefulness of spectral and temporal features is thus essential for better organization of crop classification information. This study, taking Heilongjiang Province as the study area, aims to use time-series moderate resolution imaging spectroradiometer （MODIS） surface reflectance product （MOD09A1） data to evaluate the importance of spectral and temporal features for crop classification. In doing so, a feature selection strategy based on separability index （SI） was first used to rank the most important spectro-temporal features for crop classification. Ten feature scenarios with different spectral and temporal variable combinations were then devised, which were used for crop classification using the support vector machine and their accuracies were finally assessed with the same crop samples. The results show that the normalized difference tillage index （NDTI）, land surface water index （LSWl） and enhanced vegetation index （EVI） are the most informative spectral features and late August to early September is the most informative temporal window for identifying crops in Heilongjiang for the observed year 2011. Spectral diversity and time variety are both vital for crop classification, and their combined use can improve the accuracy by about 30% in comparison with single image. The feature selection technique based on SI analysis is superior for achieving high crop classification accuracy （producers＇ accuracy of 94.03% and users＇ accuracy of 93.77%） with a small number of features. Increasing temporal resolution is not necessarily important for improving the classification accuracies for crops, and a relatively high classification accuracy can be achieved as long as the images associated with key phenological phrases are retained.

Assessing Landsat-8 and Sentinel-2 spectral-temporal features for mapping tree species of northern plantation forests in Heilongjiang Province,China

Background:Accurate mapping of tree species is highly desired in the management and research of plantation forests,whose ecosystem services are currently under threats.Time-series multispectral satellite images,e.g.,from Landsat-8(L8)and Sentinel-2(S2),have been proven useful in mapping general forest types,yet we do not know quantitatively how their spectral features(e.g.,red-edge)and temporal frequency of data acquisitions(e.g.,16-day vs.5-day)contribute to plantation forest mapping to the species level.Moreover,it is unclear to what extent the fusion of L8 and S2 will result in improvements in tree species mapping of northern plantation forests in China.Methods:We designed three sets of classification experiments(i.e.,single-date,multi-date,and spectral-temporal)to evaluate the performances of L8 and S2 data for mapping keystone timber tree species in northern China.We first used seven pairs of L8 and S2 images to evaluate the performances of L8 and S2 key spectral features for separating these tree species across key growing stages.Then we extracted the spectral-temporal features from all available images of different temporal frequency of data acquisition(i.e.,L8 time series,S2 time series,and fusion of L8 and S2)to assess the contribution of image temporal frequency on the accuracy of tree species mapping in the study area.Results:1)S2 outperformed L8 images in all classification experiments,with or without the red edge bands(0.4%–3.4%and 0.2%–4.4%higher for overall accuracy and macro-F1,respectively);2)NDTI(the ratio of SWIR1 minus SWIR2 to SWIR1 plus SWIR2)and Tasseled Cap coefficients were most important features in all the classifications,and for time-series experiments,the spectral-temporal features of red band-related vegetation indices were most useful;3)increasing the temporal frequency of data acquisition can improve overall accuracy of tree species mapping for up to 3.2%(from 90.1%using single-date imagery to 93.3%using S2 time-series),yet similar overall accuracies were achieved using S2 time-series(93.3%)and the fusion of S2 and L8(93.2%).Conclusions:This study quantifies the contributions of L8 and S2 spectral and temporal features in mapping keystone tree species of northern plantation forests in China and suggests that for mapping tree species in China's northern plantation forests,the effects of increasing the temporal frequency of data acquisition could saturate quickly after using only two images from key phenological stages.

STGSA:A Novel Spatial-Temporal Graph Synchronous Aggregation Model for Traffic Prediction

The success of intelligent transportation systems relies heavily on accurate traffic prediction,in which how to model the underlying spatial-temporal information from traffic data has come under the spotlight.Most existing frameworks typically utilize separate modules for spatial and temporal correlations modeling.However,this stepwise pattern may limit the effectiveness and efficiency in spatial-temporal feature extraction and cause the overlook of important information in some steps.Furthermore,it is lacking sufficient guidance from prior information while modeling based on a given spatial adjacency graph(e.g.,deriving from the geodesic distance or approximate connectivity),and may not reflect the actual interaction between nodes.To overcome those limitations,our paper proposes a spatial-temporal graph synchronous aggregation(STGSA)model to extract the localized and long-term spatial-temporal dependencies simultaneously.Specifically,a tailored graph aggregation method in the vertex domain is designed to extract spatial and temporal features in one graph convolution process.In each STGSA block,we devise a directed temporal correlation graph to represent the localized and long-term dependencies between nodes,and the potential temporal dependence is further fine-tuned by an adaptive weighting operation.Meanwhile,we construct an elaborated spatial adjacency matrix to represent the road sensor graph by considering both physical distance and node similarity in a datadriven manner.Then,inspired by the multi-head attention mechanism which can jointly emphasize information from different r epresentation subspaces,we construct a multi-stream module based on the STGSA blocks to capture global information.It projects the embedding input repeatedly with multiple different channels.Finally,the predicted values are generated by stacking several multi-stream modules.Extensive experiments are constructed on six real-world datasets,and numerical results show that the proposed STGSA model significantly outperforms the benchmarks.

Analysis of Temporal and Spatial Distribution and Large-Scale Circulation Features of Extreme Weather Events in Shanxi Province, China in Recent 30 Years

Extreme weather events such as persistent high temperatures, heavy rains or sudden cold waves in Shanxi Province in China have brought great losses and disasters to people’s production and life. It is of great practical significance to study the temporal and spatial distribution characteristics of extreme weather events and the circulation background field. We selected daily high temperature data (≥35°C), daily minimum temperature data and daily precipitation data (≥50 mm) from 109 meteorological stations in Shanxi Province, China from 1981 to 2010, then set the period in which the temperature is ≥35°C for more than 3 days as a high temperature extreme weather event, define the station in which 24 hour cumulative precipitation is ≥50 mm precipitation on a certain day (20 - 20 hours, Beijing time) as a rainstorm weather, and determine the cold air activity with daily minimum temperature dropped by more than 8°C for 24 hours, or decreased by 10°C for 48 h, and a daily minimum temperature of ≤4°C as a cold weather process. We statistically analyze the temporal and spatial characteristics and trends of high temperature, heavy rain and cold weather and the circulation background field. We count the number of extreme weather events such as persistent high temperatures, heavy rains and cold weather frosts in Shanxi, and analyze the temporal and spatial distribution characteristics, trends and general circulation background of extreme weather events. We analyze and find out the common features of the large-scale circulation background field in various extreme weather events. Through the study of the temporal and spatial distribution characteristics of extreme weather events in Shanxi, including persistent high temperature, heavy rain or sudden cold wave frost weather, we summarize the large-scale circulation characteristics of such extreme weather events. It will provide some reference for future related weather forecasting.

Statistical study on the spatial - temporal distribution features of the arctic sea ice extent

On the basis of the arctic monthly mean sea ice extent data set during 1953-1984, the arctic region is divided into eight subregions,and the analyses of empirical orthogonal functions, power spectrum and maximum entropy spectrum are made to indentify the major spatial and temporal features of the sea ice fluctuations within 32-year period. And then, a brief appropriate physical explanation is tentatively suggested. The results show that both seasonal and non-seasonal variations of the sea ice extent are remarkable, and iis mean annual peripheral positions as well as their interannu-al shifting amplitudes are quite different among all subregions. These features are primarily affected by solar radiation, o-cean circulation, sea surface temperature and maritime-continental contrast, while the non-seasonal variations are most possibly affected by the cosmic-geophysical factors such as earth pole shife, earth rotation oscillation and solar activity.

胶东型金矿

胶东是全球唯一已知赋存于前寒武纪变质地体中的晚中生代巨型金矿省,其成矿系统独具特色,具体表现为:(1)位于陆内复合构造域,经历了多期重大构造-热事件,大规模金成矿作用受控于120±2Ma古太平洋板块俯冲方向变化及其诱发的软流圈上涌、岩石圈改造和伸展-挤压变形交替及控矿断裂剪压-剪张转换;(2)多重控矿构造和多样赋矿建造联合控制了不同规模和类型金矿的发育,形成了三山岛、焦家、招平、栖霞、郭即和牟乳六条NE向金矿带和三山岛-栖霞EW向富金廊带,导致了金矿化类型(焦家式/破碎带蚀变岩型、玲珑式/石英脉型、蓬家夼式/蚀变砾岩±角砾岩型、辽上式/黄铁矿-碳酸盐脉型)及其地质-地球化学特征的多样性;(3)主要矿化元素Au、Ag、Cu、Pb和Zn均达到工业利用要求,并有多种共/伴生关键金属超常富集;(4)不同金矿带中硫化物Pb同位素组成与探明金资源储量及到郯庐断裂带的距离线性相关,反映距离幔源流体主通道越近、金属硫化物中放射性成因Pb含量和幔源组分占比越多、金成矿强度越大;(5)区域总体相对均一的Δ199 Hg(平均~0.012‰)及Δ199 Hg/Δ201 Hg与金品位呈线性负相关,表明成矿物质来源于富集岩石圈地幔、且地幔被俯冲洋壳及其上覆沉积物交代的程度控制了金品位的高低;(6)区域恒定的Δ33 S同位素组成(~0‰)排除了巨量金源自太古宙变质基底及其重熔花岗岩的可能,重的δ34 S(平均9.0±3.7‰)来源于俯冲的古太平洋板片及其上覆沉积物的脱挥发份;不同金矿带Δ33 S/δ34 S与探明金资源储量及其到郯庐断裂带距离线性相关,反映成矿期地壳伸展程度控制了金成矿强度;(7)区域He-Ar和H-O同位素组成显示壳幔混合特征,焦家式金矿的成矿流体组成更靠近地幔、玲珑式金矿位于地幔与地壳过渡带;三山岛、焦家和招平金矿带的成矿流体相对接近幔源流体,而郭即金矿带具有相对开放的构造环境;不同金矿带氢-氧同位素组成和探明金资源储量正相关,可能表征了从西到东成矿流体通量和流体-岩石反应强度逐渐降低。基于对上述特征的总结,提出了胶东型金矿的成因模式,明确了其成矿地球动力学背景和深部驱动、巨量金属和流体及络合物来源、输运通道和方式、源→汇过程和机制、成矿后变化和保存等成矿系统形成的关键因素,确立了“拆离断裂系与基底活化带及幔源流体通道复合控矿”的勘查思路和“四步式”的勘查模型。综上,胶东金矿不同尺度的鉴别特征及其形成的关键因素明显不同于全球已知的其他金矿类型,难以被已有成矿模式所涵盖,属于一种新的金矿类型——胶东型,其成因模式对华北、华南、西伯利亚、西澳伊尔岗、北美怀俄明和南美圭亚那等陆内金矿床具有普适性;系列找矿突破则验证了该成因模式与勘查模型的合理性和适用性。因此,本文认为胶东型金矿是全球研究热点和重要的金矿勘查方向,而该地区找矿的主攻目标是资源量大且品位和产状稳定的破碎带蚀变岩型金矿。

基于CNN-LSTM的水泥熟料f-CaO预测模型

水泥熟料中游离氧化钙(f-CaO)含量的传统人工离线检测缺乏时效性,不利于生产指导。针对离线检测的滞后问题和软测量模型中f-CaO含量与辅助变量的时序匹配问题,提出了一种基于卷积神经网络(convolutional neural network,CNN)和长短时记忆(long short-term memory,LSTM)神经网络的f-CaO含量预测模型。首先,利用滑动窗口截取辅助变量的区间数据;然后,采用CNN提取区间数据的时序特征;之后,构建LSTM神经网络模型;最后,控制截取辅助变量的延迟时间和间隔时间,根据模型预测拟合度提取辅助变量的最优时序特征。仿真结果表明,所提模型提高了水泥熟料中f-CaO含量的预测精度。

基于混合特征提取的流数据概念漂移处理方法

大数据时代,越来越多的数据以数据流的形式产生,由于其具有快速、无限、不稳定及动态变化等特性,使得概念漂移成为流数据挖掘中一个重要但困难的问题.目前多数概念漂移处理方法存在信息提取能力有限且未充分考虑流数据的时序特性等问题.针对这些问题,提出一种基于混合特征提取的流数据概念漂移处理方法(concept drift processing method of streaming data based on mixed feature extraction,MFECD).该方法首先采用不同尺度的卷积核对数据进行建模以构建拼接特征,采用门控机制将浅层输入和拼接特征融合,作为不同网络层次输入进行自适应集成,以获得能够兼顾细节信息和语义信息的数据特性.在此基础上,采用注意力机制和相似度计算评估流数据不同时刻的重要性,以增强数据流关键位点的时序特性.实验结果表明,该方法能有效提取流数据中包含的复杂数据特征和时序特征,提高了数据流中概念漂移的处理能力.

沿江城市降雨特性及雨洪关系分析——以四川泸州市为例

沿江城市气候多变,外洪和内涝之间关系复杂,相互影响、相互制约,洪涝统筹是沿江城市面临的最主要问题。分析掌握沿江城市的暴雨、洪水特征,可提前预测不同降雨条件下河道洪水的特性及其对城区排涝的影响,对于城市防洪系统和排涝系统统筹兼顾、合理规划、洪涝预报预警,都具有重要意义。以四川泸州市为例,对暴雨时空分布特征、洪水特征规律等进行了分析探讨,以期为新形势下沿江城市防洪减灾体系建设提供参考。

基于注意力特征融合时空图网络的超短期风电功率预测

为提高风电功率的预测精度,综合考虑时间和空间多维度因素的影响,提出了一种基于注意力机制和多阶段特征融合的时空图神经网络(Spatio-temporal graph neural network with attention mechanism and multistage feature fusion,AMF-STGNN)的超短期风电功率预测方法。首先基于Pearson相关系数法对数据特征进行降维,确定影响风电功率的关键因素。然后构建AMF-STGNN预测模型。该模型主要由时空关联网络构建模块和多维度时空特征抽取模块组成。通过时空关联网络构建模块构建时空图,以揭示风电气象因素的空间连接关系。通过多维度时空特征抽取模块应用时间卷积和图卷积挖掘数据的时空特征,并利用注意力机制学习重要特征。此外,该方法还引入残差结构和多阶段特征融合结构提高模型的表达能力。最后以宁夏3个风电场真实数据为例,验证了所提方法在提升风电功率预测精度方面的有效性。

基于流形学习算法的降雨数据时空分布特征提取及重构

【目的】掌握精细化的降雨时空分布特征,对于城市洪涝风险管理水平的提高具有重要的意义。我国近十几年降雨监测站网密集且数据精细程度高,但时间序列较短;历史降雨资料时间序列长,但是精细程度低。【方法】为了更有效地利用历史降雨资料,将流形学习算法引入到历史降雨资料重构中,从高分辨率降雨资料中,提取降雨的时空分别特征,基于该特征,将历史逐6 h的降雨空间数据重构为逐1 h的降雨数据,以满足城市洪涝风险分析的要求。【结果】结果表明,该方法重构数据高值区与实测值的平均误差在15%以内,低值区在20%以内,比传统插值处理的数据高值区误差降低了45%~85%,低值区降低了10%~40%。【结论】利用流形学习算法重构的历史空间降雨数据符合各地区降雨时空分布特征,可提高降雨空间数据颗粒度,实现降雨时空分布精细化特征的有效、合理的提取和总结。

基于时序图卷积的动态网络链路预测

动态网络链路预测广泛的应用前景,使得其逐渐成为网络科学研究的热点.动态网络链路演化过程中具有复杂的空间相关性和时间依赖性,导致其链路预测任务极具挑战.提出一个基于时序图卷积的动态网络链路预测模型(dynamic network link prediction based on sequential graph convolution, DNLP-SGC).针对网络快照序列不能有效反映动态网络连续性的问题,采用边缘触发机制对原始网络权重矩阵进行修正,弥补了离散快照表示动态网络存在时序信息丢失的不足.从网络演化过程出发,综合考虑节点间的特征相似性以及历史交互信息,采用时序图卷积提取动态网络中节点的特征,该方法融合了节点时空依赖关系.进一步,采用因果卷积网络捕获网络演化过程中潜在的全局时序特征,实现动态网络链路预测.在2个真实的网络数据集上的实验结果表明,DNLP-SGC在precision, recall, AUC指标上均优于对比的基线模型.

基于数字孪生的航班延误时间预测方法

为了更加准确和高效地预测大面积航班延误时间,提出了基于数字孪生的航班延误时间预测方法。首先,从航班链整体的角度出发,依据航班运行业务特点和数字孪生技术特征设计了航班链数字孪生系统框架,综合航班链全生命周期内相关航班和机场的运行状态特征;其次,基于Fastformer和GraphSAGE模型设计了航班链时空特征提取模型(ST-Former),充分挖掘航班之间的时空关联特征。实验表明,该方法预测效率和准确度显著提升,平均预测误差在3 min左右。

时空多尺度关联特征融合的二维卷积网络细粒度动作识别模型

针对传统二维(2D)卷积网络提取时空特征尺度单一以及对细粒度动作数据集中帧与帧之间的远程时间关联信息利用不足的问题,本文提出时空多尺度关联特征融合的2D卷积网络细粒度动作识别模型。首先,为建模视频多尺度空间关联以加强对细粒度视频数据的空间表征能力,模型使用多尺度“特征压缩、特征激发”方式,使网络所提取空间特征更加丰富有效。然后,为充分利用细粒度视频数据时间维度上的运动信息,本文引入时间窗口自注意力机制,利用自注意力机制强大的远程依赖建模能力同时只在时间维度上进行自注意力操作,以较低计算成本建模远程时间依赖关系。最后,考虑到所提取时空特征对不同类型动作分类的贡献不均等,本文引入自适应特征融合模块,为特征动态赋予不同权重实现自适应特征融合。模型在2个细粒度动作识别数据集Diving48和Something-somethingV1上识别准确率分别达到86.0%和46.9%,分别使原始主干网络识别准确率提升3.8%和1.3%。实验结果表明,在只使用视频帧信息作为输入的情况下,本模型达到与现有基于Transformer和三维卷积神经网络(3D CNN)算法相当的识别准确率。

基于多尺度融合和时空特征的网络入侵检测模型

针对入侵检测模型提取特征能力不足,且流量数据中含冗余噪声的问题,提出一种基于多尺度融合和时空特征的ML-PFN入侵检测模型。采用多尺度特征融合技术分别提取数据中浅层特征信息和深层特征信息,使模型学习的特征更加丰富;采用软阈值函数和注意力机制自动选择合适的阈值,减少噪声及不相关信息对模型的干扰;融合时空特征构成多尺度空间特征提取长短时记忆-并行特征网络(MSFE LSTM-parallel feature network, ML-PFN)模型,并应用于网络入侵检测。通过3个公开数据集进行性能评估,实验结果表明,ML-PFN模型对比其它5种分类模型各项指标效果最好,在训练时长适中的同时准确率达到96.45%。

基于双流自适应时空增强图卷积网络的手语识别

针对提取手语特征过程中出现的信息表征能力差、信息不完整问题,设计了一种双流自适应时空增强图卷积网络(two-stream adaptive enhanced spatial temporal graph convolutional network,TAEST-GCN)实现基于孤立词的手语识别。该网络使用人体身体、手部和面部节点作为输入,构造基于人体关节和骨骼的双流结构。通过自适应时空图卷积模块生成不同部位之间的连接,并充分利用其中的位置和方向信息。同时采用残差连接方式设计自适应多尺度时空注意力模块,进一步增强该网络在空域和时域的卷积能力。将双流网络提取到的有效特征进行加权融合,可以分类输出手语词汇。最后在公开的中文手语孤立词数据集上进行实验,在100类词汇和500类词汇分类任务中准确率达到了95.57%和89.62%。

基于激光扫描的电子档案异构时态数据检测方法

研究一种基于激光扫描的电子档案异构时态数据检测方法,该方法利用激光扫描获取数据并生成电子档案。针对电子档案中的数据实施缺失数据填补和离群数据处理。提取电子档案数据的递归率、仙农熵以及歪度三个特征。结合随机森林算法,输入递归率、仙农熵以及歪度三个特征,得出电子档案异构时态数据类型。结果表明:1处和3处边坡工程的位移处于轻微状态;2处边坡工程的位移处于严重状态;4处边坡工程的位移处于安全状态;5处边坡工程的位移处于较严重状态。所研究方法的Kappa系数达到相对最大值,说明所研究方法的检测能力更高。

基于改进时间卷积网络的微电网超短期负荷预测

为了提高微电网中用电负荷超短期预测的准确性,对时间卷积网络进行特征增强和注意力增强改进,将时间卷积网络中的一维因果膨胀卷积替换为二维卷积,同时利用时间模式注意力机制对时间卷积网络的隐藏层加权处理,提取负荷的多维特征,挖掘序列中存在的潜藏联系。根据改进的方法建立预测模型并进行对比实验以验证方法的有效性,能够对用电负荷的不确定性进行有效的处理,拓宽特征向量的维度,有效捕捉负荷序列中与时间有关的特征,提高用电负荷的预测精度。