Deepfake Video Detection Based on Improved CapsNet and Temporal–Spatial Features

Rapid development of deepfake technology led to the spread of forged audios and videos across network platforms,presenting risks for numerous countries,societies,and individuals,and posing a serious threat to cyberspace security.To address the problem of insufficient extraction of spatial features and the fact that temporal features are not considered in the deepfake video detection,we propose a detection method based on improved CapsNet and temporal–spatial features(iCapsNet–TSF).First,the dynamic routing algorithm of CapsNet is improved using weight initialization and updating.Then,the optical flow algorithm is used to extract interframe temporal features of the videos to form a dataset of temporal–spatial features.Finally,the iCapsNet model is employed to fully learn the temporal–spatial features of facial videos,and the results are fused.Experimental results show that the detection accuracy of iCapsNet–TSF reaches 94.07%,98.83%,and 98.50%on the Celeb-DF,FaceSwap,and Deepfakes datasets,respectively,displaying a better performance than most existing mainstream algorithms.The iCapsNet–TSF method combines the capsule network and the optical flow algorithm,providing a novel strategy for the deepfake detection,which is of great significance to the prevention of deepfake attacks and the preservation of cyberspace security.

Boosting Multi-modal Ocular Recognition via Spatial Feature Reconstruction and Unsupervised Image Quality Estimation

In the daily application of an iris-recognition-at-a-distance(IAAD)system,many ocular images of low quality are acquired.As the iris part of these images is often not qualified for the recognition requirements,the more accessible periocular regions are a good complement for recognition.To further boost the performance of IAAD systems,a novel end-to-end framework for multi-modal ocular recognition is proposed.The proposed framework mainly consists of iris/periocular feature extraction and matching,unsupervised iris quality assessment,and a score-level adaptive weighted fusion strategy.First,ocular feature reconstruction(OFR)is proposed to sparsely reconstruct each probe image by high-quality gallery images based on proper feature maps.Next,a brand new unsupervised iris quality assessment method based on random multiscale embedding robustness is proposed.Different from the existing iris quality assess-ment methods,the quality of an iris image is measured by its robustness in the embedding space.At last,the fusion strategy exploits the iris quality score as the fusion weight to coalesce the complementary information from the iris and periocular regions.Extensive experi-mental results on ocular datasets prove that the proposed method is obviously better than unimodal biometrics,and the fusion strategy can significantly improve therecognition performance.

Accurate machine learning models based on small dataset of energetic materials through spatial matrix featurization methods

A large database is desired for machine learning(ML) technology to make accurate predictions of materials physicochemical properties based on their molecular structure.When a large database is not available,the development of proper featurization method based on physicochemical nature of target proprieties can improve the predictive power of ML models with a smaller database.In this work,we show that two new featurization methods,volume occupation spatial matrix and heat contribution spatial matrix,can improve the accuracy in predicting energetic materials' crystal density(ρ_(crystal)) and solid phase enthalpy of formation(H_(f,solid)) using a database containing 451 energetic molecules.Their mean absolute errors are reduced from 0.048 g/cm~3 and 24.67 kcal/mol to 0.035 g/cm~3 and 9.66 kcal/mol,respectively.By leave-one-out-cross-validation,the newly developed ML models can be used to determine the performance of most kinds of energetic materials except cubanes.Our ML models are applied to predict ρ_(crystal) and H_(f,solid) of CHON-based molecules of the 150 million sized PubChem database,and screened out 56 candidates with competitive detonation performance and reasonable chemical structures.With further improvement in future,spatial matrices have the potential of becoming multifunctional ML simulation tools that could provide even better predictions in wider fields of materials science.

Sanxingdui Cultural Relics Recognition Algorithm Based on Hyperspectral Multi-Network Fusion

Sanxingdui cultural relics are the precious cultural heritage of humanity with high values of history,science,culture,art and research.However,mainstream analytical methods are contacting and detrimental,which is unfavorable to the protection of cultural relics.This paper improves the accuracy of the extraction,location,and analysis of artifacts using hyperspectral methods.To improve the accuracy of cultural relic mining,positioning,and analysis,the segmentation algorithm of Sanxingdui cultural relics based on the spatial spectrum integrated network is proposed with the support of hyperspectral techniques.Firstly,region stitching algorithm based on the relative position of hyper spectrally collected data is proposed to improve stitching efficiency.Secondly,given the prominence of traditional HRNet(High-Resolution Net)models in high-resolution data processing,the spatial attention mechanism is put forward to obtain spatial dimension information.Thirdly,in view of the prominence of 3D networks in spectral information acquisition,the pyramid 3D residual network model is proposed to obtain internal spectral dimensional information.Fourthly,four kinds of fusion methods at the level of data and decision are presented to achieve cultural relic labeling.As shown by the experiment results,the proposed network adopts an integrated method of data-level and decision-level,which achieves the optimal average accuracy of identification 0.84,realizes shallow coverage of cultural relics labeling,and effectively supports the mining and protection of cultural relics.

Geographic context-aware text mining:enhance social media message classification for situational awareness by integrating spatial and temporal features

To find disaster relevant social media messages,current approaches utilize natural language processing methods or machine learning algorithms relying on text only,which have not been perfected due to the variability and uncertainty in the language used on social media and ignoring the geographic context of the messages when posted.Meanwhile,a disaster relevant social media message is highly sensitive to its posting location and time.However,limited studies exist to explore what spatial features and the extent of how temporal,and especially spatial features can aid text classification.This paper proposes a geographic context-aware text mining method to incorporate spatial and temporal information derived from social media and authoritative datasets,along with the text information,for classifying disaster relevant social media posts.This work designed and demonstrated how diverse types of spatial and temporal features can be derived from spatial data,and then used to enhance text mining.The deep learning-based method and commonly used machine learning algorithms,assessed the accuracy of the enhanced text-mining method.The performance results of different classification models generated by various combinations of textual,spatial,and temporal features indicate that additional spatial and temporal features help improve the overall accuracy of the classification.

Soil geochemical prospecting prediction method based on deep convolutional neural networks-Taking Daqiao Gold Deposit in Gansu Province, China as an example

A method is proposed for the prospecting prediction of subsurface mineral deposits based on soil geochemistry data and a deep convolutional neural network model.This method uses three techniques(window offset,scaling,and rotation)to enhance the number of training data for the model.A window area is used to extract the spatial distribution characteristics of soil geochemistry and measure their correspondence with the occurrence of known subsurface deposits.Prospecting prediction is achieved by matching the characteristics of the window area of an unknown area with the relationships established in the known area.This method can efficiently predict mineral prospective areas where there are few ore deposits used for generating the training dataset,meaning that the deep-learning method can be effectively used for deposit prospecting prediction.Using soil active geochemical measurement data,this method was applied in the Daqiao area,Gansu Province,for which seven favorable gold prospecting target areas were predicted.The Daqiao orogenic gold deposit of latest Jurassic and Early Jurassic age in the southern domain has more than 105 t of gold resources at an average grade of 3-4 g/t.In 2020,the project team drilled and verified the K prediction area,and found 66 m gold mineralized bodies.The new method should be applicable to prospecting prediction using conventional geochemical data in other areas.

Microphone Array-Based Sound Source Localization Using Convolutional Residual Network

Microphone array-based sound source localization(SSL)is widely used in a variety of occasions such as video conferencing,robotic hearing,speech enhancement,speech recognition and so on.The traditional SSL methods cannot achieve satisfactory performance in adverse noisy and reverberant environments.In order to improve localization performance,a novel SSL algorithm using convolutional residual network(CRN)is proposed in this paper.The spatial features including time difference of arrivals(TDOAs)between microphone pairs and steered response power-phase transform(SRPPHAT)spatial spectrum are extracted in each Gammatone sub-band.The spatial features of different sub-bands with a frame are combine into a feature matrix as the input of CRN.The proposed algorithm employ CRN to fuse the spatial features.Since the CRN introduces the residual structure on the basis of the convolutional network,it reduce the difficulty of training procedure and accelerate the convergence of the model.A CRN model is learned from the training data in various reverberation and noise environments to establish the mapping regularity between the input feature and the sound azimuth.Through simulation verification,compared with the methods using traditional deep neural network,the proposed algorithm can achieve a better localization performance in SSL task,and provide better generalization capacity to untrained noise and reverberation.

Content-based retrieval based on binary vectors for 2-D medical images

In medical research and clinical diagnosis, automated or computer-assisted classification and retrieval methods are highly desirable to offset the high cost of manual classification and manipulation by medical experts. To facilitate the decision-making in the health-care and the related areas, in this paper, a two-step content-based medical image retrieval algorithm is proposed. Firstly, in the preprocessing step, the image segmentation is performed to distinguish image objects, and on the basis of the ...

Soft Computing Based Discriminator Model for Glaucoma Diagnosis

In this study, a Discriminator Model for Glaucoma Diagnosis (DMGD)using soft computing techniques is presented. As the biomedical images such asfundus images are often acquired in high resolution, the Region of Interest (ROI)for glaucoma diagnosis must be selected at first to reduce the complexity of anysystem. The DMGD system uses a series of pre-processing;initial cropping by thegreen channel’s intensity, Spatially Weighted Fuzzy C Means (SWFCM), bloodvessel detection and removal by Gaussian Derivative Filters (GDF) and inpaintingalgorithms. Once the ROI has been selected, the numerical features such as colour, spatial domain features from Local Binary Pattern (LBP) and frequencydomain features from LAWS are generated from the corresponding ROI forfurther classification using kernel based Support Vector Machine (SVM). TheDMGD system performances are validated using four fundus image databases;ORIGA, RIM-ONE, DRISHTI-GS1, and HRF with four different kernels;LinearKernel (LK), Polynomial Kernel (PK), Radial Basis Function (RBFK) kernel,Quadratic Kernel (QK) based SVM classifiers. Results show that the DMGD system classifies the fundus images accurately using the multiple features and kernelbased classifies from the properly segmented ROI.

Deep learning of DEM image texture for landform classification in the Shandong area,China

Landforms are an important element of natural geographical environment,and textures are the research basis for the spatial differentiation,evolution features,and analysis rules of the landform.Using the regional difference of texture to describe the spatial distribution pattern of macro landform features is helpful to the landform classification and identification.Digital elevation model(DEM)image texture,which gives full expression to texture difference,is key data source to reflect the surface features and landform classification.Following the texture analysis,landform features analysis is assistant to different landforms classification,even in landform boundary.With the increasing accuracy requirement of landform information acquisition in geomorphic thematic mapping,hierarchical landform classification has become the focus and difficulty in research.Recently,the pattern recognition represented by Convolutional Neural Network has made great achievements in landform research,whose multichannel feature fusion structure satisfies the network structure of different landform classification.In this paper,DEM image texture was taken as the data source,and gray level co-occurrence matrix was applied to extract texture measures.Owing to the similarity of similar landform and the difference of different landform in a certain scale,a comprehensive texture factor reflecting landform features was proposed,and the spatial distribution pattern of landform features was systematically analyzed.On this basis,the coupling relationship between texture and landform type was explored.Thus,the deep learning method of Convolutional Neural Network is used to train the texture features,and the second-class landform classification is carried out through softmax.The classification results in small relief and mid-relief low mountains,overall accuracy are 84.35%and 69.95%respectively,while kappa coefficient are 0.72 and 0.40 respectively,were compared to that of traditional unsupervised landform classification results,and the superiority of Convolutional Neural Network classification was verified,it approximately improved 6%in overall accuracy and 0.4 in kappa coefficient.

Facial expression recognition based on bidirectional gated recurrent units within deep residual network

Purpose-recent years,facial expression recognition has been widely used in human machine interaction,clinical medicine and safe driving.However,there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approach-To solve such limitation,this paper proposes a novel model based on bidirectional gated recurrent unit networks(Bi-GRUs)with two-way propagations,and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network.Since the Inception-V3 network model for spatial feature extraction has too many parameters,it is prone to overfitting during training.This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters,so as to obtain an Inception-W network with better generalization.Findings-Finally,the proposed model is pretrained to determine the best settings and selections.Then,the pretrained model is experimented on two facial expression data sets of CKþand Oulu-CASIA,and the recognition performance and efficiency are compared with the existing methods.The highest recognition rate is 99.6%,which shows that the method has good recognition accuracy in a certain range.Originality/value-By using the proposed model for the applications of facial expression,the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

Unsupervised GRNN flood mapping approach combined with uncertainty analysis using bi-temporal Sentinel-2 MSI imageries

Floods occur frequently worldwide.The timely,accurate mapping of the flooded areas is an important task.Therefore,an unsupervised approach is proposed for automated flooded area mapping from bitemporal Sentinel-2 multispectral images in this paper.First,spatial–spectral features of the images before and after the flood are extracted to construct the change magnitude image(CMI).Then,the certain flood pixels and non-flood pixels are obtained by performing uncertainty analysis on the CMI,which are considered reliable classification samples.Next,Generalized Regression Neural Network(GRNN)is used as the core classifier to generate the initial flood map.Finally,an easy-toimplement two-stage post-processing is proposed to reduce the mapping error of the initial flood map,and generate the final flood map.Different from other methods based on machine learning,GRNN is used as the classifier,but the proposed approach is automated and unsupervised because it uses samples automatically generated in uncertainty analysis for model training.Results of comparative experiments in the three sub-regions of the Poyang Lake Basin demonstrate the effectiveness and superiority of the proposed approach.Moreover,its superiority in dealing with uncertain pixels is further proven by comparing the classification accuracy of different methods on uncertain pixels.