The facial landmarks can provide valuable information for expression-related tasks.However,most approaches only use landmarks for segmentation preprocessing or directly input them into the neural network for fully con...The facial landmarks can provide valuable information for expression-related tasks.However,most approaches only use landmarks for segmentation preprocessing or directly input them into the neural network for fully connection.Such simple combination not only fails to pass the spatial information to network,but also increases calculation amounts.The method proposed in this paper aims to integrate facial landmarks-driven representation into the triplet network.The spatial information provided by landmarks is introduced into the feature extraction process,so that the model can better capture the location relationship.In addition,coordinate information is also integrated into the triple loss calculation to further enhance similarity prediction.Specifically,for each image,the coordinates of 68 landmarks are detected,and then a region attention map based on these landmarks is generated.For the feature map output by the shallow convolutional layer,it will be multiplied with the attention map to correct the feature activation,so as to strengthen the key region and weaken the unimportant region.Finally,the optimized embedding output can be further used for downstream tasks.Three embeddings of three images output by the network can be regarded as a triplet representation for similarity computation.Through the CK+dataset,the effectiveness of such an optimized feature extraction is verified.After that,it is applied to facial expression similarity tasks.The results on the facial expression comparison(FEC)dataset show that the accuracy rate will be significantly improved after the landmark information is introduced.展开更多
The motivation for this study is that the quality of deep fakes is constantly improving,which leads to the need to develop new methods for their detection.The proposed Customized Convolutional Neural Network method in...The motivation for this study is that the quality of deep fakes is constantly improving,which leads to the need to develop new methods for their detection.The proposed Customized Convolutional Neural Network method involves extracting structured data from video frames using facial landmark detection,which is then used as input to the CNN.The customized Convolutional Neural Network method is the date augmented-based CNN model to generate‘fake data’or‘fake images’.This study was carried out using Python and its libraries.We used 242 films from the dataset gathered by the Deep Fake Detection Challenge,of which 199 were made up and the remaining 53 were real.Ten seconds were allotted for each video.There were 318 videos used in all,199 of which were fake and 119 of which were real.Our proposedmethod achieved a testing accuracy of 91.47%,loss of 0.342,and AUC score of 0.92,outperforming two alternative approaches,CNN and MLP-CNN.Furthermore,our method succeeded in greater accuracy than contemporary models such as XceptionNet,Meso-4,EfficientNet-BO,MesoInception-4,VGG-16,and DST-Net.The novelty of this investigation is the development of a new Convolutional Neural Network(CNN)learning model that can accurately detect deep fake face photos.展开更多
In order to study the relationship between landmarks and spatial memory in short-nosed fruit bat, Cynopterus sphinx （Megachiroptera, Pteropodidae）, we simulated a foraging environment in the laboratory. Different la...In order to study the relationship between landmarks and spatial memory in short-nosed fruit bat, Cynopterus sphinx （Megachiroptera, Pteropodidae）, we simulated a foraging environment in the laboratory. Different landmarks were placed to gauge the spatial memory of C. sphinx. We changed the number of landmarks every day with 0 landmarks again on the fifth day （from 0, 2, 4, 8 to 0）. Individuals from the control group were exposed to the identical artificial foraging environment, but without landmarks. The results indicated that there was significant correlation between the time of the first foraging and the experimental days in both groups （Pearson Correlation： experimental group： r=-0.593, P〈0.01; control group： r=-0.581, P〈0.01）. There was no significant correlation between the success rates of foraging and the experimental days in experimental groups （Pearson Correlation： r=0.177, P〉0.05）, but there was significant correlation between the success rates of foraging and the experimental days in the control groups （Pearson Correlation： r=0.445, P〈0.05）. There was no significant difference for the first foraging time between experimental and control groups （GLM： F0.05,1=4.703, P〉0.05）; also, there was no significant difference in success rates of foraging between these two groups （GLM： F0.05,1=0.849,P〉0.05）. The results of our experiment suggest that spatial memory in C. sphinx was formed gradually and that the placed landmarks appeared to have no discernable effects on the memory of the foraging space.展开更多
文摘The facial landmarks can provide valuable information for expression-related tasks.However,most approaches only use landmarks for segmentation preprocessing or directly input them into the neural network for fully connection.Such simple combination not only fails to pass the spatial information to network,but also increases calculation amounts.The method proposed in this paper aims to integrate facial landmarks-driven representation into the triplet network.The spatial information provided by landmarks is introduced into the feature extraction process,so that the model can better capture the location relationship.In addition,coordinate information is also integrated into the triple loss calculation to further enhance similarity prediction.Specifically,for each image,the coordinates of 68 landmarks are detected,and then a region attention map based on these landmarks is generated.For the feature map output by the shallow convolutional layer,it will be multiplied with the attention map to correct the feature activation,so as to strengthen the key region and weaken the unimportant region.Finally,the optimized embedding output can be further used for downstream tasks.Three embeddings of three images output by the network can be regarded as a triplet representation for similarity computation.Through the CK+dataset,the effectiveness of such an optimized feature extraction is verified.After that,it is applied to facial expression similarity tasks.The results on the facial expression comparison(FEC)dataset show that the accuracy rate will be significantly improved after the landmark information is introduced.
基金Science and Technology Funds from the Liaoning Education Department(Serial Number:LJKZ0104).
文摘The motivation for this study is that the quality of deep fakes is constantly improving,which leads to the need to develop new methods for their detection.The proposed Customized Convolutional Neural Network method involves extracting structured data from video frames using facial landmark detection,which is then used as input to the CNN.The customized Convolutional Neural Network method is the date augmented-based CNN model to generate‘fake data’or‘fake images’.This study was carried out using Python and its libraries.We used 242 films from the dataset gathered by the Deep Fake Detection Challenge,of which 199 were made up and the remaining 53 were real.Ten seconds were allotted for each video.There were 318 videos used in all,199 of which were fake and 119 of which were real.Our proposedmethod achieved a testing accuracy of 91.47%,loss of 0.342,and AUC score of 0.92,outperforming two alternative approaches,CNN and MLP-CNN.Furthermore,our method succeeded in greater accuracy than contemporary models such as XceptionNet,Meso-4,EfficientNet-BO,MesoInception-4,VGG-16,and DST-Net.The novelty of this investigation is the development of a new Convolutional Neural Network(CNN)learning model that can accurately detect deep fake face photos.
基金supported by the National Natural Science Foundation of China(NSFC,No30800102)Natural Science Foundation of Hainan Province(309026)
文摘In order to study the relationship between landmarks and spatial memory in short-nosed fruit bat, Cynopterus sphinx （Megachiroptera, Pteropodidae）, we simulated a foraging environment in the laboratory. Different landmarks were placed to gauge the spatial memory of C. sphinx. We changed the number of landmarks every day with 0 landmarks again on the fifth day （from 0, 2, 4, 8 to 0）. Individuals from the control group were exposed to the identical artificial foraging environment, but without landmarks. The results indicated that there was significant correlation between the time of the first foraging and the experimental days in both groups （Pearson Correlation： experimental group： r=-0.593, P〈0.01; control group： r=-0.581, P〈0.01）. There was no significant correlation between the success rates of foraging and the experimental days in experimental groups （Pearson Correlation： r=0.177, P〉0.05）, but there was significant correlation between the success rates of foraging and the experimental days in the control groups （Pearson Correlation： r=0.445, P〈0.05）. There was no significant difference for the first foraging time between experimental and control groups （GLM： F0.05,1=4.703, P〉0.05）; also, there was no significant difference in success rates of foraging between these two groups （GLM： F0.05,1=0.849,P〉0.05）. The results of our experiment suggest that spatial memory in C. sphinx was formed gradually and that the placed landmarks appeared to have no discernable effects on the memory of the foraging space.
