Re-Distributing Facial Features for Engagement Prediction with ModernTCN

Automatically detecting learners’engagement levels helps to develop more effective online teaching and assessment programs,allowing teachers to provide timely feedback and make personalized adjustments based on students’needs to enhance teaching effectiveness.Traditional approaches mainly rely on single-frame multimodal facial spatial information,neglecting temporal emotional and behavioural features,with accuracy affected by significant pose variations.Additionally,convolutional padding can erode feature maps,affecting feature extraction’s representational capacity.To address these issues,we propose a hybrid neural network architecture,the redistributing facial features and temporal convolutional network(RefEIP).This network consists of three key components:first,utilizing the spatial attention mechanism large kernel attention(LKA)to automatically capture local patches and mitigate the effects of pose variations;second,employing the feature organization and weight distribution(FOWD)module to redistribute feature weights and eliminate the impact of white features and enhancing representation in facial feature maps.Finally,we analyse the temporal changes in video frames through the modern temporal convolutional network(ModernTCN)module to detect engagement levels.We constructed a near-infrared engagement video dataset(NEVD)to better validate the efficiency of the RefEIP network.Through extensive experiments and in-depth studies,we evaluated these methods on the NEVD and the Database for Affect in Situations of Elicitation(DAiSEE),achieving an accuracy of 90.8%on NEVD and 61.2%on DAiSEE in the fourclass classification task,indicating significant advantages in addressing engagement video analysis problems.

Deepfake Video Detection Employing Human Facial Features

Deepfake technology can be used to replace people’s faces in videos or pictures to show them saying or doing things they never said or did. Deepfake media are often used to extort, defame, and manipulate public opinion. However, despite deepfake technology’s risks, current deepfake detection methods lack generalization and are inconsistent when applied to unknown videos, i.e., videos on which they have not been trained. The purpose of this study is to develop a generalizable deepfake detection model by training convoluted neural networks (CNNs) to classify human facial features in videos. The study formulated the research questions: “How effectively does the developed model provide reliable generalizations?” A CNN model was trained to distinguish between real and fake videos using the facial features of human subjects in videos. The model was trained, validated, and tested using the FaceForensiq++ dataset, which contains more than 500,000 frames and subsets of the DFDC dataset, totaling more than 22,000 videos. The study demonstrated high generalizability, as the accuracy of the unknown dataset was only marginally (about 1%) lower than that of the known dataset. The findings of this study indicate that detection systems can be more generalizable, lighter, and faster by focusing on just a small region (the human face) of an entire video.

Facial Features of an Air Gun Array Wavelet in the Time-Frequency Domain Based on Marine Vertical Cables

Air gun arrays are often used in marine energy exploration and marine geological surveys.The study of the single bubble dynamics and multibubbles produced by air guns interacting with each other is helpful in understanding pressure signals.We used the van der Waals air gun model to simulate the wavelets of a sleeve gun of various offsets and arrival angles.Several factors were taken into account,such as heat transfer,the thermodynamically open quasi-static system,the vertical rise of the bubble,and air gun post throttling.Marine vertical cables are located on the seafloor,but hydrophones are located in seawater and are far away from the air gun array vertically.This situation conforms to the acquisition conditions of the air gun far-field wavelet and thus avoids the problems of ship noise,ocean surges,and coupling.High-quality 3D wavelet data of air gun arrays were collected during a vertical cable test in the South China Sea in 2017.We proposed an evaluation method of multidimensional facial features,including zeropeak amplitude,peak-peak amplitude,bubble period,primary-to-bubble ratio,frequency spectrum,instantaneous amplitude,instantaneous phase,and instantaneous frequency,to characterize the 3D air gun wave field.The match between the facial features in the field and simulated data provides confidence for the use of the van der Waals air gun model to predict air gun wavelet and facial features to evaluate air gun array.

Tracking facial features with occlusions

Facial expression recognition consists of determining what kind of emotional content is presented in a human face. The problem presents a complex area for exploration, since it encompasses face acquisition, facial feature tracking, facial ex- pression classification. Facial feature tracking is of the most interest. Active Appearance Model (AAM) enables accurate tracking of facial features in real-time, but lacks occlusions and self-occlusions. In this paper we propose a solution to improve the accuracy of fitting technique. The idea is to include occluded images into AAM training data. We demonstrate the results by running ex- periments using gradient descent algorithm for fitting the AAM. Our experiments show that using fitting algorithm with occluded training data improves the fitting quality of the algorithm.

Novel Facial Features Segmentation Algorithm

An efficient algorithm for facial features extractions is proposed. The facial features we segment are the two eyes, nose and mouth. The algorithm is based on an improved Gabor wavelets edge detector, morphological approach to detect the face region and facial features regions, and an improved T-shape face mask to locate the extract location of facial features. The experimental results show that the proposed method is robust against facial expression, illumination, and can be also effective if the person wearing glasses, and so on.

Automatic Location of Main Facial Features in Front-View Images

This paper presents a set of algorithms capable of locating main facial features automatically and effectively. Based on integral projection of local binary image pixels and pixel clustering techniques, a set of a priori knowledge based algorithms have succeeded in locating eyes, nose and mouth, and uprighting the tilt face. The proposed approach is superior to other methods as it takes account of photos with glasses and sha dows, therefore suitable for processing real ID type photos.

Quantification of Cranial Asymmetry in Infants by Facial Feature Extraction

In this paper, a facial feature extracting method is proposed to transform three-dimension （3D） head images of infants with deformational plagiocephaly for assessment of asymmetry. The features of 3D point clouds of an infant＇s cranium can be identified by local feature analysis and a two-phase k-means classification algorithm. The 3D images of infants with asymmetric cranium can then be aligned to the same pose. The mirrored head model obtained from the symmetry plane is compared with the original model for the measurement of asymmetry. Numerical data of the cranial volume can be reviewed by a pediatrician to adjust the treatment plan. The system can also be used to demonstrate the treatment progress.

Active Shape Model of Combining Pca and Ica: Application to Facial Feature Extraction

Active Shape Model （ASM） is a powerful statistical tool to extract the facial features of a face image under frontal view. It mainly relies on Principle Component Analysis （PCA） to statistically model the variability in the training set of example shapes. Independent Component Analysis （ICA） has been proven to be more efficient to extract face features than PCA. In this paper, we combine the PCA and ICA by the consecutive strategy to form a novel ASM. Firstly, an initial model, which shows the global shape variability in the training set, is generated by the PCA-based ASM. And then, the final shape model, which contains more local characters, is established by the ICA-based ASM. Experimental results verify that the accuracy of facial feature extraction is statistically significantly improved by applying the ICA modes after the PCA modes.

Constitution identification model in traditional Chinese medicine based on multiple features

Objective To construct a precise model for identifying traditional Chinese medicine(TCM)constitutions;thereby offering optimized guidance for clinical diagnosis and treatment plan-ning;and ultimately enhancing medical efficiency and treatment outcomes.Methods First;TCM full-body inspection data acquisition equipment was employed to col-lect full-body standing images of healthy people;from which the constitutions were labelled and defined in accordance with the Constitution in Chinese Medicine Questionnaire(CCMQ);and a dataset encompassing labelled constitutions was constructed.Second;heat-suppres-sion valve(HSV)color space and improved local binary patterns(LBP)algorithm were lever-aged for the extraction of features such as facial complexion and body shape.In addition;a dual-branch deep network was employed to collect deep features from the full-body standing images.Last;the random forest(RF)algorithm was utilized to learn the extracted multifea-tures;which were subsequently employed to establish a TCM constitution identification mod-el.Accuracy;precision;and F1 score were the three measures selected to assess the perfor-mance of the model.Results It was found that the accuracy;precision;and F1 score of the proposed model based on multifeatures for identifying TCM constitutions were 0.842;0.868;and 0.790;respectively.In comparison with the identification models that encompass a single feature;either a single facial complexion feature;a body shape feature;or deep features;the accuracy of the model that incorporating all the aforementioned features was elevated by 0.105;0.105;and 0.079;the precision increased by 0.164;0.164;and 0.211;and the F1 score rose by 0.071;0.071;and 0.084;respectively.Conclusion The research findings affirmed the viability of the proposed model;which incor-porated multifeatures;including the facial complexion feature;the body shape feature;and the deep feature.In addition;by employing the proposed model;the objectification and intel-ligence of identifying constitutions in TCM practices could be optimized.

Live facial feature extraction

Precise facial feature extraction is essential to the high-level face recognition and expression analysis. This paper presents a novel method for the real-time geometric facial feature extraction from live video. In this paper, the input image is viewed as a weighted graph. The segmentation of the pixels corresponding to the edges of facial components of the mouth, eyes, brows, and nose is implemented by means of random walks on the weighted graph. The graph has an 8-connected lattice structure and the weight value associated with each edge reflects the likelihood that a random walker will cross that edge. The random walks simulate an anisot- ropic diffusion process that filters out the noise while preserving the facial expression pixels. The seeds for the segmentation are obtained from a color and motion detector. The segmented facial pixels are represented with linked lists in the origi- nal geometric form and grouped into different parts corresponding to facial components. For the convenience of implementing high-level vision, the geometric description of facial component pixels is further decomposed into shape and reg- istration information. Shape is defined as the geometric information that is invariant under the registration transformation, such as translation, rotation, and isotropic scale. Statistical shape analysis is carried out to capture global facial fea- tures where the Procrustes shape distance measure is adopted. A Bayesian ap- proach is used to incorporate high-level prior knowledge of face structure. Experimental results show that the proposed method is capable of real-time extraction of precise geometric facial features from live video. The feature extraction is robust against the illumination changes, scale variation, head rotations, and hand interference.

Facial expression feature extraction method based on improved LBP

Local binary pattern(LBP)is an important method for texture feature extraction of facial expression.However,it also has the shortcomings of high dimension,slow feature extraction and noeffective local or global features extracted.To solve these problems,a facial expression feature extraction method is proposed based on improved LBP.Firstly,LBP is converted into double local binary pattern(DLBP).Then by combining Taylor expansion(TE)with DLBP,DLBP-TE algorithm is obtained.Finally,the DLBP-TE algorithm combined with extreme learning machine(ELM)is applied in seven kinds of ficial expression images and the corresponding experiments are carried out in Japanese adult female facial expression(JAFFE)database.The results show that the proposed method can significantly improve facial expression recognition rate.

Facial Index Based 2D Facial Composite Process for Forensic Investigation in Sri Lanka

The “facial composite” is one of the major fields in the forensic science that helps the criminal investigators to carry out their investigation process. The survey conducted by United States Law Enforcement Agencies confirms that 80% of the law enforcement agencies use computer automated composite systems whereas Sri Lanka is still far behind in the process of facial composite with lot of inefficiencies in the current manual process. Hence this research introduces a novel approach for the manual facial composite process, while eliminating the inefficiencies of the manual procedure in Sri Lanka. In order to overcome this situation, this study introduces an automated image processing based software solution with 2D facial feature templates targeting the Sri Lankan population. Thus, this was the first ever approach that creates the 2D facial feature templates by incorporating both medically defined indexes and relevant aesthetic aspects. Hence, this research study is comprised of two separate analyses on anthropometric indices and facial feature shapes which were carried out targeting the local population. Subsequently, several evaluation techniques were utilized to evaluate this methodology where we obtained an overall success rate as 70.19%. The ultimate goal of this research study is to provide a system to the law enforcement agencies in order to carry out an efficient and effective facial composite process which can lead to increase the success rate of suspect identification.

Race Classification Using Deep Learning

Race classification is a long-standing challenge in the field of face image analysis.The investigation of salient facial features is an important task to avoid processing all face parts.Face segmentation strongly benefits several face analysis tasks,including ethnicity and race classification.We propose a race-classification algorithm using a prior face segmentation framework.A deep convolutional neural network(DCNN)was used to construct a face segmentation model.For training the DCNN,we label face images according to seven different classes,that is,nose,skin,hair,eyes,brows,back,and mouth.The DCNN model developed in the first phase was used to create segmentation results.The probabilistic classification method is used,and probability maps(PMs)are created for each semantic class.We investigated five salient facial features from among seven that help in race classification.Features are extracted from the PMs of five classes,and a new model is trained based on the DCNN.We assessed the performance of the proposed race classification method on four standard face datasets,reporting superior results compared with previous studies.

Face Orientation Normalization Using Eye Positions

Despite the fact that progress in face recognition algorithms over the last decades has been made, changing lighting conditions and different face orientation still remain as a challenging problem. A standard face recognition system identifies the person by comparing the input picture against pictures of all faces in a database and finding the best match. Usually face matching is carried out in two steps： during the first step detection of a face is done by finding exact position of it in a complex background （various lightning condition）, and in the second step face identification is performed using gathered databases. In reality detected faces can appear in different position and they can be rotated, so these disturbances reduce quality of the recognition algorithms dramatically. In this paper to increase the identification accuracy we propose original geometric normalization of the face, based on extracted facial feature position such as eyes. For the eyes localization lbllowing methods has been used： color based method, mean eye template and SVM （Support Vector Machine） technique. Experimental investigation has shown that the best results for eye center detection can be achieved using SVM technique. The recognition rate increases statistically by 28% using face orientation normalization based on the eyes position.

FACE RECOGNITION BASED ON WAVELET-CURVELET-FRACTAL TECHNIQUE

In this paper,a novel face recognition method,named as wavelet-curvelet-fractal technique,is proposed. Based on the similarities embedded in the images,we propose to utilize the wave-let-curvelet-fractal technique to extract facial features. Thus we have the wavelet’s details in diagonal,vertical,and horizontal directions,and the eight curvelet details at different angles. Then we adopt the Euclidean minimum distance classifier to recognize different faces. Extensive comparison tests on dif-ferent data sets are carried out,and higher recognition rate is obtained by the proposed technique.

Pupil center detection with a single webcam for gaze tracking

This paper presents a user friendly approach to localize the pupil center with a single web camera.Several methods have been proposed to determine the coordinates of the pupil center in an image,but with practical limitations.The proposed method can track the user’s eye movements in real time under normal image resolution and lighting conditions using a regular webcam,without special equipment such as infrared illuminators.After the pre-processing steps used to deal with illumination variations,the pupil center is detected using iterative thresholding by applying geometric constraints.Experimental results show that robustness and speed in determining the pupil’s location in real time for users of various ethnicities,under various lighting conditions,at different distances from the webcam and with standard resolution images.

Forensic Analysis of Sexual Dimorphism from Facial Photographs

Background:Techniques for identifying facial features and proving the individuality of a person’s facial features are used in forensic anthropology.Aims and Objectives:Acommon method in this area is cranial anthropometry,which has uses in facial reconstruction,locating unidentified remains,and identifying criminal suspects.Materials and Methods:A study analyzed 200 standard full-face photographs of 50 males and 50 females to compare frontal and lateral facial images and identify male and female facial traits.We located landmarks and measured the distances between them.The data were analyzed using statistical tests like t-tests and analysis of variance(ANOVA)tests.Result:The results showed that in 100%of the suggested measurements for both males and females,there were no statistically significant differences(P>0.05)between frontal and lateral views.The ANOVA test,however,showed that all suggested measurements showed differences in the lateral aspect,while only 76.92%of them showed differences between the male and female groups in the frontal aspect.Conclusion:In conclusion,statistically significant differences were found,rejecting the null hypothesis of equal means,when the level of significance(α)was lower than the P value(0.05).On the other hand,the null hypothesis was accepted when statistically insignificant differences were found and the level of significance was higher than the P value.Although there were differences between males and females in different views,the distances between landmarks in frontal and lateral views of the same person were generally similar.Measurements taken in the lateral aspect were more accurate and displayed greater sexual dimorphism variation compared to those taken in the frontal aspect.