Robust facial landmark detection and tracking across poses and expressions for in-the-wild monocular video

We present a novel approach for automatically detecting and tracking facial landmarks acrossposesandexpressionsfromin-the-wild monocular video data,e.g.,You Tube videos and smartphone recordings.Our method does not require any calibration or manual adjustment for new individual input videos or actors.Firstly,we propose a method of robust 2D facial landmark detection across poses,by combining shape-face canonical-correlation analysis with a global supervised descent method.Since 2D regression-based methods are sensitive to unstable initialization,and the temporal and spatial coherence of videos is ignored,we utilize a coarse-todense 3D facial expression reconstruction method to refine the 2D landmarks.On one side,we employ an in-the-wild method to extract the coarse reconstruction result and its corresponding texture using the detected sparse facial landmarks,followed by robust pose,expression,and identity estimation.On the other side,to obtain dense reconstruction results,we give a face tracking flow method that corrects coarse reconstruction results and tracks weakly textured areas;this is used to iteratively update the coarse face model.Finally,a dense reconstruction result is estimated after it converges.Extensive experiments on a variety of video sequences recorded by ourselves or downloaded from You Tube show the results of facial landmark detection and tracking under various lighting conditions,for various head poses and facial expressions.The overall performance and a comparison with state-of-art methods demonstrate the robustness and effectiveness of our method.

Smart Drawing for Online Teaching

Teaching and learning-related activities have embraced digital technology especially,with the current global pandemic restrictions pervaded during the last two years.On the other hand,most of the academic and professional presentations are conducted using online platforms.But the presenter-audience interaction is hindered to a certain extent in an online case in contrary to face-to-face where real-time writing is beneficial when sketching is involved.The use of digital pens and pads is a solution for such instances,though the cost of acquiring such hardware is high.Economical solutions are essential as affordability is concerned.In this study,a real-time user-friendly,innovative drawing system is developed to address the issues related to the problems confronted with online presentations.This paper presents the development of an algorithm using Hand Landmark Detection,which replaces chalks,markers and regular ballpoint pens used in conventional communication and presentation,with online platforms.The proposed application in this study is acquired by Python and OpenCV libraries.The letters or the sketches drawn in the air were taken straight to the computer screen by this algorithm.The proposed algorithm continuously identifies the Hand Landmark using the images fed by the web camera,and text or drawing patterns are displayed on the screen according to the movements made by Hand Landmark on the image space.The developed user interface is also user-friendly.Hence the communication of letters and sketches were enabled.Although the concept has been developed and tested,with further research the versatility and accuracy of communication can be enhanced.