Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted,Soft Epidermal Electronics

The facial expressions are a mirror of the elusive emotion hidden in the mind,and thus,capturing expressions is a crucial way of merging the inward world and virtual world.However,typical facial expression recognition(FER)systems are restricted by environments where faces must be clearly seen for computer vision,or rigid devices that are not suitable for the time-dynamic,curvilinear faces.Here,we present a robust,highly wearable FER system that is based on deep-learning-assisted,soft epidermal electronics.The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions,releasing the constraint of movement,space,and light.The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample.The proposed wearable FER system is superior for wide applicability and high accuracy.The FER system is suitable for the individual and shows essential robustness to different light,occlusion,and various face poses.It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place.This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment,enabling promising human-computer interaction applications.

All-weather,natural silent speech recognition via machine-learning-assisted tattoo-like electronics

The internal availability of silent speech serves as a translator for people with aphasia and keeps human–machine/human interactions working under various disturbances.This paper develops a silent speech strategy to achieve all-weather,natural interactions.The strategy requires few usage specialized skills like sign language but accurately transfers high-capacity information in complicated and changeable daily environments.In the strategy,the tattoo-like electronics imperceptibly attached on facial skin record high-quality bio-data of various silent speech,and the machine-learning algorithm deployed on the cloud recognizes accurately the silent speech and reduces the weight of the wireless acquisition module.A series of experiments show that the silent speech recognition system(SSRS)can enduringly comply with large deformation(~45%)of faces by virtue of the electricitypreferred tattoo-like electrodes and recognize up to 110 words covering daily vocabularies with a high average accuracy of 92.64%simply by use of small-sample machine learning.We successfully apply the SSRS to 1-day routine life,including daily greeting,running,dining,manipulating industrial robots in deafening noise,and expressing in darkness,which shows great promotion in real-world applications.

Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics

The facial expressions are a mirror of the elusive emotion hidden in the mind,and thus,capturing expressions is a crucial way of merging the inward world and virtual world.However,typical facial expression recognition(FER)systems are restricted by environments where faces must be clearly seen for computer vision,or rigid devices that are not suitable for the time-dynamic,curvilinear faces.Here,we present a robust,highly wearable FER system that is based on deep-learning-assisted,soft epidermal electronics.The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions,releasing the constraint of movement,space,and light.The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample.The proposed wearable FER system is superior for wide applicability and high accuracy.The FER system is suitable for the individual and shows essential robustness to different light,occlusion,and various face poses.It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place.This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment,enabling promising human-computer interaction applications.

Chest-scale self-compensated epidermal electronics for standard 6-precordial-lead ECG

Six chest leads are the standardized clinical devices of diagnosing cardiac diseases.Emerging epidermal electronics technology shift the dangling wires and bulky devices to imperceptible wearing,achieving both comfortable experience and high-fidelity measuring.Extending small areas of current epidermal electronics to the chest scale requires eliminating interference from long epidermal interconnects and rendering the data acquisition(DAQ)portable.Herein,we developed a chest-scale epidermal electronic system(EES)for standard precordial-lead ECG and hydration monitoring,including the onlyμm-thick substrate-free epidermal sensing module and the soft wireless DAQ module.An electrical compensation strategy using double channels within the DAQ module and epidermal compensated branches(ECB)is proposed to eliminate unwanted signals from the long epidermal interconnects and to achieve the desired ECG.In this way,the EES works stably and precisely under different levels of exercise.Patients with sinus arrhythmias have been tested,demonstrating the prospect of EES in cardiac diseases.