仿生假体视觉人脸图像语义翻译研究与实现

Research and Implementation of Human Face Image Semantic Translation in Simulated Prosthetic Vision

仿生假体视觉技术可以为视力障碍患者在日常交流中提供人脸语义信息,但当前假体制造技术产生的光感信息像素低,很难清晰传递人脸语义。本文提出将“物理真实”转为“感官真实”的研究思路,将人脸语义翻译为表达更加精炼的漫画形式,构建基于间接循环重构对抗训练策略的F2Pnet模型,使用有限像素的光感信息为患者呈现较为精准的人脸关键语义。通过用户调研表明,F2Pnet的人脸语义翻译在语义相似度、表情辨识度和身份辨识度方面已接近人类基准,尤其在“愤怒”、“恐惧”、“高兴”、“惊讶”4种表情的辨识度已超越人类基准,“高兴”的辨识准确率已达0.96,在面部特征、表情等方面的表达能力优于现有其他人脸图像翻译方法。

Simulated prosthetic vision technology can provide visual disorder patients with human face semantic information in daily communication.However,the current prosthesis manufacturing technology only produces lowpixel lightsensinginformationand it is difficult to clearly convey the semantics of human faces.This paper proposes the research idea of transforming“physical reality”into“sensory reality”,translating human face semantics into a more refined cartoon form,constructing the F2Pnet model based on an indirect cyclic reconstruction adversarial training strategy,and presenting more accurate human face key semantics for patients by using the light-sensing information of limited pixels.The user survey demonstrates that the human face semantic translation of F2Pnet is close to human benchmarks in terms of semantic similarity,expression recognition and identity recognition.In particular,the recognition degree of the four expressions of“anger”,“fear”,“happy”and“surprised”has surpassed the human benchmark,and the recognition accuracy of“happy”has reached 0.96.The expressive ability of F2Pnet in terms of facial features,expressions,etc.is superior to other existing human face image translation methods.