Multi-task Learning of Semantic Segmentation and Height Estimation for Multi-modal Remote Sensing Images

Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.

A General Linguistic Steganalysis Framework Using Multi-Task Learning

Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.

Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks

In this paper, we proposed a multi-task system that can identify dish types, food ingredients, and cooking methods from food images with deep convolutional neural networks. We built up a dataset of 360 classes of different foods with at least 500 images for each class. To reduce the noises of the data, which was collected from the Internet, outlier images were detected and eliminated through a one-class SVM trained with deep convolutional features. We simultaneously trained a dish identifier, a cooking method recognizer, and a multi-label ingredient detector. They share a few low-level layers in the deep network architecture. The proposed framework shows higher accuracy than traditional method with handcrafted features, and the cooking method recognizer and ingredient detector can be applied to dishes which are not included in the training dataset to provide reference information for users.

应用于人脸识别的多任务卷积神经网络性能优化

对一种多任务卷积神经网络的人脸识别性能进行研究与优化。该神经网络采用3个独立的任务网络分别进行人脸检测、关键点定位和人脸识别。让3个任务网络在训练过程中共享底层卷积层的特征表示,使得模型能针对多个任务同时进行学习,进而提高其泛化能力和识别精度。为了增强模型对图像的学习能力,采用一种数据增强和迁移学习技术,使人脸识别系统的准确性、鲁棒性和可靠性均得到了显著提升。研究结果为发展人脸识别技术提供了新的思路,尤其在处理复杂场景和多样化人脸图像方面具有一定的应用前景。

基于改进MTCNN的人脸检测算法

针对经典的人脸检测卷积神经网络模型(CNN)对人脸检测准确率不高的问题,设计一种改进的多任务卷积神经网络(MTCNN).通过对原始MTCNN算法进行迁移学习,微调模型参数,提出人脸误检判别公式.在LFW人脸数据集上进行实验,先调整MTCNN关键参数的值,找出最合适的人脸置信度阈值,再使用人脸误检判别公式.实验结果表明:改进后的MTCNN算法较改进之前在检测准确率上有了较大提升,而且提出的改进策略使自然环境中人脸检测的速度也有了提高.

基于MTCNN和Facenet的人脸识别系统设计

由于传统人脸识别系统多采用手工进行特征设定,存在识别精度低、速度慢等缺点,因此文中设计一种基于MTCNN和Facenet的人脸识别系统。采用MTCNN模型进行人脸边框回归,通过三阶级联卷积神经网络对人脸图像进行从粗到细的提取;采用Facenet模型进行人脸特征向量提取,构建本地人脸特征库;通过比对待识别人脸特征向量与本地人脸特征库中向量间的欧氏距离,输出识别结果。为验证系统性能,从检测速度与检测精度两方面进行测试。实验结果表明,文中所设计的系统识别速度达25 f/s以上,当特征向量间的欧氏距离的阈值设定为0.60时,在数据集LFW上的识别率最高达到99.27%。该系统检测速度满足实时性的同时具有较高的检测精度。

基于改进MTCNN网络的目标人脸快速检测

传统检测网络在复杂背景下一直存在检测效率低以及准确率低等问题。针对以上问题,在MTCNN网络上进一步设计了MT-Siam网络,主要为以后的单目标独立分割、单目标图像处理等操作快速地提供精准位置定位,从而快速获取目标位置,达到提高检测效率的目的。实验部分在YOLOv3、SSD300和MTCNN基础模型上进行了全面对比,验证了MTCNN网络的优越性,并从对比实验中得出MT-Siam网络在保持高精度的前提下,检测速度与MTCNN网络相比得到70%~85%不等的提升。

铁路刷脸场景下基于MTCNN的人脸遮挡识别研究

多任务卷积神经网络(MTCNN[1])是现在工业界广泛应用的人脸识别算法,用三个级联网络实现了人脸检测与关键点标定。针对铁路刷脸进站、检票、出站等应用场景的实际需求,需要识别被验人是否佩戴墨镜、口罩等遮挡物,对MTCNN模型进行优化改造,提出了两种基于MTCNN的人脸遮挡识别模型,一个模型仍采用三个网络并对最后一个网络进行改造,另一个模型增加第四个级联网络对遮挡信息进行识别。两个模型均可实现在人脸检测和五点标定的同时,进行人脸关键点遮挡信息的检测识别。

基于MTCNN的驾驶员疲劳检测方法

驾驶疲劳是引发交通事故的重要原因之一。研究驾驶疲劳检测算法对提高道路交通安全性和人类生命财产安全具有重要意义。为此,文章提出了一种基于MTCNN的疲劳检测方法。通过对人脸图像进行处理,使MTCNN算法得到优化,然后通过训练样本进行实验验证,证实了算法对实际的驾驶疲劳检测的有用性。

基于计算机视觉的考勤系统设计与实现

针对传统课堂考勤中耗时长、效率低等问题,提出了一种基于计算机视觉的考勤系统,利用深度学习进行人脸识别与手机入袋检测,记录学生的到课情况与手机上交情况。为将考勤信息可视化,设计了3种登录模式的综合考勤系统。实验结果表明,该系统不仅能在毫秒级的时间内完成检测,而且平均准确率(mean Average Precision,mAP)0.5达到0.990,保证了精确率和召回率。

Power flow forecasts at transmission grid nodes using Graph Neural Networks

The increasing share of renewable energy in the electricity grid and progressing changes in power consumption have led to fluctuating,and weather-dependent power flows.To ensure grid stability,grid operators rely on power forecasts which are crucial for grid calculations and planning.In this paper,a Multi-Task Learning approach is combined with a Graph Neural Network(GNN)to predict vertical power flows at transformers connecting high and extra-high voltage levels.The proposed method accounts for local differences in power flow characteristics by using an Embedding Multi-Task Learning approach.The use of a Bayesian embedding to capture the latent node characteristics allows to share the weights across all transformers in the subsequent node-invariant GNN while still allowing the individual behavioral patterns of the transformers to be distinguished.At the same time,dependencies between transformers are considered by the GNN architecture which can learn relationships between different transformers and thus take into account that power flows in an electricity network are not independent from each other.The effectiveness of the proposed method is demonstrated through evaluation on two real-world data sets provided by two of four German Transmission System Operators,comprising large portions of the operated German transmission grid.The results show that the proposed Multi-Task Graph Neural Network is a suitable representation learner for electricity networks with a clear advantage provided by the preceding embedding layer.It is able to capture interconnections between correlated transformers and indeed improves the performance in power flow prediction compared to standard Neural Networks.A sign test shows that the proposed model reduces the test RMSE on both data sets compared to the benchmark models significantly.

基于边缘计算的疲劳驾驶检测方法

现有疲劳驾驶检测方法通常将驾驶过程中采集的数据传输至云端进行分析,然而在车辆移动过程中网络覆盖范围、响应速度等因素会造成检测实时性差。为在车载嵌入式设备上对驾驶人疲劳状态进行准确预警,提出一种基于边缘计算的疲劳驾驶检测方法。通过改进的多任务卷积神经网络确定人脸区域,根据人脸的面部比例关系定位驾驶人的眼部与嘴部区域,利用基于Ghost模块的轻量化AlexNet分类检测眼部与嘴部的开闭状态,并结合PERCLOS和PMOT指标值实现疲劳检测。在NHTU-DDD数据集上的实验结果表明,该方法在树莓派4B开发板上的检测准确率达到93.5%且单帧平均检测时间为180 ms,在保障检测准确率的同时大幅降低了计算量,能较好地满足疲劳驾驶的实时检测需求。

基于Caffe框架的人脸定位与识别系统的设计

人脸识别是生物特征识别的重要内容,其应用范围也越来越广,如手机的人脸解锁,支付宝的刷脸支付和公共区域的犯罪分子检测等。文章提出了一种基于Caffe框架的人脸定位与识别系统的设计方法。该方法中的人脸定位采用多任务卷积神经网络(Multi-task convolutional neural network,简称MTCNN),并基于Caffe框架实现卷积神经网络(Convolutional Neural Networks,简称CNN)来对人脸进行分类与识别,采用YouTube以及Labeled Faces数据集作为实验数据集,实现了人脸定位和人脸识别功能,取得了较好的定位与识别效果。

基于卷积神经网络的多人表情识别算法

表情识别在人机交互领域有着重要的意义,卷积神经网络是研究表情识别的一种方法。由于表情识别问题的复杂性,卷积神经网络的结构越来越复杂,参数越来越多,计算复杂度也越来越大,容易出现过拟合现象。利用MTCNN模型进行人脸检测,引入inception模型,使用1×1的卷积核,降低特征图的厚度,平衡网络深度和宽度。经实验测试,获得较好的识别结果。

基于深度学习的轻量级和多姿态人脸识别方法

目前基于深度学习的人脸识别方法存在识别模型参数量大、特征提取速度慢的问题,而且现有人脸数据集姿态单一,在实际人脸识别任务中无法取得好的识别效果。针对这一问题建立了一种多姿态人脸数据集,并提出了一种轻量级的多姿态人脸识别方法。首先,使用多任务级联卷积神经网络(MTCNN)算法进行人脸检测,并且使用MTCNN最后包含的高层特征做人脸跟踪;然后,根据检测到的人脸关键点位置来判断人脸姿态,通过损失函数为ArcFace的神经网络提取当前人脸特征,并将当前人脸特征与相应姿态的人脸数据库中的人脸特征比对得到人脸识别结果。实验结果表明,提出方法在多姿态人脸数据集上准确率为96.25%,相较于单一姿态的人脸数据集,准确率提升了2.67%,所提方法能够有效提高识别准确率。

Face anti-spoofing algorithm combined with CNN and brightness equalization

Face anti-spoofing is a relatively important part of the face recognition system,which has great significance for financial payment and access control systems.Aiming at the problems of unstable face alignment,complex lighting,and complex structure of face anti-spoofing detection network,a novel method is presented using a combination of convolutional neural network and brightness equalization.Firstly,multi-task convolutional neural network(MTCNN)based on the cascade of three convolutional neural networks(CNNs),P-net,R-net,and O-net are used to achieve accurate positioning of the face,and the detected face bounding box is cropped by a specified multiple,then brightness equalization is adopted to perform brightness compensation on different brightness areas of the face image.Finally,data features are extracted and classification is given by utilizing a 12-layer convolution neural network.Experiments of the proposed algorithm were carried out on CASIA-FASD.The results show that the classification accuracy is relatively high,and the half total error rate(HTER)reaches 1.02%.

基于活体检测和身份认证的人脸识别安防系统

人脸识别由于其便捷性和实用性而被广泛应用于各种门禁等场合,但容易受到多种形式的欺骗攻击(如照片攻击和视频攻击)。基于深度卷积神经网络(CNN)的活体检测虽然能够解决以上问题,但是却存在计算量大、对用户不友好以及难以部署于嵌入式系统等缺点,因此提出了一种实时的轻量级的人脸识别安全分类方法。通过将基于色彩纹理分析的人脸活体检测算法与人脸认证算法相融合,提出了一种在无需用户配合的单目摄像头场景下进行人脸活体检测和人脸验证的人脸识别算法。该算法支持实时人脸识别,具有更高的活体检测识别率与鲁棒性。为了验证该算法的性能,以CASIA-FASD和Replay-Attack作为实验的基准数据集,结果表明在活体检测中该算法的半错误率(HTER)为9.7%,等错误率(EER)为5.5%,而且在整个流程中处理1帧图像所需时间为0.12 s,验证了该算法的可行性和有效性。

关于人脸识别算法对不同照明技术的鲁棒性研究

对于在正常照明技术下采集得到的人脸图像,现有的人脸识别算法如MTCNN算法、RetinaFace算法,已经能够取得相当高的人脸识别率。然而在某些特殊应用中,对于在特殊照明技术下采集得到的人脸图像,现有的人脸识别算法是否具有很好的鲁棒性,保持较高的人脸识别率,并没有确切的实验结果能够给出我们结论。本文收集了6 000多张在不同照明技术下得到的人脸图像作为我们的混合人脸数据库,并利用LBPH算法、卷积神经网络(CNN)、MTCNN算法、RetinaFace算法设计出了四种有效的人脸识别网络,分别对开源的WIDER FACE人脸数据库和我们的混合人脸数据库进行了测试。最终发现RetinaFace算法对于不同照明技术得到的人脸图像具有较好的鲁棒性。我们进一步利用深度学习标注工具对RetinaFace算法误判的人脸图像进行了标注,并将标注后的图像送入到RetinaFace人脸识别网络中重新训练,优化后的RetinaFace人脸检测模型得到了98.6%的人脸识别准确率,使得RetinaFace算法对不同光照条件的鲁棒性取得了进一步的提升。

基于多任务卷积神经网络的人脸识别技术研究

深度神经网络是目前计算机机器学习领域的一个关键技术,可应用于图像处理。其中,多任务卷积神经网络(Multi-task Convolutional Neural Network,MTCNN)是一种基于卷积神经网络的多任务人脸检测框架,这里采用MTCNN人脸检测模型代替传统的卷积神经网络,在深度学习框架TensorFlow上进行人脸识别。首先,在数据预处理阶段利用灰度化方法将图像集转变为灰度图,降低图像通道。其次,基于MTCNN构建人脸检测模型,并利用Softmax函数进行分类识别。最后,实验过程中选择不同迭代次数进行准确性对比,在模型趋于稳定的情况下,得到较高的准确性。

基于深度学习的人脸遮挡检测方法

针对银行自动柜员机环境下不法分子通过遮挡面部实施违法犯罪行为的安全问题,采用将深度学习中MTCNN多任务卷积神经网络和ResNet残差分类网络相结合的方法,先利用MTCNN模型实现对图像中人脸的检测,再通过残差分类网络实现对已检测人脸有无遮挡的分类,最终实现人脸遮挡检测。实验证明本文方法优于传统方法,人脸遮挡检测率较高,具有一定的工程应用价值。