Research on Facial Fatigue Detection of Drivers with Multi-feature Fusion

In order to solve the shortcomings of current fatigue detection methods such as low accuracy or poor real-time performance,a fatigue detection method based on multi-feature fusion is proposed.Firstly,the HOG face detection algorithm and KCF target tracking algorithm are integrated and deformable convolutional neural network is introduced to identify the state of extracted eyes and mouth,fast track the detected faces and extract continuous and stable target faces for more efficient extraction.Then the head pose algorithm is introduced to detect the driver’s head in real time and obtain the driver’s head state information.Finally,a multi-feature fusion fatigue detection method is proposed based on the state of the eyes,mouth and head.According to the experimental results,the proposed method can detect the driver’s fatigue state in real time with high accuracy and good robustness compared with the current fatigue detection algorithms.

Driving fatigue fusion detection based on T-S fuzzy neural network evolved by subtractive clustering and particle swarm optimization

In order to improve the accuracy and reliability of the driving fatigue detection based on a single feature, a new detection algorithm based on multiple features is proposed. Two direct driver＇s facial features reflecting fatigue and one indirect vehicle behavior feature indicating fatigue are considered. Meanwhile, T-S fuzzy neural network（TSFNN）is adopted to recognize the driving fatigue of drivers. For the structure identification of the TSFNN, subtractive clustering（SC） is used to confirm the fuzzy rules and their correlative parameters. Moreover, the particle swarm optimization （PSO）algorithm is improved to train the TSFNN. Simulation results and experiments on vehicles show that the proposed algorithm can effectively improve the convergence speed and the recognition accuracy of the TSFNN, as well as enhance the correct rate of driving fatigue detection.

Smoke root detection from video sequences based on multi-feature fusion

Smoke detection is the most commonly used method in early warning of fire and is widely used in forest detection.Most existing smoke detection methods contain empty spaces and obstacles which interfere with detection and extract false smoke roots.This study developed a new smoke roots search algorithm based on a multi-feature fusion dynamic extraction strategy.This determines smoke origin candidate points and region based on a multi-frame discrete confidence level.The results show that the new method provides a more complete smoke contour with no background interference,compared to the results using existing methods.Unlike video-based methods that rely on continuous frames,an adaptive threshold method was developed to build the judgment image set composed of non-consecutive frames.The smoke roots origin search algorithm increased the detection rate and significantly reduced false detection rate compared to existing methods.

Recent Advances in Fatigue Detection Algorithm Based on EEG

Fatigue is a state commonly caused by overworked,which seriously affects daily work and life.How to detect mental fatigue has always been a hot spot for researchers to explore.Electroencephalogram(EEG)is considered one of the most accurate and objective indicators.This article investigated the devel-opment of classification algorithms applied in EEG-based fatigue detection in recent years.According to the different source of the data,we can divide these classification algorithms into two categories,intra-subject(within the same sub-ject)and cross-subject(across different subjects).In most studies,traditional machine learning algorithms with artificial feature extraction methods were com-monly used for fatigue detection as intra-subject algorithms.Besides,deep learn-ing algorithms have been applied to fatigue detection and could achieve effective result based on large-scale dataset.However,it is difficult to perform long-term calibration training on the subjects in practical applications.With the lack of large samples,transfer learning algorithms as a cross-subject algorithm could promote the practical application of fatigue detection methods.We found that the research based on deep learning and transfer learning has gradually increased in recent years.But as afield with increasing requirements,researchers still need to con-tinue to explore efficient decoding algorithms,design effective experimental para-digms,and collect and accumulate valid standard data,to achieve fast and accurate fatigue detection methods or systems to further widely apply.

Driver Fatigue Detection System Based on Colored and Infrared Eye Features Fusion

Real-time detection of driver fatigue status is of great significance for road traffic safety.In this paper,a proposed novel driver fatigue detection method is able to detect the driver’s fatigue status around the clock.The driver’s face images were captured by a camera with a colored lens and an infrared lens mounted above the dashboard.The landmarks of the driver’s face were labeled and the eye-area was segmented.By calculating the aspect ratios of the eyes,the duration of eye closure,frequency of blinks and PERCLOS of both colored and infrared,fatigue can be detected.Based on the change of light intensity detected by a photosensitive device,the weight matrix of the colored features and the infrared features was adjusted adaptively to reduce the impact of lighting on fatigue detection.Video samples of the driver’s face were recorded in the test vehicle.After training the classification model,the results showed that our method has high accuracy on driver fatigue detection in both daytime and nighttime.

Color space lip segmentation for drivers＇ fatigue detection

to the chroma distribution diversity （CDD） between lip color and skin color, the lip color area is segmented by the back propagation neural network （BPNN） with three typical color features. Isolated noisy points of the lip color area in binary image are eliminated by a proposed re- gion connecting algorithm. An improved integral projection algorithm is presented to locate the lip boundary. Whether a driver is fatigued is recognized by the ratio of the frame number of the images with mouth opening continuously to the total image frame number in every 20s. The experiments show that the proposed algorithm provides higher correct rate and reliability for fatigue driving detec- tion, and is superior to the single color feature-based method in the lip color segmention. Besides, it improves obviously the accuracy and speed of the lip boundary location compared with the traditional integral projection algrothm.

Vision-based fatigue crack detection using global motion compensation and video feature tracking

Fatigue cracks that develop in civil infrastructure such as steel bridges due to repetitive loads pose a major threat to structural integrity.Despite being the most common practice for fatigue crack detection,human visual inspection is known to be labor intensive,time-consuming,and prone to error.In this study,a computer vision-based fatigue crack detection approach using a short video recorded under live loads by a moving consumer-grade camera is presented.The method detects fatigue crack by tracking surface motion and identifies the differential motion pattern caused by opening and closing of the fatigue crack.However,the global motion introduced by a moving camera in the recorded video is typically far greater than the actual motion associated with fatigue crack opening/closing,leading to false detection results.To overcome the challenge,global motion compensation(GMC)techniques are introduced to compensate for camera-induced movement.In particular,hierarchical model-based motion estimation is adopted for 2D videos with simple geometry and a new method is developed by extending the bundled camera paths approach for 3D videos with complex geometry.The proposed methodology is validated using two laboratory test setups for both in-plane and out-of-plane fatigue cracks.The results confirm the importance of motion compensation for both 2D and 3D videos and demonstrate the effectiveness of the proposed GMC methods as well as the subsequent crack detection algorithm.

A new real-time eye tracking based on nonlinear unscented Kalman filter for monitoring driver fatigue

A new scheme for driver fatigue detection is presented, which is based on the nonlinear unscented Kalman filter and eye tracking. Assuming a probability distribution than to approximate an arbitrary nonlinear function or transformation, eye nonlinear tracking can be achieved using an unscented transformation (UT), which adopts a set of deterministic sigma points to match the posterior probability density function of the eye movement. Driver fatigue can be detected using the percentage of eye closure (PERCLOS) framework in a realistic driving condition after the eye nonlinear tracking. This system was tested adequately in realistic driving environments with subjects of different genders, with/without glasses, in day/night driving, being commercial/noncommercial drivers, in continuous driving time, and under different road conditions. The last experimental results show that the proposed method not only improves the robustness for nonlinear eye tracking, but also can provide more accurate estimation than the traditional Kalman filter.

COMPUTER CONTROLLED METHOD FOR MEASUREMENT OFSURFACE CRACK LENGTH ON PLATE SUBJECTEDTO FATIGUE LOADING

The calibration curves obtained using strain gages are used to predict surface crack length on plate specimen subjected to 4-point bending fatigue loading. The results shows that the proposed procedure is of high precision with the maximum error percentage being less than 6%, and it can be easily used to estimate or monitor the surface crack length under fatigue loading both in laboratory and in engineering. It is also quite meanful for nondamage detecting.

装配式建筑梁柱节点疲劳损伤检测方法研究

常规的装配式建筑梁柱节点疲劳损伤检测主要采用梁柱图像分析实现,忽略了节点内部损伤的影响,导致检测结果相对变化量系数与实际系数的差值较大。本文对装配式建筑梁柱节点疲劳损伤检测方法进行了研究,于装配式建筑梁柱周围布置测点对梁柱节点的应变状态进行分析;根据分析结果,结合梁柱内部能量耗散值,计算节点疲劳损伤能量系数;将该系数作为节点应变能密度因子分析节点轴向力,由此得出检测的梁柱节点疲劳损伤值。实验结果表明,所提方法应用后梁柱节点疲劳损伤检测结果表现出的相对变化量系数差值较小,检测结果准确率较高,满足了装配式建筑梁柱安全运维工作的现实需求。

Fatigue driving detection based on Haar feature and extreme learning machine

As the significant branch of intelligent vehicle networking technology, the intelligent fatigue driving detection technology has been introduced into the paper in order to recognize the fatigue state of the vehicle driver and avoid the traffic accident. The disadvantages of the traditional fatigue driving detection method have been pointed out when we study on the traditional eye tracking technology and traditional artificial neural networks. On the basis of the image topological analysis technology, Haar like features and extreme learning machine algorithm, a new detection method of the intelligent fatigue driving has been proposed in the paper. Besides, the detailed algorithm and realization scheme of the intelligent fatigue driving detection have been put forward as well. Finally, by comparing the results of the simulation experiments, the new method has been verified to have a better robustness, efficiency and accuracy in monitoring and tracking the drivers＇ fatigue driving by using the human eye tracking technology.

基于车载视觉的端到端驾驶员疲劳检测模型

营运驾驶员长时间疲劳驾驶是导致事故发生的重要原因,为此,企业在营运车辆上安装相机采集驾驶员面部视频,基于模型和算法自动识别驾驶员的疲劳状态,通过语音提醒甚至启用远程护航进行疲劳干预,以此提高驾驶安全。现有的疲劳检测研究大多数都是基于面部关键点检测的算法,该类算法对面部视频的质量要求严格。在真实的营运行车环境中,夜晚光线过差,相机位置安装不理想,驾驶员面部遮挡等均会造成关键点检测失效,从而影响模型的准确性。基于卷积神经网络(CNN)和长短时记忆神经网络(LSTM)设计了一种端到端营运驾驶员疲劳检测模型,该模型以相机采集的驾驶员面部视频作为输入,使用CNN网络提取视频单帧特征,在此基础上将时序单帧特征作为LSTM网络的输入来最终识别驾驶员的疲劳状态,实验表明,模型的接收者操作特征曲线下面积(AUC)为0.9,远优于现有的面部关键点模型。此外,为了提高该模型在实际行车环境中的鲁棒性,基于光线变化及相机变化的模拟操作在训练数据上进行了数据增强,通过模型重训练进一步提高了模型的精度及鲁棒性。实验结果表明,改进前,营运车辆行车环境下模型的AUC相比实验室模型下降37.3%,而改进后AUC仅下降9.7%,模型的鲁棒性得到改善,能够更好地适应复杂的营运车辆自然驾驶环境。

基于信号响应分析模型的金属结构损伤导波检出概率

飞机结构损伤导波在线监测技术作为一种新颖的无损检测手段,为了真正实现该技术在结构运营维护过程中的视情维护,必须明确其结构损伤检出概率(probability of detection,POD),以指导结构检查维修方案的制定。提出了一种基于信号响应分析模型的结构损伤导波POD计算方法,该方法通过构建在线导波监测信号的损伤指数与裂纹长度间的对应关系,得到结构损伤POD的统计计算模型,并分析了拟合参数的不确定性对计算模型的影响,构建了不同置信度下的导波POD计算模型。通过开展金属开孔和搭接结构疲劳裂纹导波监测试验,验证了该方法的有效性。试验结果表明,损伤指数类型、对应关系拟合函数和传感器监测方案均对结构损伤导波POD具有影响,且在95%置信度90%POD下金属开孔和搭接结构的可检裂纹长度分别约为2.6 mm和9.5 mm。

多周期动态循环应力下的J-A-N力磁耦合机理模型

目前的力磁耦合J-A模型未考虑钉扎场的畴壁厚度与多周期动态循环应力的影响,导致J-A模型的应力磁化数值偏大,不能准确描述早期疲劳损伤。为此,基于畴壁理论和Burgers位错理论,引入畴壁厚度因子,改进J-A模型的钉扎场方程,进一步考虑循环载荷的应力幅、循环周次N等因素,建立了多周期动态循环应力下的J-A-N力磁耦合机理模型。获得了不同动态循环的应力幅值σa与平均应力σm下的磁化规律:相同循环周次下,σa主要影响应力磁化率,σm主要影响应力饱和磁化大小;σm相同时,随着σa的增大,达到应力饱和状态的速度升高;σa相同时,随着σm的增大,应力饱和磁化强度逐渐减小。为验证J-A-N模型有效性,对45钢三点弯曲试件进行了多周期动态循环应力下的磁场信号检测实验,实验结果与模型结果一致。

车用SLM成形DP780钢的疲劳损伤非线性电磁超声换能检测

为了提高对车用选区激光熔化(SLM)成形DP780钢的疲劳损伤检测精度,提出了一种基于非线性电磁超声换能的疲劳加载信号分析方法,通过高次谐波检测技术测定合金的损伤情况。研究结果表明:时域信号波形相对较为稳定,基波幅值在频率为5 MPa时出现,在5000次疲劳测试后,与初始试件相比,试样中产生信号的幅值更显著。非线性电磁超声换能检测很容易影响二次谐波的产生。二次谐波具有更高的幅值,大量疲劳裂纹存在于基体内部。本研究可满足精准检测疲劳损伤条件。在对疲劳损伤程度确定时,参考疲劳寿命和循环加载周次,超声非线性系数在周边测试点处增长幅度相对较小,中心测试点较大,主要原因是内部结构变形情况更明显。

基于卷积神经网络的疲劳检测改进算法

为了解决当前的疲劳检测算法准确率低或实时性差的缺点,提出一种改进的卷积神经网络疲劳检测算法。使用HOG检测算法结合KCF跟踪算法对采集的人脸进行检测和跟踪;随后调用Dlib库进行脸部关键点的提取;通过引入可变形卷积神经网络对提取的眼部和嘴部进行状态识别;通过CEW和YAWDD数据集进行测试,疲劳检测准确率达到94.36%。实验表明,与当前的疲劳检测算法相比,提出的方法能够实时地检测驾驶员疲劳,并且具有较高的准确率。

基于YOLOv7-DCA的疲劳检测方法研究

针对疲劳检测中小尺度检测效果不佳和实时性差等问题,以矿井提升机司机疲劳检测为目标,对YOLOv7的结构进行精简并且基于AIoU(Area Intersection over Union)损失函数优化预测框与验证框的回归过程.在模型中引入双通道注意力机制实现小尺度特征的信息增强,通过融合眨眼频率、闭眼时长和打哈欠时长来判断司机的状态.实验结果表明,本文方法对疲劳检测精度达到98.85%,检测速度达到70 FPS,与其他算法相比,本文算法具有更好的准确性和实时性.

基于语谱图的管制员疲劳状态检测研究

现阶段利用陆空通话语音对管制员疲劳状态的研究中,大多只考虑了语音在时域或频域的变化,而忽视了疲劳会同时在时域与频域上产生影响。将三种疲劳状态下的陆空通话语音分别转化为可同时反映时域与频域特性的语音频谱图像,利用灰度共生矩阵提取四维典型的特征参数,对比管制员在不同状态下特征参数的变化情况,构建管制员疲劳检测模型并对输入特征进行检测。结果表明:利用语谱图特征结合传统特征作为输入特征的检测准确率最高,达到95.49%,较单一使用传统特征的检测准确率高出4%;管制员疲劳状态的变化会直观地反映在语谱图上,会对其特征值产生影响,利用这种影响对管制员疲劳状态进行检测,可以得到良好的检测结果。

结合视角矫正和改进ViViT的驾驶员睡意判断方法

针对传统检测方法中摄像头视角受限问题,提出了一种结合面部姿态矫正和改进ViViT的多视角下人脸疲倦检测方法。采用Mediapipe Face Mesh定位面部三维特征点并将其矫正为正面,利用提出的FGR-ViViT模型来捕捉矫正后的眼睛、眉毛、嘴巴线条图像帧序列变化。FGR-ViViT通过在ViViT的Temporal Transformer Encoder中添加部件选择模块来捕捉特征在时间维度中的细微差异,同时融合2次dropout和改进的对比损失函数来调整样本的相似性,降低模型过拟合风险并提高泛化能力。实验结果表明,提出的方法在YawDD和DROZY矫正后的线条图像帧的测试集上,F1-分数达到了94.5%和97.6%,相较于原始人脸图像帧分别提高了3.2%和10.4%,其FGR-ViViT相较于原始ViViT分别提高了6.1%和0.7%。所提方法适用于摄像头灵活摆放的多种应用场景,对解决多视角人脸睡意判断具有积极意义。

基于卷积神经网络的学习疲劳检测研究

学习疲劳检测有助于教师获取学生的不良学习状态,从而进行针对性的干预,提高教学质量,促进学生身心健康发展。本文提出一种基于卷积神经网络的学习疲劳检测方法,该方法基于改进的SSD目标检测算法实现学生面部的实时精准检测,然后将面部图像输入改进的VGG16深度卷积神经网络进行学习疲劳特征的全面有效提取,实现学习疲劳的高效识别。实验结果表明,该方法既实现了人脸的精准定位,又显著提升了人脸检测速度,并明显地提高了疲劳识别的准确度。