A Survey and Tutorial of EEG-Based Brain Monitoring for Driver State Analysis

The driver’s cognitive and physiological states affect his/her ability to control the vehicle.Thus,these driver states are essential to the safety of automobiles.The design of advanced driver assistance systems(ADAS)or autonomous vehicles will depend on their ability to interact effectively with the driver.A deeper understanding of the driver state is,therefore,paramount.Electroencephalography(EEG)is proven to be one of the most effective methods for driver state monitoring and human error detection.This paper discusses EEG-based driver state detection systems and their corresponding analysis algorithms over the last three decades.First,the commonly used EEG system setup for driver state studies is introduced.Then,the EEG signal preprocessing,feature extraction,and classification algorithms for driver state detection are reviewed.Finally,EEG-based driver state monitoring research is reviewed in-depth,and its future development is discussed.It is concluded that the current EEGbased driver state monitoring algorithms are promising for safety applications.However,many improvements are still required in EEG artifact reduction,real-time processing,and between-subject classification accuracy.

Methodical Approach to Integrate Human Movement Diversity in Real-Time into a Virtual Test Field for Highly Automated Vehicle Systems

Recently, virtual realities and simulations play important roles in the development of automated driving functionalities. By an appropriate abstraction, they help to design, investigate and communicate real traffic scenario complexity. Especially, for edge cases investigations of interactions between vulnerable road users (VRU) and highly automated driving functions, valid virtual models are essential for the quality of results. The aim of this study is to measure, process and integrate real human movement behaviour into a virtual test environment for highly automated vehicle functionalities. The overall system consists of a georeferenced virtual city model and a vehicle dynamics model, including probabilistic sensor descriptions. By motion capture hardware, real humanoid behaviour is applied to a virtual human avatar in the test environment. Through retargeting methods, which enable the independency of avatar and person under test (PuT) dimensions, the virtual avatar diversity is increased. To verify the biomechanical behaviour of the virtual avatars, a qualitative study is performed, which funds on a representative movement sequence. The results confirm the functionality of the used methodology and enable PuT independence control of the virtual avatars in real-time.

Intelligent speed adaptation for visibility technology affects drivers’speed selection along curves with sight limitations

Sight obstructions along road curves can lead to a crash if the driver is not able to stop the vehicle in time.This is a particular issue along curves with limited available sight,where speed management is necessary to avoid unsafe situations(e.g.,driving off the road or invading the other traffic lane).To solve this issue,we proposed a novel intelligent speed adaptation(ISA)system for visibility,called V-ISA(intelligent speed adaptation for visibility).It estimates the real-time safe speed limits based on the prevailing sight conditions.V-ISA comes with three variants with specific feedback modalities(1)visual and(2)auditory information,and(3)direct intervention to assume control over the vehicle speed.Here,we investigated the efficiency of each of the three V-ISA variants on driving speed choice and lateral behavioural response along road curves with limited and unsafe available sight distances,using a driving simulator.We also considered curve road geometry(curve direction:rightward vs.leftward).Sixty active drivers were recruited for the study.While half of them(experimental group)tested the three V-ISA variants(and a V-ISA off condition),the other half always drove with the V-ISA off(validation group).We used a linear mixed-effect model to evaluate the influence of V-ISA on driver behaviour.All V-ISA variants were efficient at reducing speeds at entrance points,with no discernible negative impact on driver lateral behaviour.On rightward curves,the V-ISA intervening variant appeared to be the most effective at adapting to sight limitations.Results of the current study implies that V-ISA might assist drivers to adjust their operating speed as per prevailing sight conditions and,consequently,establishes safer driving conditions.

Advances in Vision-Based Lane Detection：Algorithms,Integration,Assessment,and Perspectives on ACP-Based Parallel Vision

Lane detection is a fundamental aspect of most current advanced driver assistance systems(ADASs). A large number of existing results focus on the study of vision-based lane detection methods due to the extensive knowledge background and the low-cost of camera devices. In this paper, previous visionbased lane detection studies are reviewed in terms of three aspects, which are lane detection algorithms, integration, and evaluation methods. Next, considering the inevitable limitations that exist in the camera-based lane detection system, the system integration methodologies for constructing more robust detection systems are reviewed and analyzed. The integration methods are further divided into three levels, namely, algorithm, system,and sensor. Algorithm level combines different lane detection algorithms while system level integrates other object detection systems to comprehensively detect lane positions. Sensor level uses multi-modal sensors to build a robust lane recognition system. In view of the complexity of evaluating the detection system, and the lack of common evaluation procedure and uniform metrics in past studies, the existing evaluation methods and metrics are analyzed and classified to propose a better evaluation of the lane detection system. Next, a comparison of representative studies is performed. Finally, a discussion on the limitations of current lane detection systems and the future developing trends toward an Artificial Society, Computational experiment-based parallel lane detection framework is proposed.

Methodical Approach to the Development of a Radar Sensor Model for the Detection of Urban Traffic Participants Using a Virtual Reality Engine

New approaches for testing of autonomous driving functions are using Virtual Reality (VR) to analyze the behavior of automated vehicles in various scenarios. The real time simulation of the environment sensors is still a challenge. In this paper, the conception, development and validation of an automotive radar raw data sensor model is shown. For the implementation, the Unreal VR engine developed by Epic Games is used. The model consists of a sending antenna, a propagation and a receiving antenna model. The microwave field propagation is simulated by a raytracing approach. It uses the method of shooting and bouncing rays to cover the field. A diffused scattering model is implemented to simulate the influence of rough structures on the reflection of rays. To parameterize the model, simple reflectors are used. The validation is done by a comparison of the measured radar patterns of pedestrians and cyclists with simulated values. The outcome is that the developed model shows valid results, even if it still has deficits in the context of performance. It shows that the bouncing of diffuse scattered field can only be done once. This produces inadequacies in some scenarios. In summary, the paper shows a high potential for real time simulation of radar sensors by using ray tracing in a virtual reality.

A surface electromyography controlled steering assistance interface

Purpose–Two-handed automobile steering at low vehicle speeds may lead to reduced steering ability at large steering wheel angles and shoulder injury at high steering wheel rates(SWRs).As afirst step toward solving these problems,this study aims,firstly,to design a surface electromyography(sEMG)controlled steering assistance interface that enables hands-free steering wheel rotation and,secondly,to validate the effect of this rotation on path-following accuracy.Design/methodology/approach–A total of 24 drivers used biceps brachii sEMG signals to control the steering assistance interface at a maximized SWR in three driving simulator scenarios:U-turn,908 turn and 458 turn.For comparison,the scenarios were repeated with a slower SWR and a game steering wheel in place of the steering assistance interface.The path-following accuracy of the steering assistance interface would be validated if it was at least comparable to that of the game steering wheel.Findings–Overall,the steering assistance interface with a maximized SWR was comparable to a game steering wheel.For the U-turn,908 turn and 458 turn,the sEMG-based human–machine interface(HMI)had median lateral errors of 0.55,0.3 and 0.2 m,respectively,whereas the game steering wheel,respectively,had median lateral errors of 0.7,0.4 and 0.3 m.The higher accuracy of the sEMG-based HMI was statistically significant in the case of the U-turn.Originality/value–Although production automobiles do not use sEMG-based HMIs,and few studies have proposed sEMG controlled steering,the results of the current study warrant further development of a sEMG-based HMI for an actual automobile.

Influence of automated driving on driver’s own localization:a driving simulator study

Purpose–Level 3 automated driving,which has been defined by the Society of Automotive Engineers,may cause driver drowsiness or lack of situation awareness,which can make it difficult for the driver to recognize where he/she is.Therefore,the purpose of this study was to conduct an experimental study with a driving simulator to investigate whether automated driving affects the driver’s own localization compared to manual driving.Design/methodology/approach–Seventeen drivers were divided into the automated operation group and manual operation group.Drivers in each group were instructed to travel along the expressway and proceed to the specified destinations.The automated operation group was forced to select a course after receiving a Request to Intervene(RtI)from an automated driving system.Findings–A driver who used the automated operation system tended to not take over the driving operation correctly when a lane change is immediately required after the RtI.Originality/value–This is a fundamental research that examined how the automated driving operation affects the driver's own localization.The experimental results suggest that it is not enough to simply issue an RtI,and it is necessary to tell the driver what kind of circumstances he/she is in and what they should do next through the HMI.This conclusion can be taken into consideration for engineers who design automatic driving vehicles.

Analysis of drivers’characteristic driving operations based on combined features

Purpose–Analysis of characteristic driving operations can help develop supports for drivers with different driving skills.However,the existing knowledge on analysis of driving skills only focuses on single driving operation and cannot reflect the differences on proficiency of coordination of driving operations.Thus,the purpose of this paper is to analyze driving skills from driving coordinating operations.There are two main contributions:the first involves a method for feature extraction based on AdaBoost,which selects features critical for coordinating operations of experienced drivers and inexperienced drivers,and the second involves a generating method for candidate features,called the combined features method,through which two or more different driving operations at the same location are combined into a candidate combined feature.A series of experiments based on driving simulator and specific course with several different curves were carried out,and the result indicated the feasibility of analyzing driving behavior through AdaBoost and the combined features method.Design/methodology/approach–AdaBoost was used to extract features and the combined features method was used to combine two or more different driving operations at the same location.Findings–A series of experiments based on driving simulator and specific course with several different curves were carried out,and the result indicated the feasibility of analyzing driving behavior through AdaBoost and the combined features method.Originality/value–There are two main contributions:the first involves a method for feature extraction based on AdaBoost,which selects features critical for coordinating operations of experienced drivers and inexperienced drivers,and the second involves a generating method for candidate features,called the combined features method,through which two or more different driving operations at the same location are combined into a candidate combined feature.

Evaluation of the Effectiveness of Awareness Messages for Road Traffic Hazards in Experimental Tests

Advanced driver assistance systems, especially autonomous emergency braking and forward collision warnings, have becomepopular in Japan. To reduce the number of road traffic accidents, safety information should be provided to a driver earlier thanavoidance or warning messages so as to avoid a risky situation. A series of actual running tests was conducted to evaluate theactivation timing and effectiveness of awareness messages. Objective analysis showed that the drivers could avoid an obstaclewith a sufficient safety margin thanks to any of the awareness messages. Subjective ratings showed that the best timing is 10 sbefore encountering the obstacle. The results of objective analysis are limited in the present paper, and further analyses arerequired.

Feature mapping and state estimation for highly automated vehicles

The past decade has witnessed an acceleration of autonomous vehicle research and development,with technological advances contributed by academia,government,and the industrial and consumer sectors.These advancements hold the potential to improve society by enhancing transportation safety and throughput,where decreased congestion saves time and reduces vehicle emissions.Two of the key technologies to enable vehicle infrastructure interaction,advanced traffic management,and automated vehicles are automated roadway mapping and reliable vehicle state estimation.In this paper,we present an overview and new methods for the problems automated roadway mapping plus a discussion of the extension of these methods to the problem of vehicle state estimation.Results from the application of these methods to feature mapping and state estimation are presented.

Concept Study of a Self-localization System for Snow-covered Roads Using a Four-layer Laser Scanner

Many advanced driver assistance systems have entered the market,and automated driving technologies have been developed.Many of them may not work in adverse weather conditions.A forward-looking camera,for example,is the most popular system used for lane detection but does not work for a snow-covered road.The present paper proposes a self-localization system for snowy roads when the roadsides are covered with snow.The system employs a four-layer laser scanner and onboard sensors and uses only pre-existing roadside snow poles provided for drivers in a snowy region without any other road infrastructure.Because the landscape greatly changes in a short time during a snowstorm and snow removal works,it is necessary to restrict the landmarks used for self-localization to tall objects,like snow poles.A system incorporating this technology will support a driver’s efforts to keep to a lane even in a heavy snowstorm.

Using naturalistic driving data to identify driving style based on longitudinal driving operation conditions

Purpose–An individual’s driving style significantly affects overall traffic safety.However,driving style is difficult to identify due to temporal and spatial differences and scene heterogeneity of driving behavior data.As such,the study of real-time driving-style identification methods is of great significance for formulating personalized driving strategies,improving traffic safety and reducing fuel consumption.This study aims to establish a driving style recognition framework based on longitudinal driving operation conditions(DOCs)using a machine learning model and natural driving data collected by a vehicle equipped with an advanced driving assistance system(ADAS).Design/methodology/approach–Specifically,a driving style recognition framework based on longitudinal DOCs was established.To train the model,a real-world driving experiment was conducted.First,the driving styles of 44 drivers were preliminarily identified through natural driving data and video data;drivers were categorized through a subjective evaluation as conservative,moderate or aggressive.Then,based on the ADAS driving data,a criterion for extracting longitudinal DOCs was developed.Third,taking the ADAS data from 47 Kms of the two test expressways as the research object,six DOCs were calibrated and the characteristic data sets of the different DOCs were extracted and constructed.Finally,four machine learning classification(MLC)models were used to classify and predict driving style based on the natural driving data.Findings–The results showed that six longitudinal DOCs were calibrated according to the proposed calibration criterion.Cautious drivers undertook the largest proportion of the free cruise condition(FCC),while aggressive drivers primarily undertook the FCC,following steady condition and relative approximation condition.Compared with cautious and moderate drivers,aggressive drivers adopted a smaller time headway(THW)and distance headway(DHW).THW,time-to-collision(TTC)and DHW showed highly significant differences in driving style identification,while longitudinal acceleration(LA)showed no significant difference in driving style identification.Speed and TTC showed no significant difference between moderate and aggressive drivers.In consideration of the cross-validation results and model prediction results,the overall hierarchical prediction performance ranking of the four studied machine learning models under the current sample data set was extreme gradient boosting>multi-layer perceptron>logistic regression>support vector machine.Originality/value–The contribution of this research is to propose a criterion and solution for using longitudinal driving behavior data to label longitudinal DOCs and rapidly identify driving styles based on those DOCs and MLC models.This study provides a reference for real-time online driving style identification in vehicles equipped with onboard data acquisition equipment,such as ADAS.