A Review of Lane Detection Based on Deep Learning Methods

Lane detection is animportant aspect of autonomous driving,aiming to ensure that vehicles accurately understand road structures as well as improve their ability to drive in complex traffic environments.In recent years,lane detection tasks based on deep learning methods have made significant progress in detection accuracy.In this paper,we provide a comprehensive review of deep learning-based lane detection tasks in recent years.First,we introduce the background of the lane detection task,including lane detection,the lane datasets and the factors affecting lane detection.Second,we review the traditional and deep learning methods for lane detection,and analyze their features in detail while classifying the different methods.In the deep learning methods classification section,we explore five main categories,including segmentation-based,object detection,parametric curves,end-to-end,and keypoint-based methods.Then,some typical models are briefly compared and analyzed.Finally,in this paper,based on the comprehensive consideration of current lane detection methods,we put forward the current problems still faced,such as model generalization and computational cost.At the same time,possible future research directions are given for extreme scenarios,model generalization and other issues.

Advances in Vision-Based Lane Detection：Algorithms,Integration,Assessment,and Perspectives on ACP-Based Parallel Vision

Lane detection is a fundamental aspect of most current advanced driver assistance systems(ADASs). A large number of existing results focus on the study of vision-based lane detection methods due to the extensive knowledge background and the low-cost of camera devices. In this paper, previous visionbased lane detection studies are reviewed in terms of three aspects, which are lane detection algorithms, integration, and evaluation methods. Next, considering the inevitable limitations that exist in the camera-based lane detection system, the system integration methodologies for constructing more robust detection systems are reviewed and analyzed. The integration methods are further divided into three levels, namely, algorithm, system,and sensor. Algorithm level combines different lane detection algorithms while system level integrates other object detection systems to comprehensively detect lane positions. Sensor level uses multi-modal sensors to build a robust lane recognition system. In view of the complexity of evaluating the detection system, and the lack of common evaluation procedure and uniform metrics in past studies, the existing evaluation methods and metrics are analyzed and classified to propose a better evaluation of the lane detection system. Next, a comparison of representative studies is performed. Finally, a discussion on the limitations of current lane detection systems and the future developing trends toward an Artificial Society, Computational experiment-based parallel lane detection framework is proposed.

A Lane Detection Method Based on Semantic Segmentation

This paper proposes a novel method of lane detection,which adopts VGG16 as the basis of convolutional neural network to extract lane line features by cavity convolution,wherein the lane lines are divided into dotted lines and solid lines.Expanding the field of experience through hollow convolution,the full connection layer of the network is discarded,the last largest pooling layer of the VGG16 network is removed,and the processing of the last three convolution layers is replaced by hole convolution.At the same time,CNN adopts the encoder and decoder structure mode,and uses the index function of the maximum pooling layer in the decoder part to upsample the encoder in a counter-pooling manner,realizing semantic segmentation.And combined with the instance segmentation,and finally through the fitting to achieve the detection of the lane line.In addition,the currently disclosed lane line data sets are relatively small,and there is no distinction between lane solid lines and dashed lines.To this end,our work made a lane line data set for the lane virtual and real identification,and based on the proposed algorithm effective verification of the data set achieved by the increased segmentation.The final test shows that the proposed method has a good balance between lane detection speed and accuracy,which has good robustness.

LDNet:structure-focused lane detection based on line deformation

Lane detection is a fundamental necessary task for autonomous driving.The conventional methods mainly treat lane detection as a pixel-wise segmentation problem,which suffers from the challenge of uncontrollable driving road environments and needs post-processing to abstract the lane parameters.In this work,a series of lines are used to represent traffic lanes and a novel line deformation network(LDNet) is proposed to directly predict the coordinates of lane line points.Inspired by the dynamic behavior of classic snake algorithms,LDNet uses a neural network to iteratively deform an initial lane line to match the lane markings.To capture the long and discontinuous structures of lane lines,1 D convolution in LDNet is used for structured feature learning along the lane lines.Based on LDNet,a two-stage pipeline is developed for lane marking detection:(1) initial lane line proposal to predict a list of lane line candidates,and(2) lane line deformation to obtain the coordinates of lane line points.Experiments show that the proposed approach achieves competitive performances on the TuSimple dataset while being efficient for real-time applications on a GTX 1650 GPU.In particular,the accuracy of LDNet with the annotated starting and ending points is up to99.45%,which indicates the improved initial lane line proposal method can further enhance the performance of LDNet.

In-Vehicle Stereo Vision Systems with Improved Ant Colony Optimization Based Lane Detection: A Solution to Accidents Involving Large Goods Vehicles Due to Blind Spots

This paper presents an in-vehicle stereo vision system as a solution to accidents involving large good vehicle due to blind spots using Nigeria as a case study. In this paper, a stereo-vision system was attached to the front of Large Good Vehicles (LGVs) with a view to presenting live feeds of vehicles close to the LGV vehicles and their distance away. The captured road images using the stereo vision system were optimized for effectiveness and optimal vehicle maneuvering using a modified metaheuristics algorithm called the simulated annealing Ant Colony Optimization (saACO) algorithm. The concept of simulated annealing is strategies used to automatically select the control parameters of the ACO algorithm. This helps to stabilize the performance of the ACO algorithm irrespective of the quality of the lane images captured in the in-vehicle vision system. The system is capable of notifying drivers through lane detection techniques of blind spots. This technique enables the driver to be more aware of what surrounds the vehicle and make decisions early. In order to test the system, the stereo-vision device was mounted on a Large good vehicle, driven in Zaria (a city in Kaduna state in Nigeria), and data were in the record. Out of 180 events, 42.22% of potential accident events were caused by Passenger Cars, while 27.22%, 18.33% and 12.22% were caused by two-wheelers, Large Good Vehicles and road users, respectively. In the same vein, the in-vehicle lane detection system shows a good performance of the saACO-based lane detection system and gives a better performance in comparison with the standard ACO method.

A Novel Ego Lanes Detection Method for Autonomous Vehicles

Autonomous vehicles are currently regarded as an interesting topic in the AI field.For such vehicles,the lane where they are traveling should be detected.Most lane detection methods identify the whole road area with all the lanes built on it.In addition to having a low accuracy rate and slow processing time,these methods require costly hardware and training datasets,and they fail under critical conditions.In this study,a novel detection algo-rithm for a lane where a car is currently traveling is proposed by combining simple traditional image processing with lightweight machine learning(ML)methods.First,a preparation phase removes all unwanted information to preserve the topographical representations of virtual edges within a one-pixel width around expected lanes.Then,a simple feature extraction phase obtains only the intersection point position and angle degree of each candidate edge.Subsequently,a proposed scheme that comprises consecutive lightweight ML models is applied to detect the correct lane by using the extracted features.This scheme is based on the density-based spatial clustering of applications with noise,random forest trees,a neural network,and rule-based methods.To increase accuracy and reduce processing time,each model supports the next one during detection.When a model detects a lane,the subsequent models are skipped.The models are trained on the Karlsruhe Institute of Technology and Toyota Technological Institute datasets.Results show that the proposed method is faster and achieves higher accuracy than state-of-the-art methods.This method is simple,can handle degradation conditions,and requires low-cost hardware and training datasets.

Lane Line Detection Based on Improved PINet

Accurate perception of lane line information is one of the basic requirements of unmanned driving technology, which is related to the localization of the vehicle and the determination of the forward direction. In this paper, multi-level constraints are added to the lane line detection model PINet, which is used to improve the perception of lane lines. Predicted lane lines in the network are predicted to have real and imaginary attributes, which are used to enhance the perception of features around the lane lines, with pixel-level constraints on the lane lines;images are converted to bird’s-eye views, where the parallelism between lane lines is reconstructed, with lane line-level constraints on the predicted lane lines;and vanishing points are used to focus on the image hierarchy, with image-level constraints on the lane lines. The model proposed in this paper meets both accuracy (96.44%) and real-time (30 + FPS) requirements, has been tested on the highway on the ground, and has performed stably.

A self-organization formation configuration based assignment probability and collision detection

The formation control of multiple unmanned aerial vehicles(multi-UAVs)has always been a research hotspot.Based on the straight line trajectory,a multi-UAVs target point assignment algorithm based on the assignment probability is proposed to achieve the shortest overall formation path of multi-UAVs with low complexity and reduce the energy consumption.In order to avoid the collision between UAVs in the formation process,the concept of safety ball is introduced,and the collision detection based on continuous motion of two time slots and the lane occupation detection after motion is proposed to avoid collision between UAVs.Based on the idea of game theory,a method of UAV motion form setting based on the maximization of interests is proposed,including the maximization of self-interest and the maximization of formation interest is proposed,so that multi-UAVs can complete the formation task quickly and reasonably with the linear trajectory assigned in advance.Finally,through simulation verification,the multi-UAVs target assignment algorithm based on the assignment probability proposed in this paper can effectively reduce the total path length,and the UAV motion selection method based on the maximization interests can effectively complete the task formation.

Lane Detection:A Survey with New Results

Lane detection is essential for many aspects of autonomous driving,such as lane-based navigation and high-definition(HD)map modeling.Although lane detection is challenging especially with complex road conditions,considerable progress has been witnessed in this area in the past several years.In this survey,we review recent visual-based lane detection datasets and methods.For datasets,we categorize them by annotations,provide detailed descriptions for each category,and show comparisons among them.For methods,we focus on methods based on deep learning and organize them in terms of their detection targets.Moreover,we introduce a new dataset with more detailed annotations for HD map modeling,a new direction for lane detection that is applicable to autonomous driving in complex road conditions,a deep neural network LineNet for lane detection,and show its application to HD map modeling.

LANE MARKING DETECTION IN CLUTTERED ENVIRONMENT

Objective To determine the positions of marking in the presence of distracting shadows, highlight, pavement cracks, etc. Methods RGB color space is transformed into I 1 I 2 I 3 color space and I 2 component was used to form a new image with less effect of the clutter. Using an improved edge detection operator, an edge strength map was produced, and binarilized by adaptive thresholds. The binary image was labeled and circularity of all connected components is calculated. The Self Organizing Mapping is adopted to extract regions which imply potential marking. Finally the position of marking was obtained by curve fitting. Results Color information was utilized fully, all thresholds were set adaptively and lane marking could be detected in challenging images with shadows, highlight or other cars. Conclusion The method based on circularity of connected components shows its outstanding robustness to lane marking detection and has a wide variety of applications in the areas of vehicle autonomous navigation and driver assistance system.

The analysis of lane detection algorithms using histogram shapes and Hough transform

Purpose–The purpose of this paper is to develop a lane detection analysis algorithm by Hough transform and histogram shapes,which can effectively detect the lane markers in various lane road conditions,in driving system for drivers.Design/methodology/approach–Step 1:receiving image:the developed system is able to acquire images from video files.Step 2:splitting image:the system analyzes the splitting process of video file.Step 3:cropping image:specifying the area of interest using crop tool.Step 4:image enhancement:the system conducts the frame to convert RGB color image into grayscale image.Step 5:converting grayscale image to binary image.Step 6:segmenting and removing objects:using the opening morphological operations.Step 7:defining the analyzed area within the image using the Hough transform.Step 8:computing Houghline transform:the system operates the defined segment to analyze the Houghline transform.Findings–This paper presents the useful solution for lane detection by analyzing histogram shapes and Hough transform algorithms through digital image processing.The method has tested on video sequences filmed by using a webcam camera to record the road as a video file in a form of avi.The experimental results show the combination of two algorithms to compare the similarities and differences between histogram and Hough transform algorithm for better lane detection results.The performance of the Hough transform is better than the histogram shapes.Originality/value–This paper proposed two algorithms by comparing the similarities and differences between histogram shapes and Hough transform algorithm.The concept of this paper is to analyze between algorithms,provide a process of lane detection and search for the algorithm that has the better lane detection results.

Robust lane recognition for structured road based on monocular vision

A robust lane detection and tracking system based on monocular vision is presented in this paper. First, the lane detection algorithm can transform raw images into top view images by inverse perspective mapping （ IPM）, and detect both inner sides of the lane accurately from the top view im- ages. Then the system will turn to lane tracking procedures to extract the lane according to the infor- mation of last frame. If it fails to track the lane, lane detection will be triggered again until the true lane is found. In this system, 0-oriented Hough transform is applied to extract candidate lane mark- ers, and a geometrical analysis of the lane candidates is proposed to remove the outliers. Additional- ly, vanishing point and region of interest（ROI） dynamically planning are used to enhance the accura- cy and efficiency. The system was tested under various road conditions, and the result turned out to be robust and reliable.

A MODEL-ORIENTED ROAD DETECTION APPROACH USING FUZZY SVM

This paper presents an approach of model-oriented road detection based on trapezoidal model proposed by H. Jeong, et al and fuzzy Support Vector Machine (SVM). Firstly, the frames ex-tracted from the video are preprocessed by Pulse Coupled Neural Network (PCNN), and then handled by Kalman filter and Expectation Maximization (EM) algorithms. Next, according to the road's dif-ferent feathers, using fuzzy algorithm chooses a corresponding SVM for further lane detection, and then using morphological filters obtains the final detecting result. For different types of roads, this method uses fuzzy algorithm to choose different SVMs. Furthermore, in preprocessing using PCNN removes the shadow in the road to reduce the effect of illumination variations. Experimental results show that our method can receive better lane detecting results than the trapezoidal model and BP proposed by H. Jeong, et al..

ST-LaneNet: Lane Line Detection Method Based on Swin Transformer and LaneNet

The advancement of autonomous driving heavily relies on the ability to accurate lane lines detection.As deep learning and computer vision technologies evolve,a variety of deep learning-based methods for lane line detection have been proposed by researchers in the field.However,owing to the simple appearance of lane lines and the lack of distinctive features,it is easy for other objects with similar local appearances to interfere with the process of detecting lane lines.The precision of lane line detection is limited by the unpredictable quantity and diversity of lane lines.To address the aforementioned challenges,we propose a novel deep learning approach for lane line detection.This method leverages the Swin Transformer in conjunction with LaneNet(called ST-LaneNet).The experience results showed that the true positive detection rate can reach 97.53%for easy lanes and 96.83%for difficult lanes(such as scenes with severe occlusion and extreme lighting conditions),which can better accomplish the objective of detecting lane lines.In 1000 detection samples,the average detection accuracy can reach 97.83%,the average inference time per image can reach 17.8 ms,and the average number of frames per second can reach 64.8 Hz.The programming scripts and associated models for this project can be accessed openly at the following GitHub repository:https://github.com/Duane 711/Lane-line-detec tion-ST-LaneNet.

LaneDraw: Cascaded lane and its bifurcation detection with nested fusion

Lane and its bifurcation detection is a vital and active research topic in low cost camera-based autonomous driving and advanced driver assistance system(ADAS). The common lane detection pipeline usually predicts lane segmentation mask firstly, and then makes line fitting by parabola or spline post-processing. However, if the speed of the lane and its bifurcation detection is fast and robust enough, we think curve fitting is not a necessary step. The goal of this work is to get accurate lane segmentation,identification of every lane, adaptability of lane numbers and the right combination of lane bifurcation. In this work, we relabeled lane and its bifurcation with solid line if the image of Tu Simple dataset has both of them. In the data training process, we apply a data balance strategy for the heavily biased lane and non-lane data. In such a way, we develop a competitive cascaded instance lane detection model and propose a novel bifurcation pixel embedding nested fusion method based on full binary segmentation pixel embedding with self-grouping cluster, called Lane Draw. Our method discards curve fitting process, therefore it reduces the complexity of post-processing and increases detection speed at 35 fps. Moreover, the proposed method yields better performance and high accuracy on the relabeled Tu Simple dataset. To the best of our knowledge, this is the first attempt in 2 D lane and bifurcation detection, which more often happens in actual situations.

Lane departure warning systems and lane line detection methods based on image processing and semantic segmentation:A review

Recently,the development and application of lane line departure warning systems have been in the market.For any of the systems,the key part of lane line tracking,lane line identification,or lane line departure warning is whether it can accurately and quickly detect lane lines.Since 1990 s,they have been studied and implemented for the situations defined by the good viewing conditions and the clear lane markings on road.After then,the accuracy for particular situations,the robustness for a wide range of scenarios,time efficiency and integration into higher-order tasks define visual lane line detection and tracking as a continuing research subject.At present,these kinds of lane marking line detection methods based on machine vision and image processing can be divided into two categories:the traditional image processing and semantic segmentation(includes deep learning)methods.The former mainly involves feature-based and model-based steps,and which can be classified into similarity-and discontinuity-based ones;and the model-based step includes different parametric straight line,curve or pattern models.The semantic segmentation includes different machine learning,neural network and deep learning methods,which is the new trend for the research and application of lane line departure warning systems.This paper describes and analyzes the lane line departure warning systems,image processing algorithms and semantic segmentation methods for lane line detection.

YOLOP:You Only Look Once for Panoptic Driving Perception

A panoptic driving perception system is an essential part of autonomous driving.A high-precision and real-time perception system can assist the vehicle in making reasonable decisions while driving.We present a panoptic driving perception network(you only look once for panoptic(YOLOP))to perform traffic object detection,drivable area segmentation,and lane detection simultaneously.It is composed of one encoder for feature extraction and three decoders to handle the specific tasks.Our model performs extremely well on the challenging BDD100K dataset,achieving state-of-the-art on all three tasks in terms of accuracy and speed.Besides,we verify the effectiveness of our multi-task learning model for joint training via ablative studies.To our best knowledge,this is the first work that can process these three visual perception tasks simultaneously in real-time on an embedded device Jetson TX2(23 FPS),and maintain excellent accuracy.To facilitate further research,the source codes and pre-trained models are released at https://github.com/hustvl/YOLOP.