This tutorial shows a method for detecting lane markers and the road vanishing point from driving footage.
Hayk Martirosyan, Brady Von Quist
February 23, 2015
Import OpenCV, Numpy, and Matplotlib.
In [1]:
import cv2
import numpy as np
import matplotlib.pyplot as plt
Enable inline plotting.
In [2]:
%pylab inline
Get the path to our example image.
In [3]:
import os
img_filename = os.path.join('..', '..', 'data', 'Stereo', 'Color_cam1', '0000000000.png')
Read in the image as grayscale.
In [4]:
img = cv2.imread(img_filename, cv2.IMREAD_GRAYSCALE)
plt.imshow(img, 'gray');
We will be detecting the lane and the road vanishing point in this image.
The first step is to apply a filter that makes lane markings stand out. This method is adapted from Marcos Nieto, from this post:
First, we define the parameter tau as the approximate X width of the lane markers in our image.
In [5]:
tau = 15 # Lane marker is about 15 pixels wide
Convert the image to float representation, and cut off the top third of the image, with the assumption that it is above the horizon line.
In [6]:
img_float = img.astype(float)
cutoff_min_y = int(img.shape[0] * 1/3)
img_float[0:cutoff_min_y, :] = 0
Create copies of the image shifted left and right by tau.
In [7]:
img_left = img_float[:, 2*tau:]
img_right = img_float[:, :-2*tau]
Apply the filter to get img_out.
In [8]:
img_out = np.zeros(img_float.shape)
img_out[:, tau:-tau] = img_float[:, tau:-tau] * 2
img_out[:, tau:-tau] -= img_left + img_right
img_out[:, tau:-tau] -= np.abs(img_left - img_right)
The filter highlights regions that are approximately tau in width.
In [9]:
plt.imshow(img_out, 'gray');
Convert the image back to unit8 representation and apply thresholding to get a binary image. We use Otsu thresholding to automatically choose the threshold value.
In [20]:
img_filter = np.clip(img_out, 0, 255)
img_filter = np.uint8(img_filter)
_, img_filter = cv2.threshold(img_filter, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
In [21]:
plt.imshow(img_filter, 'gray');
Finally, we apply erosion with a small elliptical kernel to get rid of the smaller noisy features of the image. This step is not adapted from the link above, but helps clean up the image for line detection.
In [24]:
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
img_thresh = cv2.erode(img_filter, kernel=kernel)
In [25]:
plt.imshow(img_thresh, 'gray');
We use a probabilistic Hough transform to detect lines in this image. We use fine values for rho and theta to ensure accuracy. The minLineLength and maxLineGap parameters are very useful for detecting the lane marker lines, especially the center line which can be dashed.
In [33]:
lines = cv2.HoughLinesP(
image=img_thresh,
rho=1,
theta=np.pi*1/180,
threshold=80,
minLineLength=170,
maxLineGap=120
)
Create a copy of the image and plot the detected line segments on top of it.
In [34]:
img_lines = np.array(img)
img_lines = cv2.cvtColor(img_lines, cv2.COLOR_GRAY2RGB)
lines = [line[0] for line in lines]
for line in lines:
x1, y1, x2, y2 = line
cv2.line(img_lines, (x1, y1), (x2, y2), (255, 255, 0), 2)
plt.imshow(img_lines);
From these lines, we will apply RANSAC to approximate the vanishing point where the lane marker lines must intersect. RANSAC will randomly sample two lines, calculate their intersection point, and check how many other lines come within some distance of the intersection point. It will take the point that came closest to the most lines as the vanishing point
In this example, 20 iterations and 50 pixel difference will work well.
In [35]:
iterations = 20
distance = 50
In [36]:
# Number of lines
N = len(lines)
# Maximum number of consistant lines
max_num_consistent_lines = 0
# Best fit point
best_fit = None
# Loop through all of the iterations to find the most consistent value
for i in range(0, iterations):
# Randomly choosing the lines
random_indices = np.random.choice(N, 2, replace=False)
i1 = random_indices[0]
i2 = random_indices[1]
x1, y1, x2, y2 = lines[i1]
x3, y3, x4, y4 = lines[i2]
# Find the intersection point
x_intersect = ((x1*y2-y1*x2)*(x3-x4)-(x1-x2)*(x3*y4-y3*x4))/((x1-x2)*(y3-y4)-(y1-y2)*(x3-x4))
y_intersect = ((x1*y2-y1*x2)*(y3-y4)-(y1-y2)*(x3*y4-y3*x4))/((x1-x2)*(y3-y4)-(y1-y2)*(x3-x4))
this_num_consistent_lines = 0
# Find the distance between the intersection and all of the other lines
for i2 in range(0, N):
tx1, ty1, tx2, ty2 = lines[i2]
this_distance = (np.abs((ty2-ty1)*x_intersect - (tx2-tx1)*y_intersect + tx2*ty1 - ty2*tx1)
/ np.sqrt((ty2-ty1)**2 + (tx2-tx1)**2))
if this_distance < distance:
this_num_consistent_lines += 1
# If it's greater, make this the new x, y intersect
if this_num_consistent_lines > max_num_consistent_lines:
best_fit = int(x_intersect), int(y_intersect)
max_num_consistent_lines = this_num_consistent_lines
vanishing_point = best_fit
Plot the vanishing point as a large red dot.
In [38]:
cv2.circle(img_lines, vanishing_point, 1, (255, 0, 0), 15)
plt.imshow(img_lines);
Now that we have the vanishing point, we can use the lines identified previously to find the boundaries of the lane and the location of the lane markers.
TODO
In [54]:
plt.figure(), plt.imshow(img_filter, 'gray'), plt.title('Line Marker Filter');
plt.figure(), plt.imshow(img_lines), plt.title('Detected lines and vanishing point');
In [ ]: