Project 4

• Hough Transform to detect lane lines

In this, Hough transform for lines has been used to detect lane lines from the above-given video. It consists of the following steps:

• Read Video

This is done using cv2.VideoCapture(). The dimension of the video is found and using the dimensions, a VideoWriter() Object is instantiated.

cap = cv2.VideoCapture(video_path)
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
size = (frame_width, frame_height)
fourcc = cv2.VideoWriter_fourcc(*"mp4v")
out = cv2.VideoWriter('output.mp4', fourcc, 23.976, size, isColor = True)

• Pre-processing

To apply the Hough Transform to the video it is required to iterate over every frame of the video and detect the edge of each frame.

The preprocessing steps are :

• Noise Reduction

Since edge detection algorithms are highly susceptible to noise, each frame of the video needs to be smoothened and noise reduction is to be done. This is done here using a gaussian filter. The function used to perform gaussian filtering is given below:

def gaussian_filter(image: ndarray, filter_size:int, sigma: int) -> ndarray:
    filtered_image = cv2.GaussianBlur(image, (filter_size, filter_size), sigma)
    return filtered_image

The cv2.GaussianBlur() has been used to convolve the gaussian filter across all the frames of the video.

• Edge Detection

The Edge detection has been done here using the Canny Edge detection algorithm. The function used is given below:

def canny_edge_detector(image: ndarray, low_threshold: int = 50, high_threshold: int = 150) -> ndarray:
    canny_edge_detected_img = cv2.Canny(image, low_threshold, high_threshold)
    return canny_edge_detected_img

Here, cv2.Canny() has been used to generate the edges for each frame.

• Masking

Since Hough Transform is a computationally expensive operation, masking has been applied to all the frames of the video to reduce the number of Hough lines computed. The masking is done by assuming a polygon that segregates the lanes from the rest of the frame. The mask was constructed using the cv2.polyfill() function. And the masked was overlayed on the frame using the cv2.bitwise_and() function. The function used to return the masked image is given below:

def ROI_masking(image: ndarray) -> ndarray:
    height, width = image.shape[:2]
    image = np.array(image, dtype=np.uint8)
    coordinates = np.array([(0, height), (0, round(3*height/5)), (round(2*width/5), round(55*height/100)), (round(4*width/5), round(55*height/100)), (width, round(3*height/5)), (width, height)])
    mask = np.zeros((height, width))
    masked_image = cv2.fillPoly(mask, [coordinates], 255)
    masked_image = np.array(masked_image, dtype=np.uint8)
    masked_image = cv2.bitwise_and(image, masked_image)
    return masked_image

• Hough transform

To detect lanes, Hough Transform for lines has been used to detect the lines for which the accumulator exceeds the threshold. The function used to perform hough transform on each frame and draw the lines has been given below:

def apply_hough_transform(edge_image: ndarray, org_img:ndarray, hough_threshold:int = 150) -> ndarray:
    lines = hough_transform(edge_image, hough_threshold)
    line_image = np.zeros_like(org_img)
    if lines is not None:
        for i in range(0, len(lines)):
            rho = lines[i][0][0]
            theta = lines[i][0][1]
            a = math.cos(theta)
            b = math.sin(theta)
            x0 = a * rho
            y0 = b * rho
            pt1 = (int(x0 + 1000*(-b)), int(y0 + 1000*(a)))
            pt2 = (int(x0 - 1000*(-b)), int(y0 - 1000*(a)))
            cv2.line(line_image, pt1, pt2, (0,0,255), 3, cv2.LINE_AA)
    imposed_img = cv2.addWeighted(line_image, 0.8, org_img, 1.0, 1.0)
    return imposed_img

The masked edge image from the pre-processing step is given as input to this function which is used to detect Hough lines. The original frame is also passed to function to superimpose the lines detected by the Hough transform onto the frame.

In this, the Hough lines have been detected using the function hough_transform(edge_image, hough_threshold). The function is given below:

def hough_transform(image: ndarray, threshold:int, pixel_resolution:int = 1, angle_resolution: float = np.pi / 180):
    lines = cv2.HoughLines(image, pixel_resolution, angle_resolution, threshold)
    return lines

An empty array having the same shape as the image is declared. The lines detected using the above function are used to determine the coordinates for drawing the line. The lines are drawn using cv2.line() function. The line image is then superimposed onto the original frame and returned.

The superimposed image which contains the Hough lines on the original frames is written to the cv2.VideoWriter() object using the write() method.

Once all the frames are done, the cv2.VideoWriter() and cv2.VideoCapture() objects are released and all frames are destroyed using the cv2.destroyAllWindows().

• Output

The video attached below shows that the Hough lines have been detected and superimposed on the original video feed.
The key takeaway is that for all the lanes in all the frames of the video to be detected, the threshold for the Hough transform needs to be lowered. The downside to this is that not just the lane lines, but also the imperfections in the road are also recognised as edges. This is considered a false positive to the requirement of the task and a better ML-based approach is required to keep the false positives to a minimum.

Output(alternative link) - https://drive.google.com/drive/folders/1hLg8g3amtBlVQz4RCgb3KV63GSjYPc_F?usp=sharing