Applying CV for Autonomous Vehicles using Python

This 3 month program from Skill-Lync teaches the student everything there is to know about computer vision. MATLAB will be used as a tool.

Enroll Now View demo

A Quick Overview

As of 2020, Google has more than 4 trillion photos stored in its data centers. However, when we search for a picture of Skill Lync, the results are surprisingly accurate each time! 

These accurate results within seconds have been made possible by Computer Vision. Computer Vision is a technology that is working towards giving cameras the ability to see things, detect them and make decisions accordingly, similar to what the human eye does. 

Computer Vision has seen it’s applications in the field of medicine, business,  agriculture, law enforcement etc. When we talk about it’s application in Autonomous Vehicle, Computer vision enables the vehicle to: 

  • Detect objects on the road, such as pedestrians, traffic lights and vehicles. 
  • Create situational awareness for the vehicle depending upon road conditions
  • Low light detection capabilities
  • Create a map based on what it sees 

These few aspects make computer vision the backbone of Autonomous Vehicle. Keeping this in mind, we at Skill lync have curated a 12 week program that is entirely detected towards teaching students computer vision applied to autonomous vehicle. 

The course also includes two practical sessions per week and two major projects to help the student get a better understanding of the subject. To help you know more about what we plan on covering each week, a descriptive syllabus is given below. 


GET COURSE COUNSELLING TODAY

Get a 1-on-1 demo to understand what is included in the course and how it can benefit you from an experienced sales consultant. The demo session will help you enroll in this course with a clear vision and confidence.

Request a Demo Session

COURSE SYLLABUS

1Introduction to Computer Vision

  • Introduction to Autonomous Vehicles
  • Introduction to Computer Vision
  • Applications of Computer Vision
  • Course Content - Introduction 
  • Understanding Images 

2Image Processing Techniques – I

  • Image Filters 
  • Correlation 
  • Convolution
  • Noise in Images
  • Types of Noise
  • Filters for Noise
  • Image Gradients 
  • Edge Detection Techniques

3Image Processing . Edge and Line Detection

  • Canny Edge Detector 
  • Hough Transformation - Lines
  • Hough Transformation - Circles 
  • Domain Transformation 
  • Understanding Frequency Domain 
  • Spatial to Frequency Domain transformations

4Projective and Stereo Geometry

  • Image coordinate systems 
  • Projective Geometry
  • Perspective Projections
  • Multiview Geometry 
  • Stereography and Depth Imaging 
  • Stereo Correspondence

53D Computer Vision

  • Projective Geometry for 3D
  • Camera Calibration methods 
  • Epipolar Geometry
  • Stereovision

6Feature Extraction , Neural Networks and Image Classification

  • Image Classification
  • Dimensionality Reduction
  • Principal Component Analysis
  • Convolutional Neural Networks 
  • Datasets
  • Mobile Net Architecture
  • Image Classification - Performance Metrics

7Feature Detectors and Descriptors

  • Feature Detectors
    1. Moravec’s Detector
    2. Harris Corner Detector
  • Feature Descriptors
    1. SIFT 
    2. ORB
  • Feature matching methods

8Optical Flow

  • Optical Flow
  • Horn-Shunck method
  • Lucas Kanade sparse optical flow 
  • Gunnar-Farneback Optical flow 
  • Deep learning based Optical flow models 

9Object Tracking

  • Introduction to Object Tracking 
  • Deep SORT
  • Lucas-Kanade-Tomasi(KLT) Tracker 
  • Minimum Of Sum Squared Error (MOSSE) Tracker
  • Mean Shift Tracker  

10Image Segmentation

  • Introduction to Image Segmentation
  • Methods of Segmentation
  • Applications of Segmentation
  • Thresholding based segmentation
  • Otsu’s Thresholding
  • Morphological Operations
  • Connected Components
  • Datasets for Image segmentation
  • Deep learning architectures for Image segmentation

11Object Detection

  • Introduction to Object Detection
  • Region Proposals
  • Graph cut segmentation
  • Selective search
  • Object Detection Datasets
  • Object Detection Models
  • Tensorflow model zoo
  • Evaluation metrics for object detection models 

123D Object Detection

  • Introduction to 3D Object Detection
  • Types of 3D Object Detection
  • Stereo Image based Detection
  • Monocular 3D Object Detection


Projects Overview

Project 1

Highlights

Learning Objectives 

Learn basic OpenCV and NumPy functions to load, resize and perform basic image manipulations on the images  

Description:

  1. Basic image manipulations such as reading, displaying, resizing, cropping and translation will be explained 
  2. A recording of scripting of the codes for the above-mentioned functions will be provided 

Task 

  1. Script a function to perform image rotation by deriving the rotation matrix and performing affine transformations using OpenCV functions
  2. Submit the rotated image 

Project 2

Highlights

Learning Objectives 

Gaining hands on experience in applying noise removal, edge detection and other filtering techniques. 

Description

  1. Following content will be covered in the coding tutorial 
    1. Salt and Pepper noise generation
    2. Gaussian smoothing 
    3. Mean filtering 
    4. Median filtering 
    5. Sobel edge filter

Task 

  1. Script two functions to perform edge detection on a given image using Prewitt filter (first function/method) and Laplacian filter (second function/method)
  2. Submit the filtered images (one from each function) 

 

Project 3

Highlights

Learning Objectives

Understand and gain hands on experience on the working of Canny edge detector  

Description

  1. Complete coding implementation of Canny edge detector will be done in the recorded coding tutorial. The methods/functions covered are
    1. Gaussian filtering 
    2. Sobel edge filter for edge detection
    3. Non maximum suppression 
    4. Double thresholding 
    5. Edge Tracking – Hysteresis 

Task 

  1. Perform Canny edge detection on a custom image – Students can choose any image 
  2. Submit one processed image for each stage of canny edge detection. The stages are listed below 
    1. Gaussian filtering
    2. Sobel filter
    3. Non maximum suppression
    4. Double thresholding 
    5. Edge tracking 

Note: Perform Gaussian filtering on an image and submit the processed. Repeat the process for all stages of Canny edge detection. 

Project 4

Highlights

Learning Objectives 

Understand the processes involved in lane detection using Hough Transformation for line

Description

  1. Lane detection along with the steps involved in it will be explained in the coding tutorial
  2. Concepts covered are listed below 
    1. Edge detection
    2. Region of Interest extraction
    3. Hough transformation
    4. Slope and intercept derivation
    5. Drawing lines on images 

Task 

Choose a custom recording video of lane markings from the front of a car. Extract images from the video sequence. Perform lane detection on it. 

Project 5

Highlights

Learning Objectives 

Understand and gain hands on experience in measuring depth of an object using stereo images

Description

  1. The concepts covered in the coding tutorial are listed below 
    1. Generating Homography matrix for two array of points – This can be readily extended to two images with known feature points 
    2. Constructing a disparity map of an object using images taken from two angles

Task 

  1. Construct a disparity map from two custom images. Students shall click these images from their own camera (ex: phone’s camera)
  2. Submit the images taken from the camera (1st and 2nd position) and the corresponding disparity map image 

Project 6

Highlights

Learning Objectives 

Understand and gain hands on experience of the working of a dimensionality reduction method – Principal Component Analysis 

Description

  1. Coding of steps involved in finding the principal components of a dataset will be shown. The following concepts will be covered
    1. Normalization
    2. Covariance matrix estimation
    3. Eigen value and vectors 
    4. Matrix Projection  

Mini Project

Highlights

Learning Objectives 

Understand and implement an image classification model using MobileNet architecture

Description

  1. Recording of image classification network coding will be provided. The concepts covered are as follows
    1. Network Architecture 
    2. Data processing steps 
    3. Training procedure 
    4. Running Inference

Task 

  1. Train the same image classification with a custom dataset provided 
  2. Report out the results with the metrics discussed in Week 6’s lecture

Project 7

Highlights

Learning Objectives 

Understand the working of a feature detector and descriptor. Understand it to perform image stitching to develop surround view perception. 

Description

  1. Perform image stitching on two images taken from two different positions of the rear of the car. 
  2. Images were taken from the rear of the car and can be used to provide surround view perception for a vehicle. 
  3. The feature detector and descriptor used are SIFT and K-Nearest Neighbor feature matching is used to match the features. 

Task 

  1. Take images from different positions of the car to perform image stitching. 
  2. Any feature detector, descriptor and feature matching method can be used perform the task.  

Project 8

Highlights

Learning Objectives 

Perform Optical flow on a sequence of images in a video

Description

  1.  Use Lucas Kanade Optical Flow method to perform sparse Optical Flow for pixels of interest. 
  2. Pixels or features of interest are derived from Shi-Tomasi corner detector algorithm 

Task  

  1. Choose any video sequence of moving vehicles
  2. Perform Optical flow following the same method mentioned in the code sample provided in the coding lecture. 
  3. Submit the processed video along with its masked video sequence to the portal 

Project 9

Highlights

Learning Objectives 

Understand Object tracking along with Object ID assignment. The Deep SORT Algorithm taught in the lecture will be used to explain these concepts. 

Description

  1. Explain the working of Deep SORT object tracking algorithm
  2. Choose a video sequence with hand annotated bounding box for objects of Interest 
  3. Track the objects over a sequence of frames with the Deep SORT tracking algorithm 

Task 

  1. Choose a custom video sequence to perform tracking on the images 
  2. Initiate the sequence with hand drawn bounding boxes 
  3. Assign ID and track the objects of Interest 
  4. Submit the processed video to the portal 

 

Project 10

Highlights

Dataset training using MaskRCNN architecture, for the purpose of image segmentation and object recognition/tracking.

Learning outcomes:

  • Understanding the implementation of a deep neural network algorithm in detecting various objects in an image. 
  • Figuring out how confident the algorithm is on the detected object 
  • Adding a mask over the specified object which helps differentiate it from the other objects in the image.

Problem statement:

  • Train the given dataset using MaskRCNN architecture, and detect the various objects given in an image, along with masks, bounding boxes and confidence levels.
  • Your code should call the MaskRCNN architecture training codes (model.py)  and train the datasets. Then call the data file which has the names of various objects and assign ids to it.
  • You should then implement the trained dataset in object detection (using weights subclass in model.py)
  • The above implementation gives you the data for bounding boxes, object names, confidence levels, masks. Visualise the same outputs in the form of an image and display the image using matplotlb or imshow (you need to generate a code on your own for this, don’t use visualize.py code in the MaskRCNN architecture)

Deliverables:

  • The code which implements the above said problem statement.
  • A report on how your implementation works in detail.
  • The input image which you feed in for the object detection process.
  • The output image which has the objects detected along with their names, bounding boxes, confidence levels and masks.

Note:

The files and the architecture needed for this assignment will be shared with you in the form of a drive link. You need your files before you start with your problem statement

Project 11

Highlights

Perform 2D Object Detection with tensorflow’s object detection API

Learning Objectives

Understand the working of an Object Detector 

Gain experience in curating a dataset and preparing it for training 

Evaluating the performance of the model with the Object Detection Metrics 

Description

  • An explanation about the tensorflow’s object detection API will be provided 
  • Dataset preparation and training will be performed on any publicly available Dataset 

Task 

  • Perform the training on a dataset with one of the object detection models of Tensorflow API
  • Provide a summary of the working of the object Detector 
  • Submit an evaluation report on the performance of the model 

Project 12

Highlights

Note: Capstone project. Few research papers will be provided. Projects would be developed based on it. The topics and the project description are not finalized yet. However , tentative topics are mentioned below 

  1. 3D Object detection from 2D Images – Monocular 3D , Object Detection Leveraging Accurate Proposals and Shape Reconstruction
  2. Human Pose Estimation – Deep Cut : Joint Subset Partition and Labeling for Multi Person Pose Estimation 
  3. Panoptic segmentation 
  4. COCO-GAN : Generation by Parts via Conditional Coordinating 
  5. Hierarchical Multi-Scale Attention for Semantic Segmentation

 


WHO IS THIS COURSE FOR ?


  • Graduates with a bachelors in ECE, EEE or Mechanical

SOFTWARE COVERED


COURSE FEE

Get an edge over your peers by investing in your future.

6 month 0% EMI
Total Course Fee is Rs. 30,000.
Check your 0% EMI Eligibility

GET A CHANCE TO WIN A 10% SCHOLARSHIP FOR THIS COURSE

Take Scholarship Test

Frequently Asked Questions

1Who can take your course?

The course is designed for individuals aspiring to establish a career in Computer Vision. This course is structured to provide a clear and deep understanding of the basics of computer vision with an application-oriented approach. The applications will be channelized predominantly towards Autonomous Vehicles. However, the concepts learnt in the course can be applied in multifarious domains.

2What is included in your course?

The course is essentially a sequential approach towards learning computer vision. The course covers the basics of computer vision, understanding the perception of an image/Image sequence (video), working of a camera and applications of computer vision in Industry. The course provides hands on experience by providing exercises and projects that are relevant in the autonomous vehicle industry. The projects would code in Python and the students can familiarize themselves with standard tools used in computer vision such as OpenCV, Tensorflow / Pytorch and other standard Python libraries. There will be a total of 12 projects, one every week, and quiz /challenges to test the understanding of the course materials.

3What will the student gain from your course?

A complete understanding of computer vision, rationale behind the application of methodologies and hands on experience in developing and implementing state-of-the-art computer vision algorithms. The students will gain research and development experience in solving the problems / exercise every week.

4What software skills are you teaching and how well are these tools used in the industry?

TensorFlow, Keras, Python, OpenCV, Machine learning and Deep learning approaches will be taught in this course. The concepts and algorithms learnt in the course is agnostic to the programming language and can be readily applied across different languages including Matlab. Additional skills gained would be best coding practices, hardware and computation conscious implementations.

5What is the real world application for the tools and techniques will you teach in this course?

The real-world applications in this course would include computer vision methodologies and algorithms implemented for Autonomous vehicles. The exercises will follow a sequential approach in developing algorithms for autonomous driving from detecting lanes to developing perception of complex environments.

6Which companies use these techniques and for what?

Computer vision is increasingly being adapted and applied by companies across multiple domains such as banking, surveillance, Automotive, Sports Analytics, Virtual / Augmented reality, Medical Imaging etc. The essence of applying computer vision methodologies is to enable machine vision and provide the perception and logical thinking like that of a human being.

7How is your course going to help me in my path to MS or PhD?

The concepts learnt and the experience gained upon completion of this course will be a foundation in computer vision. Such a foundation is essential for a master’s course focussed in research. The exercises and projects completed in this course will certify the candidate’s knowledge and research experience. Such a background will make a compelling profile for Master’s / PHD study’s application.

8How is this course going to help me get a job?

The course provides hands on experience in developing and implementing computer vision methodologies and algorithms to problems in real life industries. The codes scripted and projects developed can make a great GitHub profile for the candidate. This will provide more visibility and subsequently competitive and lucrative job offers.

9What is the ratio of theory to hands-on practical content?

The course establishes a fine balance between imparting theoretical knowledge and providing hands on experience. 50% of the course will focus on theoretical knowledge and 50% will focus on practical experience. However , we strongly recommend the student to explore more research materials and algorithm implementation to gain enhanced understanding of the subject.


SKILL LYNC WORKS TO GET YOU A JOB

See all

Certification

  • Top 5% of the class will get a merit certificate
  • Course completion certificates will be provided to all students
  • Build a professional portfolio
  • Automatically link your technical projects
  • E-verified profile that can be shared on LinkedIn

Testimonials

Companies hire from us

See all

You Might Also Be Interested In

Related Courses

See all

The Skill-Lync Advantage

See all