← Course overviewFree lesson 1 of 3 · lesson 1 of 43Next lesson →

Unlock all 43 lessons of Introduction to ROS/GitHub/LinuxFull course ₹40,000 · paid

Price & syllabus
Free lesson · 22 min · no payment

Autonomous Driving Basics: Perception, Control and Sensor Trade-Offs

A free lesson from Skill-Lync's Introduction to ROS/GitHub/Linux course, on Grow by Skill-Lync · Last updated 16 Nov 2026 · Session: “Week 1 Session 1”

What will you learn in this free Introduction to ROS/GitHub/Linux lesson?

The instructor splits autonomous driving into perception and control, using Waymo and Tesla as examples and the 2018 Uber and Tesla fatalities as limits of the technology. It compares camera, lidar, radar and ultrasonic sensors, explaining why sensor fusion is needed.

  • Perception versus control in autonomous driving
  • Camera, lidar, radar and ultrasonic trade-offs
  • Why sensor fusion is needed
  • Waymo and Tesla approaches, and 2018 fatalities

What comes after lesson 1?

The next free lesson is lesson 2, “Self-Driving Software Stack, Localization and Levels of Autonomy” (36 min). Later in Skill-Lync's Introduction to ROS/GitHub/Linux course (paid): lesson 4, Week 1 Session 4; lesson 5, Week 1 Session 5; lesson 6, Week 2 Session 1. The full course has 43 lessons (18 h 39 min of video); see the course overview or its price and syllabus.

Lesson transcript

Machine transcript of the lesson audio (lightly cleaned). Technical terms may contain errors; the video is the reference.

Hello guys, welcome to the week one of this introduction to ROS, GitHub and Linux course. In the first week, we'll be learning about some introductory lessons on autonomous vehicles, some concepts of ROS and how we are going to progress in this course. In this course, we'll be learning more about what is an autonomous driving, what are self-driving vehicles, what are the levels of autonomy, what exactly is ROS or robot operating system, next we'll look at the importance of ROS and then we'll also look at some of the disadvantages of ROS.

After that, we'll focus on Python and C++, we'll mostly look into the basics of these two programming languages. Now, why we want to learn about Python and C++ is that because autonomous driving and robotics and all the modern technologies that are used right now, including machine learning, deep learning, all of them use either Python or C++. So, what are the learning objectives from this lecture? So, by the end of this lecture, a student should be able to understand the basics of autonomous driving. He or she should be able to understand what exactly an autonomous vehicle is.

We will also look into the different levels of autonomy for self-driving vehicles and understand the difference between each of them. So, this step includes like understanding the different levels of autonomy, like is everything completely autonomous or semi-autonomous and we'll look into these concepts and delve into them individually. After that, we will understand what ROS or robot operating system is. Next, we'll need to know why ROS is needed for autonomous vehicles. And after that, we will look into the advantages and disadvantages of using ROS for autonomous driving.

After that, as I mentioned in the last slide, we will go through the basics of Python and C++. Autonomous driving. So, what is exactly autonomous driving? So, vehicles were traditionally thought of as traditional mechanical ventures. So, what does this statement mean? So, if any of you have driven a vehicle before, you might know like how to drive a vehicle. So, you get into the car, you close the door. Then you start the car or you use a key to start the ignition of the vehicle. And once the engine starts running, then you press the clutch and you put the car in the first gear.

And then you slowly release the clutch and then press the accelerator to drive the vehicle. Once the car starts moving, you can use a steering wheel to control the direction of the motion of the car. When you need to slow down or stop the car, you need to press the brake. So, till modern times, this was enough for humans to drive vehicles like this way. But unfortunately, as the number of people in the world are increasing, which is also proportionally increasing with the number of vehicles, it has become a bit difficult or let's say a bit complex to manage so many different cars and so many different drivers on our current existing road infrastructure.

As you might understand that the amount of roads cannot be increased exponentially because there is not enough land space which are available for constructing new roads. But unfortunately, the number of vehicles in this world has been continuously increasing. So, one of the ways to solve this problem of ever increasing number of cars but limited number of roads is to have autonomous vehicles which can run around without an human intervention and which can then safely ferry humans around the clock without anyone needing to own their own cars.

So, as I mentioned that cars were completely mechanical things. So, how do we go from a completely mechanical thing to something which is completely autonomous? So, as you might understand, there is no way for a car to press the clutch and shift the gear and press the brakes autonomously. Like, they are all mechanical systems. So, how do we get to something which can be driven autonomously? So, this problem actually can be broken down into two main problems. One is of the perception and another one is of control.

So, what do we mean by perception? So, let's say when you as a driver is driving the car, you not only just move and operate the various systems in the car, like you do not only turn the steering wheel, you do not just press the clutch and the brake. You constantly look and hear and you are aware of the surroundings. So, you are aware of if there is some roadblock directly in front of the car, if there is a traffic signal which is red and you have to stop the car, then you have to be aware of pedestrians or cyclists who are also crossing in front of the car and you have to act accordingly.

Along with that, you also have to hear horns coming from other cars. You have to hear if there is some big other loud noise that is happening and then you have to be very cautious and then slow down the car and so on. So, this part of driving which has nothing to do with any mechanism of the car is called perception is very difficult to do because there is no device in the car in the car systems currently that can help the car to perceive the environment and that is the most important component of driving a car.

So, in order to do perception, we have to rely on various kinds of sensors and we will see that not all sensors are suitable for all kinds of environment. So, we don't only need just one type of sensor to perceive the environment, we need multitude of them and we have to use signals from all of them and combine them or fuse them to help the car perceive its surrounding environment. After that, when the perception part is handled, we also need to take care of the control part.

So, for that, we have various kinds of motors and various kinds of controllers which help us to do this task and the input of these motors and controllers come from the output of the sensors that I mentioned. So, autonomous driving vehicles are actually a fusion of hardware and software, a very huge fusion of various kinds of sensors and controllers and the combination of all of that actually helps us to drive the car autonomously. Now, autonomous driving commonly refers to a research area where an autonomous vehicle tries to operate with minimal or human assistance.

So, like I mentioned that currently the state of the art is that the human driver needs to do all the perception part and he or she needs to provide instructions to the car through the steering wheels, clutch, through the brake, through the accelerator and so on and drive the car. So, if a car has to drive autonomously, then it needs to do all of the perception tasks, all of the control tasks and so on. And in order for an autonomous vehicle to be considered completely autonomous, it needs to operate with minimal or no human assistance.

So, minimal, what does minimal human assistance mean? So, minimal human assistance can mean that the car can travel or can drive autonomously for some stretch, let's say on a highway where driving is mainly traveling in a straight line, the car can just maintain a very certain kind of speed and just drive along the road in a straight line where it does not need to change lanes or take too many left or right turns. And in case where a car needs to make a certain decision, let's say it needs to slow down and merge with another lane and so on, then the human can provide some assistance by turning the steering wheel and so on.

That can be an example of a minimal human assistance. And in case of no human assistance, it means that the car can drive from location A to location B without the human needing to do any task like turning the steering wheel or pressing the brakes or pressing emergency brakes in case of certain things that might come up in front of the car. And so, if the car is able to do that without any human assistance, then the car can be considered truly autonomous. So, now that we have looked into the concept of autonomous driving or what an autonomous driving is, now we can look into what a self-driving car is.

A self-driving car is capable of perceiving the local surroundings, as I mentioned, with the help of various kinds of sensors like lidar, radar, camera, and so on, which we will cover in the following slides. And then using the data from these sensors, others. To navigate autonomously throughout the environment without or no minimal human assistance. So example of an autonomous driving is like in the current scenarios in the last few years, Waymo, which is a company of Google, has driven its self-driving car for almost 10 million miles or 16 million kilometers.

So how did Waymo do it? Bought a car which was available in the market, as you can see from the picture here. Then it mounted a LIDAR sensor, which is this black object here that you can see. So with the help of LIDAR, it was able to scan the environment the car was driving in, that is, it did the perception task. And based on the data from the LIDAR scanner, it fed it into a computer, which did all the processing based on the input data. And then it used that data to compute the direction of the car, the action that the car needs to perform, and accordingly sent signals to control the car.

Pressing the clutch, changing the gear, pressing the brakes, pressing the acceleration, and so on. So this is what Waymo did. Now we'll look into the use case of Tesla. So Tesla has been, has also deployed its own autonomous driving system in all of its car. And Tesla, the current state of the art of Tesla is that with Tesla, the human driver can choose when to drive himself and when to switch to autonomous driving mode. And the main advantage of Tesla is that you can choose, since you can choose when to switch to autonomous driving mode and when to switch to manual driving mode.

Most users have used this autonomous driving mode on highways and where free stretches of roads are easily available. So in those stretches of road, Tesla has driven for almost 1 billion miles. That is, you can understand almost 100 times more as what Waymo did. And because Tesla has, has a great advantage that most of its cars are equipped with camera and equipped with sensors to do autonomous driving. So whenever a user or whenever a driver of Tesla switches onto autonomous driving mode, it can easily capture all of that data and use that data to make its autonomous driving model better with every driven kilometer.

So before we move on to the key technologies, I want to highlight a few drawbacks of autonomous driving. So is autonomous driving always perfect when without any failures? Actually, no. Uber, as you might have heard of this company, this is taxi driving company and you, I'm sure you've already used the Uber app. So in Uber also had his, had its own autonomous driving division, where it was trying to build autonomous driving technologies so that it can have a complete fleet of robot access. Uber faced a very serious drawback in 2018 when one of its autonomous driving car, which was driving around the road autonomously with just a driver behind the wheel, but he did not intervene in time.

And this incident resulted in death of a pedestrian. And the main fault of the, of the driver was, even though he was sitting behind the wheel, he was not exactly paying attention to the road. And because of some lighting, different lighting conditions, the car was not able to perceive the pedestrian quite well. And it resulted in the car colliding with the pedestrian and killing her. After that, Uber had to shut down its autonomous driving department because of this fatality. And it is now focusing on other parts of its business.

And the level of autonomous driving in that car was L3, L4. So it was not completely autonomous, but it was autonomous driving with very minimal human intervention. So apart from Uber, another car company, Tesla also had its own case of fatality. It happened in 2018 as well, where a driver was actually driving his Tesla on a highway and the driver had activated the autonomous driving mode in the car. And he was, similar to the other Uber case, he wasn't also paying too much attention to the road.

And what happened is, there was a huge crash on the highway, a truck overturned, and then the car perceived the overturned truck and it could not determine if it was a collision or not. And it resulted in the car, that is the Tesla car, colliding with the truck. And it resulted in the, in a death of the driver as well. And Tesla was made liable for this fault in the autonomous driving accident that happened. But one thing to be noted is that despite of these two incidents, which shows that autonomous driving is not perfect, we have to consider that the number of deaths that happened because of failure of the human driver to control the situation.

And so, road accidents actually kill one person every 23 seconds in the world. So, in a year, it kills more than 40,000 people. So, till now, autonomous driving vehicles have killed only two people and human drivers have killed almost like 40,000 people in the whole year. So, you can well understand that even though this technology needs to be, like, probed more and verified and examined more, but we can still say just based on the number of miles that an autonomous car has driven and number of miles that human user drives, percentage-wise autonomous vehicle technology even now is much more safer than a human driver.

So, now, we'll look into the key technologies that are associated with an autonomous driving vehicles. So, the main technology, as I mentioned, that the autonomous driving can be divided into two parts, perception and control. So, for the perception part, we have various kinds of sensors. So, we have, from the picture that you can see, we have a camera, we have cameras for the sides, we have cameras for the rear, and we have cameras in the front. We have forward lidar, which is the laser sensor that I showed you in the last picture, which is mounted on top of the Waymo car.

We have long-range radar, which is a bit different from lidar, so lidar depends on optical technology, and radar depends on ultrasonic wave technology. Then, along with forward long-range radar, we have ultrasound sensors, which are mainly used for near object detection, and we have short and medium-range radar. Along with all of this, we also have thermal imaging sensors. So, why do we need so many different kinds of sensors? So, for that, we will come into the various drawbacks and the various advantages of each sensor. We will look into the advantages and disadvantages of the various sensors that we use for the perception stack.

So, camera, as you might know, camera gives us a very clear image of what lies in front of us, like it acts as good as the human eye. But the main drawback of camera is, based on different lighting conditions, the range or the functionality of a camera changes quite a bit. So, let's say if a camera, with a camera, we are able to detect a few objects, let's say a roadblock, clearly when there is ample sunlight, but if the weather condition changes to cloudy, if we do not have ample, then the object detection using camera can fail quite a bit.

So, this is the main disadvantage of camera, that based on different lighting conditions, the performance of the camera can change. Let's say, now we consider the case of LiDAR. So, LiDAR sensor, basically what it does is it sends out a laser signal, and based on the reflection of the laser signal, it's able to detect if there is some object or obstacle in front of it or not. So, LiDAR works in most weather conditions quite well, but unfortunately when there is rainfall, raindrops can reflect the laser scanner data in various ways and can actually scatter them, which results in a bad signal to noise ratio.

And hence, those are the situations where LiDAR fail as well. So, what are the advantages of radar? So, radar doesn't rely on any optical sense, and it relies directly on the ultrasound technology, which is it sends out a signal, and based on the signal which is reflected back to the sensor, it can determine if there is an object or an obstacle in front of it. So, this is how radar works. So, radar is also good for detecting various kinds of signals with ultrasound technology, but the main disadvantage of radar is that the cost of radar is quite high, and it has a medium range, but it does not provide any color contrast.

So, let's say if we have to detect a traffic signal with radar, it will not be possible because radar won't be able to differentiate between red and green light. Even LiDAR is disadvantaged in this case because with LiDAR we are... You cannot determine what color the light of the traffic signal is. And this is one case where camera is the best solution. So what is the advantage of LiDAR then? So LiDAR works in various kinds of lighting conditions. Let's say it doesn't matter if there is sunny or cloudy or in dark, LiDAR can still give us quite a bit of information about the current environment, which unfortunately a camera cannot do.

And with ultrasonic sensors, the cost of the sensor is very high, but it does not detect speed and the range of an ultrasonic sensor is quite low. So it can be only used for near object detection. And passive visual, that is the camera sensor, can help us determine the range and the resolution, but as I mentioned, it does not detect quite a bit of objects in various lighting conditions, and it also detects speed quite poorly. So that is where sensor fusion comes in, where we combine the effect of camera, we combine the effect of range, and we combine the effect of radar and LiDAR to give us a various, a complete 360 degree view of a car when it's traveling, and this actually helps us with the perception task.

And after we have been able to perceive the environment quite correctly, we use that to go into the control mode. So we need to get shut down. tec. So enjoy. Thank you. Bye. Bye.

Want the rest of Introduction to ROS/GitHub/Linux?

Get the full course syllabus and fees.

The full course is paid (₹40,000). We'll send the syllabus and call you back within one working day.

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.

Full course · paid

Unlock the full Introduction to ROS/GitHub/Linux course

The remaining 40 lessons, mentor-reviewed projects and a certificate come with the paid course (₹40,000). Leave your details: we'll send the syllabus and fees and call you back.

Get the syllabus and fees

A counsellor calls you back within one working day.

By submitting you agree to our Terms and Privacy Policy.

Grow by Skill-Lync · free lesson

Start watching

Free, no payment. Leave your details once and every free lesson on Grow plays straight away. A Skill-Lync counsellor may call you about the full Introduction to ROS/GitHub/Linux course.

By submitting you agree to our Terms and Privacy Policy.

Talk to a career counsellor

Talk to a career counsellor

Free 20-minute call. We'll map the right course to where you want to work.

By submitting you agree to our Terms and Privacy Policy.