Self-driving cars and sensors

A self-driving car is a vehicle that can sense its surroundings and drive itself with little or no human help. It uses sensors like cameras and radar to 'see' the road, and a computer that decides how to steer, speed up, and brake. The goal is to travel safely from one place to another without a person controlling it.

Sensors act like the car's eyes and ears, giving the computer information about the world around it. Without sensors, the car would have no way to detect other vehicles, pedestrians, lane lines, or traffic lights. The data from sensors lets the computer make safe driving decisions.

A camera captures images of the road just like our eyes do, letting the car 'see' colors and shapes. It is especially useful for reading traffic lights, road signs, and lane markings because these rely on color and text. Computer programs then analyze the pictures to recognize what each object is.

LiDAR stands for Light Detection and Ranging. It sends out many laser pulses and measures how long they take to bounce back from objects, using that time to calculate distance. By doing this thousands of times, it builds a detailed 3D 'point cloud' map of everything around the car.

Radar sends out radio waves that bounce off objects and return to the sensor, revealing how far away things are and how fast they are moving. It works well in rain, fog, and darkness when cameras struggle. This makes radar very useful for spotting other vehicles and keeping a safe distance.

An ultrasonic sensor sends out high-pitched sound waves and listens for the echo to measure short distances. Cars use them mainly for parking, because they are good at detecting nearby objects like curbs or walls. They are cheap and reliable but only work over short ranges.

Each sensor has strengths and weaknesses, so combining them gives a more complete and reliable picture. For example, cameras see colors but struggle in the dark, while radar works in fog but cannot read signs. Mixing their data, called sensor fusion, helps the car stay safe in many conditions.

Sensor fusion is the process of combining data from several sensors into one clear understanding of the environment. The computer merges information from cameras, radar, LiDAR, and others so that the strengths of each cover the others' weaknesses. This produces a more accurate and trustworthy view than any single sensor alone.

They use machine learning, where a computer is shown thousands of labeled example images of pedestrians, cars, and other objects. By studying these examples, the program learns the patterns that make each object look the way it does. Later it can recognize new objects it has never seen before.

Training data is the large collection of examples, like images or sensor readings, that the AI studies to learn. Each example is usually labeled, for instance marking which pixels are a 'pedestrian' or a 'stop sign'. The more varied and accurate the training data, the better the car learns to handle real situations.

Engineers describe self-driving ability on a scale from Level 0 to Level 5. Level 0 means no automation and the human does all the driving, while Level 5 means the car can drive itself anywhere with no human needed. The middle levels gradually shift more of the driving from the human to the car.

At Level 5, the car can drive itself in all conditions and on all roads without any human help at all. There is no need for a steering wheel or pedals because a person never has to take over. This is the highest and most complete level of self-driving, and it is still very hard to achieve.

Detecting a red light tells the car it must stop to avoid crashing into crossing traffic or pedestrians. If the car misreads the light, it could run through an intersection and cause a serious accident. Because the stakes are so high, the system must recognize lights very reliably in all weather and lighting.

A point cloud is a collection of many tiny dots in 3D space, where each dot marks a spot the laser bounced off. Together these dots form a detailed shape of objects and surfaces around the car. The computer studies this shape to figure out where cars, people, and obstacles are.

Snow and rain can block or scatter the signals from sensors, making it hard to see the road clearly. Cameras may be covered by raindrops, lane lines can be hidden under snow, and LiDAR pulses can bounce off falling flakes. This is why bad weather is one of the biggest challenges for self-driving cars.

An edge case is a rare or unusual situation that the car does not see often, like a person in a costume or a couch lying on the highway. These are important because the AI may not have learned them well from its training data. Handling edge cases safely is one of the hardest parts of building self-driving cars.

GPS uses signals from satellites to tell the car roughly where it is on a map. This helps with planning routes and knowing which roads to take. However, GPS alone is not precise enough for safe driving, so cars also use sensors and detailed maps to know their exact position.

Localization is the car figuring out exactly where it is in the world, often within a few centimeters. It combines GPS, sensor data, and detailed maps to pinpoint its position on the road. Knowing its precise location helps the car stay in its lane and plan safe movements.

HD maps store very detailed information about roads, such as the exact position of lanes, curbs, signs, and traffic lights. The car compares what its sensors see to the map to confirm where it is and what to expect ahead. This extra knowledge helps the car drive more safely and smoothly.

A neural network is a computer program loosely inspired by how brain cells connect, and it is very good at finding patterns in images. After training on many labeled pictures, it can look at a new camera image and decide whether it shows a car, a person, or something else. Self-driving cars rely heavily on neural networks for this kind of recognition.

Object detection is the task of finding and labeling important things in sensor data, like drawing a box around each car, pedestrian, or cyclist. It tells the system not just that something is there but also what it is and where. This information is essential for the car to predict movement and avoid collisions.

Other drivers, cyclists, and pedestrians move and change direction, so the car must guess their next actions to stay safe. For example, it should anticipate that a pedestrian near a crosswalk might step into the road. Good predictions let the car slow down or change course before a dangerous situation happens.

Path planning is deciding the route and exact movements the car should make to reach its destination safely. It chooses things like when to change lanes, when to turn, and how to avoid obstacles. The plan must follow traffic rules and keep a safe distance from other road users.

Simulations let engineers create thousands of virtual driving situations, including dangerous ones, without risking real people or cars. The AI can practice rare events like a child running into the street safely inside the computer. This speeds up learning and improves safety before the car ever drives on a real road.

Many crashes are caused by human mistakes such as distraction, tiredness, or speeding. Self-driving cars do not get tired or distracted and can react very quickly, so supporters hope they will reduce accidents and save lives. However, this benefit depends on the technology being extremely reliable first.

Sometimes a crash may be unavoidable, and the car's software might have to choose between two bad outcomes, such as risking its passenger or a pedestrian. This is similar to the famous 'trolley problem' in ethics. There is no easy answer, and engineers, lawmakers, and society must discuss how cars should decide.

Responsibility could fall on the car maker, the software developer, the owner, or even a human who was supposed to supervise, depending on the situation and the law. This is a difficult legal and ethical question that countries are still working out. Clear rules are needed so victims can be treated fairly.

Self-driving cars collect lots of data, including where you go, when you travel, and sometimes video of the surroundings. If this data is not protected, it could reveal private details about people's lives or be misused. So companies must store and use this information carefully and respect people's privacy.

A false negative happens when the system fails to detect something that is actually there, like missing a pedestrian crossing the road. This is dangerous because the car might not slow down or stop in time. Reducing false negatives is a top priority for keeping self-driving cars safe.

Many sensors measure distance by timing how long a signal takes to travel to an object and back, then using the signal's speed. For sound, distance equals speed multiplied by time, divided by two because the signal makes a round trip. A short program can turn an echo time into a distance estimate.

# Estimate distance from an ultrasonic echo
speed_of_sound = 343  # meters per second
echo_time = 0.02      # seconds for sound to go and return
distance = (speed_of_sound * echo_time) / 2
print(round(distance, 2), "meters")  # 3.43 meters

Real time means the car must sense, think, and act almost instantly, within fractions of a second. Roads change quickly, so a slow decision could lead to a crash. This is why self-driving cars need powerful computers that can process sensor data extremely fast.

Redundancy means having extra or backup systems so that if one part fails, another can take over. For example, if one sensor stops working, others can still provide information to keep the car safe. This makes the whole system more reliable and reduces the chance of a single failure causing a crash.

A human driver can get tired, bored, or distracted by a phone, but a computer pays constant attention to all its sensors at once. The AI can watch in many directions at the same time without losing focus. However, humans are often better at understanding unusual situations and the intentions of others.

Overfitting happens when an AI learns its training examples too closely, almost memorizing them, instead of learning general patterns. Such an AI does well on familiar data but fails on new, slightly different situations on the real road. Engineers use varied training data and testing to avoid overfitting so the car works in the real world.