Which machine learning algorithm is used in self-driving cars?

Question

Which deep neural network is used in Google's driverless cars to analyze the surroundings? Is this information open to the public?

Answer 1

It will not be single DNN architecture, rather it will be a collection of different DNN architectures that are used together to make the final decision. Convolutions are using the images/videos from the camera. Other architectures use other sensory sources. These DNNs will be trained to compute the high-level features from their sensory sources and then those high-level features will probably be fed into an LSTM (or some other form of RNN) that is trained with some form of Reinforcement learning algorithm to compute the action (like slowing down, applying breaks etc).

Answer 2

What you are calling 'analyzing the surroundings' is generally referred to as perception. Self-driving cars sense their surroundings using cameras, radars, lidars often combining or fusing more than one sensor to paint a picture of the environment. A lot of algorithms get used for fusing the sensor data and then deriving an understanding of the surrounding. One such example is semantic scene segmentation of camera data that tries to identify object boundaries in camera images. Typically a fully convolutional neural network is used to achieve this.

To the best of my knowledge Google does not disclose the exact algorithms anywhere.

Answer 3

Self-driving cars use a combination of both supervised as well as reinforcement learning.

Huge amounts of sensor data are recorded in real-time. This data can be used to train all sorts of supervised classifiers, e.g. for predicting rain or switching on lights. You can also set up a model to predict pedestrians and other cars. This is supervised learning.

Reinforcement learning can be used in situations positive or negative signals appear when driving a car: Traffic lights, blinking signals from other vehicles and street signs in general. These signals can be used to train a reinforcement model and decide on best actions (adjust speed, steer,..) to get the maximal reward (or better minimize costs of a crash)

Answer 4

The most common machine learning algorithms found in self driving cars involve object tracking based technologies used in order to pinpoint and distinguish between different objects in order to better analyse a digital landscape.

Algorithms are designed to become more efficient at this by modifying internal parameters and testing these changes.

I hope that provides a general overview of the subject.

Since Google's cars are in development and are proprietary, they will probably not share their specific algorithm, however you can take a look at similar technologies to learn more.

To find out more, take a look at an Oxford-based initiative in self driving cars and how they work.