I was able to extract the license plate from a given car image, using Matlab. I would like to use deep neural networks to recognize the characters on the plate now. How can i proceed further? I don't have any experience with deep neural networks implementation.

  • $\begingroup$ Welcome to AI.SE! I'm sorry, questions about the implementation of machine learning are off-topic here. For an intro to our site, see the tour. You might be able to get help at Cross Validated after looking through their asking guidelines. $\endgroup$ – Ben N May 15 '17 at 18:07
  • $\begingroup$ Send this question over to data science rather. $\endgroup$ – JahKnows May 15 '17 at 22:05

Before jumping into any machine learning task it is good to consider your dataset and the features you wish to use. License plates are unique thus feeding the entire plate into a machine learning algorithm would not yield very accurate results. First, you will want to make sure that the extracted license plate images are similar in size. Then you will want to separate the license plate such that you can separate the different digits and letters into their own respective images. Thus your dataset that will be used to train your network will consists of many single character images which contain either a letter or a digit. When you have finished training your model, future license plates will also have to be broken down in this way, classified and then the outputs be combined to get the license plate number.

What is a neural network?

A neural network (NN) is a series of nodes which contain a simple, continuous, differentiable function such as logistic regression. Alone these nodes are not very powerful, however, the beauty comes in when they are connected in networks. The tuning of such a network can learn highly complex functions. Typically, a NN is set up with three layers. The input layer, where you will feed in your features, the hidden layer and the output layer. Naturally, if you want a deep NN then you just need to add more hidden layers. But, there is a trap. The deeper your network is the harder it will be to train, thus you will need MORE data.

The input-layer

This is where you will be feeding in your images. If you have a 16*16 pixel image then you will have 256 input nodes. This layer typically does not perform any function on the input, it is simply taking in the input. It will then feed the input to the hidden layer. You can use some dimensionality reduction techniques to try and eliminate some useless information from your dataset such as the white spaces in the corner of the image. The less input nodes you have, the more informative your samples become, thus with higher information entropy your network will be able to learn faster, or more rigidly with the same amount of data.

The hidden layers

You are free here to choose how many layers you want and how many nodes you want in each layer. I suggest using grid search and cross-validation as a means to determine which configuration is ideal for your dataset. Too many layers and nodes and you will not end up with a fully trained networks once you run out of training data (too much bias). Too little nodes, your network will not be able to learn the subtleties which are embedded within the data (too much variance). You want to find yourself at the crossroad where these two are minimized. Choose wisely.

The output layer

This layer can be configured in a few different ways. Typically people like to do one output node for each class. In your case there are 26 letters in the alphabet and 10 digits. This gives 36 possible output classes. So you will need 36 nodes in your output layer.

How do I train this?

The training of the NN is by tuning the weights that will be applied to the inputs of each of your nodes. Try to draw your dense network, for each line connecting nodes you will have a weight. I am sure you can see how this becomes quite huge. You will initialize your weights randomly to start. Then you will take each of your examples and pass them through your network. This is called a feed-forward pass through. This will provide you with an output. You will compare your output with your ground truth value using a cost function. This is often the root mean squared error (RMSE) but there are many other options. Then using the derivative you will back-propagate the error through your network to see which weights caused the error and to what extent. The derivative will then allow you to update those weights proportionately.

WOW do I need to program all that?

Nope not anymore!!! You can use Keras in python (my recommendation) or other similar frameworks which have made it a real tea sipping pleasure to program a variety of NN configurations.

My recommendation

I would start with a simple NN and see what kind of accuracy I can achieve. From there I would try to use convolutional neural networks which have been shown to garner much better results on datasets comprised of images. These can also be implemented in Keras. CNN require more data than typical NN thus you might need to get access to more images.

It is always best to train your dataset only on images you have in your original data. However, when it comes to images, adding to your data by artificially shifting images by 1-5 pixels or rotating by 1-5 degrees can add variability which can supplement your original data.

Make sure that the license plate pictures you are using to train your network are taken with a similar system as the images you will be using in the future. You will not get very good results if you train your network on images taken directly behind cars and then you expect to implement your solution in traffic cameras monitoring traffic from above.

  • $\begingroup$ Thank you for your answer. Assuming that I will use a pretrained CNN on digits recognition for recognizing only the digits on the plate, do you think i will need segmentation before ? Because as far as I know CNNs can only recognize single objects in an image. $\endgroup$ – F.Lin May 15 '17 at 20:09
  • $\begingroup$ Hmm no matter what algorithm you use you will need to segment the images to get one digit per image. Nothing stops you from keeping the sequence of the images you are evaluating in memory and then appending their results to get the entire license plate. Consider what you are doing when reading a license plate. You read each value, therefore the computer must do the same. Unfortunately we are much better segmenters than computers are. $\endgroup$ – JahKnows May 15 '17 at 20:30
  • $\begingroup$ Also using a pretrained model may be problematic if your pictures look different than the ones in your training set. $\endgroup$ – JahKnows May 15 '17 at 20:30
  • 1
    $\begingroup$ In that case, It might be better if i train a new CNN on a dataset of various license plates. I think i will need many layers and a very big dataset for that. $\endgroup$ – F.Lin May 15 '17 at 20:39
  • $\begingroup$ But you will still need to separate the digits and letters and feed them alone through your network. How many examples do you have so far? You will need hundreds of thousands of examples!! $\endgroup$ – JahKnows May 15 '17 at 21:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.