I am looking for a solution that I can use to identify cars.

I have a database with images of cars. About 3-4 per car. What I want to do is upload a picture to the web of the car (picture taken with camera/phone), and then let my pc recognize the car.

Example. Let's say I have these 2 pictures in my database (Mazda cx5)(I can only upload 2 links at max. atm. but you get the idea).

First car

Now, I am going to upload this picture of a Mazda cs5 to my web app:

Picture of mazda cs5

Now, I want an AI to recognize that this picture is of a Mazda CX5 with greyish color.

I have looked on the net and found 2 interesting libraries I can use, TensorFlow and Clarifai, but I don't know if these are going to work.

So, my question to you is: What would be my best bet to go with here?

  • 1
    $\begingroup$ Please, do not roll back my edit unless it interferes with the meaning of your original post. I edited your post to fix some language issues and to improve the title: I didn't change anything else. Isn't the title that I propose more descriptive of your problem? If you are not comfortable with the idea of your contributions being collaboratively edited by other trusted users, this may not be the site for you.. Note that you still appear as the author of this post. $\endgroup$
    – nbro
    Jun 5 at 12:20
  • 1
    $\begingroup$ Since this question has been receiving several flags and had some edit wars going on, I'd just like to note again that I also looked into the edit applied by @nbro and that edit looks perfectly fine and warranted, no reason to continue rolling it back. See the links provided by nbro above for more info. I'm going to try temporarily locking this post once more since the edit war resumed again after the previous lock. Please stop this. If there are specific issues with the edit that change the original intent of the question, please address only those issues specifically and explicitly $\endgroup$
    – Dennis Soemers
    Jun 15 at 10:53

Model of the car

What you want to do is close to one-shot image recognition. You have not 1, but 3-4 examples of each car, but that is still a small amount, especially considering the car looks different from different angles (are you supposed to recognize them from any point of view, including sideways, rear, front, and 45 degrees etc.? maybe you also want to recognize them photographed from the top?).

One interesting article I found is: Siamese Neural Networks for One-shot Image Recognition by Koch, Zemel, and Salakhutdinov.

I also found that Caffee supports Siamese networks.

You may want to read other literature about the One-shot learning.

One trick you can do is to utilize the fact that cars are symmetric, so you can double the number of learning examples by reflecting each image.


Determining the color is not as simple as it seems. Your algorithm need to determine where is the car at your picture, and then determine the color, taking into account the lighting conditions, as well as light effects, most notably reflection. For example, consider the following image: .

We see strawberries as red, but there are no red pixels on this picture. The images of strawberries consist on grey pixels.

Maybe you also need a convolutional neural network, or just a neural network, for this task.

  • $\begingroup$ If you are going to use a CNN to do the classification you are going to need a large data set to train the network. The Stanford Cars Dataset has 16,185 images and 196 classes of cars. It can be found and downloaded from Kaggle at kaggle.com/jessicali9530/stanford-cars-dataset $\endgroup$
    – Gerry P
    Mar 6 '20 at 16:21

There are several ways you can do this. One way would involve several steps and would probably work best:

  1. Use a trained Gaussian detector to filter out the car from the rest of the image
  2. Use a convolutional neural network to classify the car
  3. Use a neural network or a simple most common color algorithm to find the color of the car

You would be able to implement this method most easily in MATLAB but you would also be able to do it in python with tensorflow or torch. You would probably be able to implement the trained Gaussian detector in tensorflow.

Method 2:

  1. Use a spatial transformer network to "transform" the image of the car for easy classification
  2. Use the output of the spatial transformer network for classification via a convolutional neural network.
  3. Use another neural network or a most common color algorithm to find the color of the car.

This method would also work pretty well but using a spatial transformer network with a convolutional neural network may be hard to code because it is an area of developing research where there are many problems because the spatial transformer network and the convolutional neural network have to work well together and this is usually hard to get right.

Method 3:

  1. Use a convolutional neural network straight up on the input image maybe with down sampling to classify the car
  2. Use another neural network to find the color of the car

I would personally go with method #1 because it would be fairly simple to implement with existing libraries such as tensorflow and it would most likely provide a high accuracy.

As always I would also recommend that you use LIME during the development process to debug your model and determine what features you could add in or remove to help your model perform better.

**Edit* Since you need to detect certain patterns on the cars for color classification I would recommend that you use a convolutional neural network to classify these patterns. So your method would now look like this:
1. Use a spatial transformer network or a filtered Gaussian detector to filter out the car 2. Use a convolutional neural network to classify the make and model of the car. 3. Use another neural network that has either a convolutional or deep architecture to classify patterns and solid colors. So the outputs would contain all of the colors that you want and all of the patterns that you want to detect.

  • $\begingroup$ Hi Aiden, thank you for your answer. I will look into all of these. Would option 1 also be viable for recognizing 2 different color, for example zebra stripes? $\endgroup$
    – anon
    Mar 4 '17 at 20:48
  • $\begingroup$ Using a neural network yes, you would be able to detect two different colors. $\endgroup$ Mar 4 '17 at 22:34
  • $\begingroup$ Hi, aiden. I have looked into it a bit more, another question I have is would pattern recognition work with these CNN's? when a car has for example the american flag printed on it would I be able to detect that pattern? $\endgroup$
    – anon
    Mar 6 '17 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy