I am looking for a solution that I can use with identifying cars. So I have a database with images of cars. About 3-4 per car. What I want to do is upload a picture to the web of car(Picture taken with camera/phone) and then let my pc recognize the car.

Example: Lets say I have these 2 pictures in my database(Mazda cx5)(I can only upload 2 links at max. atm. but you get the idea). First car

Now I am going to upload this picture of a mazda cs5 to my web app: Picture of mazda cs5

Now I want an AI to recognize that this picture is of an Mazda CX5 with greyish color. I have looked on the net and found 2 interesting AI's I can use: Tensorflow and Clarifai, but I don't know if these are going to work so my question to you what would be my best bet to go with here?


Model of the car

What you want to do is close to one-shot image recognition. You have not 1, but 3-4 examples of each car, but that is still a small amount, especially considering the car looks different from different angles (are you supposed to recognize them from any point of view, including sideways, rear, front, and 45 degrees etc.? maybe you also want to recognize them photographed from the top?).

One interesting article I found is: Siamese Neural Networks for One-shot Image Recognition by Koch, Zemel, and Salakhutdinov.

I also found that Caffee supports Siamese networks.

You may want to read other literature about the One-shot learning.

One trick you can do is to utilize the fact that cars are symmetric, so you can double the number of learning examples by reflecting each image.


Determining the color is not as simple as it seems. Your algorithm need to determine where is the car at your picture, and then determine the color, taking into account the lighting conditions, as well as light effects, most notably reflection. For example, consider the following image: .

We see strawberries as red, but there are no red pixels on this picture. The images of strawberries consist on grey pixels.

Maybe you also need a convolutional neural network, or just a neural network, for this task.


There are several ways you can do this. One way would involve several steps and would probably work best:

  1. Use a trained Gaussian detector to filter out the car from the rest of the image
  2. Use a convolutional neural network to classify the car
  3. Use a neural network or a simple most common color algorithm to find the color of the car

You would be able to implement this method most easily in MATLAB but you would also be able to do it in python with tensorflow or torch. You would probably be able to implement the trained Gaussian detector in tensorflow.

Method 2:

  1. Use a spatial transformer network to "transform" the image of the car for easy classification
  2. Use the output of the spatial transformer network for classification via a convolutional neural network.
  3. Use another neural network or a most common color algorithm to find the color of the car.

This method would also work pretty well but using a spatial transformer network with a convolutional neural network may be hard to code because it is an area of developing research where there are many problems because the spatial transformer network and the convolutional neural network have to work well together and this is usually hard to get right.

Method 3:

  1. Use a convolutional neural network straight up on the input image maybe with down sampling to classify the car
  2. Use another neural network to find the color of the car

I would personally go with method #1 because it would be fairly simple to implement with existing libraries such as tensorflow and it would most likely provide a high accuracy.

As always I would also recommend that you use LIME during the development process to debug your model and determine what features you could add in or remove to help your model perform better.

**Edit* Since you need to detect certain patterns on the cars for color classification I would recommend that you use a convolutional neural network to classify these patterns. So your method would now look like this:
1. Use a spatial transformer network or a filtered Gaussian detector to filter out the car 2. Use a convolutional neural network to classify the make and model of the car. 3. Use another neural network that has either a convolutional or deep architecture to classify patterns and solid colors. So the outputs would contain all of the colors that you want and all of the patterns that you want to detect.

  • $\begingroup$ Hi Aiden, thank you for your answer. I will look into all of these. Would option 1 also be viable for recognizing 2 different color, for example zebra stripes? $\endgroup$ – Gertjan Brouwer Mar 4 '17 at 20:48
  • $\begingroup$ Using a neural network yes, you would be able to detect two different colors. $\endgroup$ – Aiden Grossman Mar 4 '17 at 22:34
  • $\begingroup$ Hi, aiden. I have looked into it a bit more, another question I have is would pattern recognition work with these CNN's? when a car has for example the american flag printed on it would I be able to detect that pattern? $\endgroup$ – Gertjan Brouwer Mar 6 '17 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.