I'm currently working on license plate recognition. My system consist of 2 stage: (1) License Plate region extraction & (2) License Plate region recognition.
I'm doing (1) with Raspberry pi 3 model b. I find license plate candidate first by merging bounding boxes based on their similarity. In this way, i have only 1~7 license plate region proposals. And it took less than .3 seconds.
Now i have to reduce number of region proposal to be around only 1~2 so that i can send these images to server to do job (2). For license plate extraction, I made my own classifier function in tensorflow and the code is below. It gets proposed license plate as input.
First, I resize all license plate to be [120, 60] and converted to gray image. And there are 2 classes: 'plate', 'non_plate'. For non_plate image, i collected various image that might appear in image as background. I have 181 images for 'plate' class and 56 images for 'non_plate' for now, i trained for about 3000 steps so far and current loss is .53 .
When i did prediction on test set, i encountered problem that for some of plate image, it doesn't recognize license plate which is very obviously license plate image from my eyes. It is okay for me to wrongly recognize non plate image as plate but it is problem if it wrongly recognize plate as non_plate because it will not be sent to server to be fully recognized.
It happens like 10 out of 100 test images and this rate is far worse than i expected. I need help for adressing this problem. Would there be any improvement that i can make?
(1) Is my training set too small to classify between license plate and non license plate? Or is number of steps is too small?
(2) Is my graph structure bad? I needed to have small graph structure for my raspberry pi to recognize less than 1 second. Could you suggest better structure if it is bad?
(3) Is it bad to resize any proposed image to [120, 60] to be used as input for graph? I think it loses some information. But isn't this close to roi pooling like used in fast rcnn?
inputs=tf.reshape(features[FEATURE_LABEL],[-1,120 , 60 ,1],name="input_node") #120 x 60 x 1, which is gray conv1=tf.layers.conv2d(inputs=inputs, filters=3, kernel_size=[3,3], padding='same', activation=tf.nn.leaky_relu ) #conv1 output shape: (batch_size,120,60,3) pool1=tf.layers.max_pooling2d(inputs=conv1,pool_size=[2,2],strides=2,padding='valid') #pool1 output shape: (batch_size,60,30,3) conv2=tf.layers.conv2d(inputs=pool1,filters=6,kernel_size=[1,1],padding='same',activation=tf.nn.leaky_relu) #conv2 output shape: (batch_size, 60,30,6) pool2=tf.layers.max_pooling2d(inputs=conv2,pool_size=[2,2],strides=2,padding='valid') #pool2 output shape: (batch_size, 30,15,6) conv3=tf.layers.conv2d(inputs=pool2,filters=9,kernel_size=[3,3],padding='same',activation=tf.nn.leaky_relu) #conv3 output shape: (batch_size, 30,15,9) pool3=tf.layers.max_pooling2d(inputs=conv3,pool_size=[2,2],strides=2,padding='valid') #pool3 output shape: (batch_size, 15,7,9) #dense fully connected layer pool2_flat=tf.reshape(pool3,[-1,15*7*9]) #flatten pool3 output to feed in dense layer dense1=tf.layers.dense(inputs=pool2_flat,units=120,activation=tf.nn.relu) logits=tf.layers.dense(dense1,2) #input for softmax layer
[training non plate image example] 4 [training plate image example. It is region proposed image]