4
$\begingroup$

I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of training, ie. training on full images. Thats quite problem for me, because the biggest real dataset with full traffic sign images is Belgian one with 9000 images in 210 classes, which is unfortunately not enough to train good detector.

To overcome this problem, I've created DatasetGenerator, which does quite good job in generating synthetic datasets, you can see in the results directory.

Recently I came across GAN's which can (besides others) generate or extend existing dataset and I would like to use these networks to compare with my dataset generator. I've tried this introduction to GANs succesfully.

The problem is it's unsupervised learning and so there are no annotations. It means it's able to extend my dataset of traffic signs, but the generated dataset won't be annotated at all, which is problem.

So my question is: Is there any way how to use GAN's to extend my dataset of full traffic sign images with annotations of traffic sign class and position? Actually the class is not important, because I can do it separately for each class, but what matters is the position of traffic sign in generated image.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this loss function is going to take a little work!

Making synthetic data realistic enough to allow models to generalize successfully in the real world is a very active and exciting area of research, not least with respect to robotics, and so the work you are doing now should make you very attractive indeed to the right employer.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ That's amazing! $\endgroup$
    – kocica
    Commented Jan 31, 2019 at 13:08
  • 1
    $\begingroup$ Glad you liked it! I would be most interested in reading your thesis when you are finished - I think there is a good chance that lots of other folk will want to read it too. $\endgroup$
    – Edward Dixon
    Commented Jan 31, 2019 at 14:09
  • $\begingroup$ Indeed, I think these dataset generative tools/papers would help a lot of guys at least training some nets. Unfortunately after reading work from Apple Iam not sure I can do such complex system in just few months but I'll do my best and let you know :-) $\endgroup$
    – kocica
    Commented Jan 31, 2019 at 20:24
  • 1
    $\begingroup$ A Floridian MSc student by the name of Connor Shorten - active on Twitter, Medium, etc. - has very similar interests. Probably you two should have a Skype call. I think you under-estimate yourself (and also, since the Apple work, there'll be many others on the same road). $\endgroup$
    – Edward Dixon
    Commented Jan 31, 2019 at 22:45
0
$\begingroup$

You could add the desired traffic sign location to the latent vector and then arrange that the generator incurs loss if the traffic sign is not at the right place in the generated image.

Improve this answer
$\endgroup$
3
  • $\begingroup$ If I knew position of traffic sign in the generated image, I could have just use that position as annotation of image. But since it's generated image based on let's say 1000 real images, I down know position of traffic sign in the generated image. $\endgroup$
    – kocica
    Commented Jan 31, 2019 at 12:54
  • 1
    $\begingroup$ The sign location could be random and the generator's task would be to generate the sign at that particular location. This would be similar to a conditional GAN: instead of the desired class label (CGAN), here you would supply the desired object location. $\endgroup$
    – ssegvic
    Commented Jan 31, 2019 at 13:34
  • $\begingroup$ Thats nice idea, but still the YOLO detection network learns (besides others) postion of the object in the full image. That means if all the images in the training dataset would have traffic sign on the same place, YOLO would learn that feature and then, most-likely, would detect traffic signs in that desired location only (but I am not entirely sure with that tho). I will give it a try and we shall see :) $\endgroup$
    – kocica
    Commented Feb 1, 2019 at 19:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .