# How to use a Generative Adversarial Network to generate images for developmental analysis?

I want to generate images of childrens' drawings consistent with the developmental state of children of a given age. The training data set will include drawings made by real children in a school setting. The generated images will be used for developmental analysis.

I have heard that Generative Adversarial Networks are a good tool for this kind of problem. If this is true, how would I go about applying a GAN to this challenge?

• So, you can find interesting stuff at this github repo, or this berkeley course... I have to add that none of this is my work, and I'm not an expert of adversarial nets so I can't say that here lies your answer. Moreover, please developp your question, expose your project with more details, because I doubt anyone can really help you with so little info... – 16Aghnar Sep 9 '18 at 18:25
• Welcome to AI.SE @SidneyGuaro. As the other commenters note, it is difficult to understand what you are asking here. In particular, what do you mean by "combine"? GANs are a useful tool for generating new images in certain style. Are you hoping to use a GAN to generate images in the style of children at different developmental ages? – John Doucette Sep 9 '18 at 21:08
• @JohnDoucette yes, exactly – Sidney Guaro Sep 10 '18 at 6:37
• @SidneyGuaro Ok, I edited your question to reflect that. Hopefully it is clearer now, while still matching your intent? – John Doucette Sep 10 '18 at 9:32
• You won't be able to get anything meaningful out of a GAN trained on 200 images. The data set will be too disjoint for the NN to model features at the semantic/stylistic level you are interested in. If all the images are coloured pencils on white paper, you might expect the GAN to roughly produce lines and blobs of the right colour on a white background. Such output might bring to mind "a bit like a child's drawing", but it will not be convincing or realistic, and you won't be able to analyse what is represented. – Neil Slater Sep 13 '18 at 12:09

A generative adversarial network is probably not the best approach for generating the images desired. We can assume from the comments that the data is not collected. That's a good thing, because a set of rasterized images, labeled with student age or grade is an inferior input form.

It appears that access to a student population is planned or already negotiated, which is also good.

Although the drawing, as it is being drawn, is seen through each student's eyes, the primary features correlated with drawing skill development is motor control, shape formation, and color choice. If the sheet of paper is placed over a drawing tablet, the tablet's incoming USB stream events are captured to a file, and the color selection is somehow recorded or automatically determined by having students hold the pencil or crayon up to the computer's camera before using it, a much better in natura input stream can be developed.

Pre-processing can lead to an expression of each drawing experience as a sequence of events arranged in temporal order with the following dimensions for each event.

• Relative time from the instruction to draw in seconds
• Color
• Nearest x grid
• Nearest y grid
• Pressure

Determining color from camera input may be developed using LSTM approaches.

The dimensions of the label for each of these sequences would be those demographics and rankings that would most closely correlated with developmental stages.

• Student age
• Student gender
• Curriculum grade (-1, 0, 1, 2, ... 12, where -1 is preschool and 0 is kindergarten)
• Identifier of the drawing instructions given to the class
• Grade ranking of the student in the class

The micro-analysis attached to each ELEMENT in the sequence includes these additional dimensions.

• Drawing rate of the utensil given by $$r = \frac {\sqrt{(x - x_p)^2 + (y - y_p)^2}} {t - t_p}$$ where the subscript p indicates the values are drawn from the previous event in the sequence.
• Drawing direction given by $$\theta = \arctan (x - x_p, \; y - y_p)$$
• Curvature $$\kappa$$ calculated using cubic splines or some other data fitting approach
• FFT spectrum $$\vec{a}$$ and Lyapunov exponent $$\lambda$$ applied to auto-correlation results

This is a modification of the system Google uses to synthesize speech, based on the WaveNet design. In the diagram, the residual function is defined as follows.

$$z = \tanh \, (W_{f,k} x + V_{f,k} y) \, \odot \, \sigma \, (W_{g,k} x + V_{g,k} y)$$

The development required is that the $$\vec{a}$$ must now be accompanied with scalars $$r, \theta, \kappa, and \lambda$$, but the resulting drawings are likely to have many of the hand-eye developmental features of the examples.

• Sir @Douglas, can I ask for a translation of your answer in simple terms or maybe Python code? I'm having trouble understanding the text since I'm neophyte on the field. – Sidney Guaro Oct 21 '18 at 8:00

$\$

Let's assume 5-year-old children.

You have numbers of pictures that drew by them. (Let, these pictures are your [training set].)

And, you want to synthesis similar pictures with the training set.

Because you need more pictures for your study.

Am I right?

$\$

OK.

From the pictures, you want to extract some meaningful information about a real child who drew them, right?

Then, I think GAN is not faithful for your study.

Of course, GAN can make very similar pictures with your training set.

However, it does not mean the synthesized images can contain the things what you want!

GAN just synthesizes "fake pictures" that cannot be distinguished from your training set.

The synthesized pictures may do not have any meaningful thing.

Because it is not drawn by the real child.

$\$

But it is worth to do.

May GAN captures some features of "children-like". (But I think it is too hard.)

You can find lots of GAN for your research, especially DCGAN.

• I want to use GAN to generate a generalized image for a certain age (4-12). Just one per age. That image will be used for manual analysis. – Sidney Guaro Sep 13 '18 at 13:30