2
$\begingroup$

I have a neural network with 2 inputs and one output, like so:

input    | output
____________________
 a    | b   |  c       
 5.15 |3.17 | 0.0607
 4.61 |2.91 | 0.1551

etc..

I have 75 samples and I am using 50 for training and 25 for testing.

However, I feel that the training samples are not enough. Because I can't provide more real samples (due to time limitation), I would like to train the network using fake data:

For example, I know that the range for the a parameter is from 3 to 14, and that the b parameter is ~65% of the a parameter. I also know that c is a number between 0 and 1 and that it increases when a & b increase.

So, what I would like to do is to generate some data using the above restrictions (about 20 samples). For example, assume a = 13 , b = 8 and c= 0.95, and train the network with these samples before training it with the real samples.

Has anybody studied the effect of doing this on the neural network? Is it possible to know if the effect will be better or worse on the networks? and are there any recommendations/guidelines if I want to do this?

$\endgroup$
  • 3
    $\begingroup$ Welcome to ai.se..If you can generate data for an NN using a formula then why use NN in the first place..Use the formula to to get answer..And NN learns what you throw at it..So if you generate data using a formula it'll learn the formula that's all...So your problem.becomes redundant $\endgroup$ – DuttaA Apr 1 '18 at 13:37
  • $\begingroup$ @DuttaA there is no clear formula to get from the input to the output. I am trying to assume fake parameters with some restrictions to make the weights less random $\endgroup$ – Mhmd Apr 1 '18 at 13:44
  • $\begingroup$ @Dutta what I mean is, the numbers generated are not necessarily correct numbers but they follow the general restrictions that I have. $\endgroup$ – Mhmd Apr 1 '18 at 13:51
  • $\begingroup$ Well you are still not pulling out values from.air...You are following a rule and the NN will also follow the same rule that's all. $\endgroup$ – DuttaA Apr 1 '18 at 13:52
  • $\begingroup$ @DuttaA yes I see. I will be training the network with real data after that. I want to know if using generated data for pretraining can have a negative effect (vs using truly random data). $\endgroup$ – Mhmd Apr 1 '18 at 13:57
2
$\begingroup$

Yes, this is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise.

You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless, you know some correlation between the inputs a,b and the output c you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data you might want to consider a statistical model, rather than a neural network.

$\endgroup$
  • $\begingroup$ Thank you. Is there any way I can know what a minimum number of samples should be? I used to use a rule of thumb that the examples should be at least 10 times the degree of freedom. $\endgroup$ – Mhmd Apr 1 '18 at 17:28
  • $\begingroup$ How can you predict there will always be a negative effect? Dampening is quite good in case of over fitting data...So if we have a Very noisy dataset we can find the trend using stat methods, generate new data and it may actually benefit the prediction $\endgroup$ – DuttaA Apr 1 '18 at 18:05
  • $\begingroup$ (@DuttaA) That was bad writing on my part, I rewrote the first sentence to address your first point. The second part to your comment is what I was alluding to in my last sentence. So I think we are in agreement? Thanks for the edit, let me know if I need to address anything else. $\endgroup$ – Andrew Butler Apr 1 '18 at 18:51
  • 1
    $\begingroup$ (@Mhmd) I use a slightly different method that I was taught in school. Train your model on 80% of your training set and see how much this hurts your performance. If you have enough data you won't see much of a difference between 80% and 100% of your training set. If you don't have enough data you performance will suffer. $\endgroup$ – Andrew Butler Apr 1 '18 at 18:55
  • 1
    $\begingroup$ OP has a correlation that says the points are increasing as a and b increase. Sure we could generate data that fit the rules provided and this would bring the error rate down, but error rate isn't the end all be all. Say we want to detect fraudulent transactions and assume 99% of transactions are non-fraudulent. A model could get 99% precision on predicting non-fraudulent transactions by always predicting transactions to be non-fraudulent. This is essentially what we would be doing by generating values. We'd be lowering the error rate by bounding our output, but lose a lot of predictive power. $\endgroup$ – Andrew Butler Apr 1 '18 at 20:17
1
$\begingroup$

If you add fake samples to the training set, your Neural Network learns new dataset that you just made, your fake samples are estimations so you add noise to your training set.

you can use Leave one out cross validation technic for evaluating your model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.