2
$\begingroup$

I have a neural network with 2 inputs and one output, like so:

input    | output
____________________
 a    | b   |  c       
 5.15 |3.17 | 0.0607
 4.61 |2.91 | 0.1551

etc.

I have 75 samples and I am using 50 for training and 25 for testing.

However, I feel that the training samples are not enough. Because I can't provide more real samples (due to time limitation), I would like to train the network using fake data:

For example, I know that the range for the a parameter is from 3 to 14, and that the b parameter is ~65% of the a parameter. I also know that c is a number between 0 and 1 and that it increases when a & b increase.

So, what I would like to do is to generate some data using the above restrictions (about 20 samples). For example, assume a = 13 , b = 8 and c= 0.95, and train the network with these samples before training it with the real samples.

Has anybody studied the effect of doing this on the neural network? Is it possible to know if the effect will be better or worse on the networks? Are there any recommendations/guidelines if I want to do this?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

This is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise.

You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless you know some correlation between the inputs a, b and the output c, you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data, you might want to consider a statistical model, rather than a neural network.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you. Is there any way I can know what a minimum number of samples should be? I used to use a rule of thumb that the examples should be at least 10 times the degree of freedom. $\endgroup$
    – Mohammad
    Apr 1, 2018 at 17:28
  • 2
    $\begingroup$ (@Mhmd) I use a slightly different method that I was taught in school. Train your model on 80% of your training set and see how much this hurts your performance. If you have enough data you won't see much of a difference between 80% and 100% of your training set. If you don't have enough data you performance will suffer. $\endgroup$
    – Andrew Butler
    Apr 1, 2018 at 18:55
  • $\begingroup$ Actually we know that generalizing and creating random samples is bad..I was looking for the particular answer for the correlation op has presented $\endgroup$
    – user9947
    Apr 1, 2018 at 19:00
  • $\begingroup$ Yes, we know it is bad, but we don't know if it will have a negative effect on the resulting networks. It could happen by chance that we create a set of perfectly correct examples. $\endgroup$
    – Andrew Butler
    Apr 1, 2018 at 20:17
  • 1
    $\begingroup$ OP has a correlation that says the points are increasing as a and b increase. Sure we could generate data that fit the rules provided and this would bring the error rate down, but error rate isn't the end all be all. Say we want to detect fraudulent transactions and assume 99% of transactions are non-fraudulent. A model could get 99% precision on predicting non-fraudulent transactions by always predicting transactions to be non-fraudulent. This is essentially what we would be doing by generating values. We'd be lowering the error rate by bounding our output, but lose a lot of predictive power. $\endgroup$
    – Andrew Butler
    Apr 1, 2018 at 20:17
1
$\begingroup$

If you add fake samples to the training set, your Neural Network learns new dataset that you just made, your fake samples are estimations so you add noise to your training set.

you can use Leave one out cross validation technic for evaluating your model.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .