2
$\begingroup$

I don't know if I worded the title correctly.

I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.

Now, I have a neural network (Using keras with tf). My question is, is it better to train each file individually for X epochs (File 1 for 5 epochs, then File 2 for 5 epochs, etc.), or should I do an epoch for each, repeatedly (File 1 for an epoch, File 2 for an epoch, etc., and repeat for 5 times).

I've used the first and I get an accuracy of about 88%. Whould I get an improvement by doing the latter?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Small set for multiple epochs causes overfit, so expose entire data in one epoch, then augment (change a littel bit), and train again. Use small amount of dropout also.

I have a big dataset (300000 of images after augumentation) and I've splitted it into 10 parts, because I can't convert the images into a numpy and save it, the file would be too large.

Do not store augmented data, just do augment with function right after next epoch and overwrite the dataset array

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .