0
$\begingroup$

I have a simple text classifier, with the following structure:

    input = keras.layers.Input(shape=(len(train_x[0]),))

    x=keras.layers.Dense(500, activation='relu')(input)
    x=keras.layers.Dropout(0.5)(x)
    x=keras.layers.Dense(250, activation='relu')(x)
    x=keras.layers.Dropout(0.5)(x)
    preds = keras.layers.Dense(len(train_y[0]), activation="sigmoid")(x)

    model = keras.Model(input, preds)

When training it with 300,000 samples, with a batch size of 500, I get an accuracy value of .95 and loss of .22 in the first iteration, and the subsequent iterations are .96 and .11.

Why does the accuracy grow so quickly, and then just stop growing?

$\endgroup$
  • $\begingroup$ By any chance you are running it on a Jupyter notebook, can you share more of your code, especially about the training part ? $\endgroup$ – Semih Korkmaz Sep 2 at 20:44
  • $\begingroup$ I'm not. I'm new to this. I don't even know what a jupyter notebook is :P $\endgroup$ – hjf Sep 2 at 21:03
  • 3
    $\begingroup$ Without more information I don't think we can help you. A few things I can thinkg of: First of all, are you sure you mean iteration and not epoch? Secondly, are your data balanced (i.e. same number of samples in each class)? If not it is possible, if let's say 95% of your samples belong to a single class that your model is predicting this class only and achieving 0.95 accuracy. Finally, it is possible that it is just a very simple task and your model actually scores so high. $\endgroup$ – Djib2011 Sep 2 at 21:21
4
$\begingroup$

It actually depends on a couple of things here -

  1. How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.
  2. As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes (95%), then this accuracy you see makes sense but note that your model won't do well in production.
  3. Do not try to evaluate your model on the basis of your training accuracy. Test it on data your model has never seen before and then evaluate the classification accuracy making sure that your training/testing data is not heavy with one of the classes.
$\endgroup$
1
$\begingroup$

As you have trained your model in batch_size of 500. Weights has been updated for each batch therefore 600 times(300000/500) by the end of one epoch.
So, Your model generalized well. Check the predictions. If Predictions are well. Your model is ready.

$\endgroup$
1
$\begingroup$

It can be normal and there might be nothing wrong with your model. If there is a very strong and clear correlation in your data(good separability) then a network can achive very high accuracy very fast. After reaching some value learning gets harder.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.