I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)

I understand basic principle: Look if model overfit or underfit - according change the parameters. In theory. I would ask for help with two graphs:

  1. If I understand correctly what is going on
  2. How would you attack the situation.

1. Graph

enter image description here

  • I use LightGBM
  • learning_rate = 1
  • max_depth = 3
  • num_leaves = 2**15,
  • number of iterations = 4000

If I understand this model is Underfitting. The validation and training is falling, but not very much... BUT: The number of iteration is now too large and place higher number is not ok. Learning rate is 1 (as hight as it gets). Only the max_depth is low, but if it is higher (I try 30) the graph is same, just the values are worse.

So, what to do, so that model would not underfit.

  1. Graph

enter image description here

  • I use Neural Nets
  • epochs=200,
  • batch_size=64

The model

i = Input(shape=(100,), name='input')
x = Dense(128)(i)
x = Dense(64)(i)
o = Dense(1, activation='relu', name='output')(x)

Here I am not sure. This doesn't really looks like underfit, but more that doesn't converged. Is this right?

So, should I create more complex model (more neurones or more layers?)?

And how much epochs do I need to see this behaviour? Because, in the beginning I use only 10 epochs, for faster development, and I thought that model is overfitting. Only when I use more epochs I see that I was wrong.

How would you start to "debug" this neural net? What would be the plan of attack?



You must log in to answer this question.