# Why do I get a straight line as an output from a neural network?

I am using feedforward neural network for regression and what I get as a result of prediction is a constant value visible on the graph below:

Data I use are typical standardised tabular numbers. The architecture is as follows:

model.add(Dropout(0.2))

reduce_lr = ReduceLROnPlateau(
monitor='val_loss',
factor=0.9,
patience=10,
min_lr=0.0001,
verbose=1)

tensorboard = TensorBoard(log_dir="logs\{}".format(NAME))

history = model.fit(
x_train,
y_train,
epochs=500,
verbose=10,
batch_size=128,
callbacks=[reduce_lr, tensorboard],
validation_split=0.1)


It seems to me that all weights are zeroed and only constant bias is present here, since for different data samples from a test set I get the same value, but I am not sure.

I understand that the algorithm has found smallest MSE for such a constant value, but is there a way of avoiding such situation, since straight line is not really good solution for my project?

• Have you tried experimenting with a lower learning rate as a starting point? You're starting with 0.1 which is quite high for most of the cases, and only reduce by 0.9 which is not much. – razvanc92 Aug 27 '19 at 7:31
• I have now and it worked out! Lowering initial learning rate to 0.005 eliminated straight line as an output. You can post it as an answer :) Thanks! – Makintosz Aug 27 '19 at 8:18
• and if you have some idea why this happens so, with greater learning rate, please do share as well. – Makintosz Aug 27 '19 at 8:26