2
$\begingroup$

I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the Java Neural Network Simulator, JavaNNS, program.

In one practice I have to build a neural network to use backpropagation on it.

I have created a neural network with one input layer with 12 nodes, one hidden layer with 6 nodes and one output layer with 1 node.

I'm using Kaggle's titanic competition data with this format following Dataquest course Getting Started with Kaggle for Titanic competition:

Pclass_1,Pclass_2,Pclass_3,Sex_female,Sex_male,Age_categories_Missing,Age_categories_Infant,Age_categories_Child,Age_categories_Teenager,Age_categories_Young Adult,Age_categories_Adult,Age_categories_Senior,Survived
0,0,1,0,1,0,0,0,0,1,0,0,0
1,0,0,1,0,0,0,0,0,0,1,0,1
0,0,1,1,0,0,0,0,0,1,0,0,1
1,0,0,1,0,0,0,0,0,1,0,0,1
0,0,1,0,1,0,0,0,0,1,0,0,0
0,0,1,0,1,1,0,0,0,0,0,0,0
1,0,0,0,1,0,0,0,0,0,1,0,0
0,0,1,0,1,0,1,0,0,0,0,0,0

If you want see the same data better in an Spreadsheet: enter image description here

But they preprocess the data to use it with linear regression and I don't know if I can use these data with backpropagation

I think something is wrong because when I run backpropagation in JavaNNS I get these data:

opened at: Sat Feb 17 17:29:40 CET 2018
Step 200 MSE:   0.5381023044692738  validation: 0.11675894327003862
Step 400 MSE:   0.5372328944712378  validation: 0.11700781497209432
Step 600 MSE:   0.5370386219557437  validation: 0.11691717861750939
Step 800 MSE:   0.5370348711919518  validation: 0.11696104763606407
Step 1000 MSE:  0.5369724294992798  validation: 0.11697568840154722
Step 1200 MSE:  0.5369697016710676  validation: 0.11665485957481342
Step 1400 MSE:  0.5370053339270906  validation: 0.11684215268609244
Step 1600 MSE:  0.5370121961199371  validation: 0.11670833992558485
Step 1800 MSE:  0.5370200812483633  validation: 0.11673550099633925
Step 2000 MSE:  0.5367923502149529  validation: 0.11675956129361797

Nothing changes, it is like it doesn't learn anything.

How many hidden layers does the network have with how many nodes on each hidden layer?

Maybe the problem is that the data have been prepared to be used in Linear regression and I using it with Backpropagation.

I have only created the neural network, I haven't implemented the backpropagation algorithm because it is already implemented in JavaNNS.

Improve this question
$\endgroup$
5
  • $\begingroup$ Can you please tell us which gradient descent algorithm you use and which learning rate? $\endgroup$ – Molnár István Feb 19 '18 at 15:21
  • $\begingroup$ @MolnárIstván The learning rate is 2.0 but I don't know which gradient descent algorithm is JavaNNS is using. $\endgroup$ – VansFannel Feb 19 '18 at 15:58
  • $\begingroup$ The data has been prepared in Dataquest tutorial to be used with Linear Regression. Can I use it with backpropagation? $\endgroup$ – VansFannel Feb 19 '18 at 16:19
  • $\begingroup$ Well, I pretty sure that 2.0 for learning rate is way too high. Try 1e-2, 1e-3 or 1e-4 (it differs which gradient descent algorithm you use) . Besides, I think you have some misunderstanding about backpropagation. It is for adjusting the weights in your network through the loss. Well of course the prepocessing can be bad in a lot of cases, but here I think it should work. $\endgroup$ – Molnár István Feb 19 '18 at 18:34
  • $\begingroup$ @MolnárIstván I have talke with my professor and he said me that there isn't enough data to train the network. With 12 inputs and only 718 cases, it only covers the 17% of the combinations. I don't understand why I have to use JavaNNS. The train data is part of a Kaggle competition and it works with linear regression. $\endgroup$ – VansFannel Feb 21 '18 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.