I made a simple feedforward neural network (FFNN) to predict $x$ from $\sin(x)$. It failed. Does it mean the model has overfitted? Why doesn't it work?

enter image description here

Var3 <- runif(500, 0, 20)
mydata3 <- data.frame(Sin=sin(Var3),Var=Var3)
winit <- runif(5500, -1, 1)
#hidUnit <- c(9,1)
nn3 <-neuralnet(formula = Var~Sin,data = mydata3,
                hidden =c(4,2,1),startweights =winit,
              learningrate = 0.01,act.fct = "tanh")

plot(mydata3, cex=2,main='Predicting x from Sin(x)',
     pch = 21,bg="darkgrey",
points(mydata3[,1],predict(nn3,mydata3), col="darkred", 

legend("bottomleft", legend=c("true","predicted"), pch=c(21,21),
       col = c("darkgrey","red"),cex = 0.65,bty = "n")
  • $\begingroup$ Im not familiar with r code. you are just feeding the x value into the network and adding tanh function to the hidden layers in the network am i right ? Christopher Bishop's machine learning and pattern recognition textbook has an example of sin(x) function I believe. However, he tries to fit sin(x) with a polynomial function. $\endgroup$ – calveeen Dec 13 '20 at 16:01
  • $\begingroup$ it's the other way around! feeding the network sin x and try to predict x. $\endgroup$ – homa taha Dec 13 '20 at 23:49
  • 1
    $\begingroup$ Any loss function metrics that you can report ? Eg. training loss $\endgroup$ – calveeen Dec 14 '20 at 0:54

The labels are not unique for the input domain [0,20]. Think about sin(x)=0, x could be 0, pi, 2*pi, 3*pi, ..., n*pi, all are correct from a mathematical point of view, but this is not reflected in your MSE loss. At this point your NN has to guess the correct label from your input data. Predicting the mean of your input data is the safest bet for the network.

In essence you're trying to build a arcsin function with a NN. If only consider x values in [-0.5*pi,0.5*pi], the labels are unique and your network should work.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.