# Heavy loss and inaccurate answer in pytorch

As my first AI model I have decided to make an AI model to predict multiplication of two numbers EX - [2,4] = [8]. I wrote the following code, but the loss is very high, around thousands, and it's very inaccurate. How do I make it more accurate?

import torch
import torch.nn as nn
import torch.nn.functional as F

data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)
values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)
lossfun = torch.nn.MSELoss()
model=Net()

class Net(nn.Module):
def __init__(self):

super(Net,self).__init__();

self.fc1 = nn.Linear(in_features=2,out_features=3)
self.fc2 = nn.Linear(in_features=3,out_features=6)
self.out = nn.Linear(in_features=6,out_features=1)

def forward(self,x):
x = self.fc1(x)
x = F.relu(x)
x = self.fc2(x)
x = F.relu(x)
x = self.out(x)
return x
for epoch in range(1000):

y_pred=model.forward(data)

loss = lossfun(y_pred,values)

print(loss.item())

loss.backward()

optim.step()


Note: I am a newbie in AI and ML.

• To help you solve this code, its better you create question in StackOverflow (stackoverflow.com). – Guilherme IA Mar 27 '19 at 17:52

There are a few things you could do to improve this NN, but are probably worth covering in different questions.

Your main problem though is that you forgot to reset the gradient after each training batch. You need to call optim.zero_grad() in order to do this, at the start of each training loop. Otherwise, using PyTorch, the gradient values keep accumulating inside the model's training data (sometimes you want this effect if you are adding gradients from multiple sources, that's why PyTorch is not clearing them automatically for you).

In addition, a learning rate of 0.5 is very high for the Adam optimiser - it is very common to leave it at the default value because Adam is an adaptive optimiser that will adjust step sizes depending on gradients seen so far.

Here is a working version of your code:

import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
def __init__(self):

super(Net,self).__init__();

self.fc1 = nn.Linear(in_features=2,out_features=3)
self.fc2 = nn.Linear(in_features=3,out_features=6)
self.out = nn.Linear(in_features=6,out_features=1)

def forward(self,x):
x = self.fc1(x)
x = torch.relu(x)
x = self.fc2(x)
x = torch.relu(x)
x = self.out(x)
return x

data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)
values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)

lossfun = torch.nn.MSELoss()
model=Net()

for epoch in range(20000):