1
$\begingroup$

As my first AI model I have decided to make an AI model to predict multiplication of two numbers EX - [2,4] = [8]. I wrote the following code, but the loss is very high, around thousands, and it's very inaccurate. How do I make it more accurate?

import torch
import torch.nn as nn
import torch.nn.functional as F

data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)
values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)
lossfun = torch.nn.MSELoss()
model=Net()
optim = torch.optim.Adam(model.parameters(),lr=0.5)

class Net(nn.Module):
    def __init__(self):

        super(Net,self).__init__();

        self.fc1 = nn.Linear(in_features=2,out_features=3)
        self.fc2 = nn.Linear(in_features=3,out_features=6)
        self.out = nn.Linear(in_features=6,out_features=1)

    def forward(self,x):
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        x = F.relu(x)
        x = self.out(x)
        return x
for epoch in range(1000):

    y_pred=model.forward(data)

    loss = lossfun(y_pred,values)

    print(loss.item())

    loss.backward()

    optim.step()

Note: I am a newbie in AI and ML.

$\endgroup$
  • $\begingroup$ To help you solve this code, its better you create question in StackOverflow (stackoverflow.com). $\endgroup$ – Guilherme IA Mar 27 at 17:52
1
$\begingroup$

There are a few things you could do to improve this NN, but are probably worth covering in different questions.

Your main problem though is that you forgot to reset the gradient after each training batch. You need to call optim.zero_grad() in order to do this, at the start of each training loop. Otherwise, using PyTorch, the gradient values keep accumulating inside the model's training data (sometimes you want this effect if you are adding gradients from multiple sources, that's why PyTorch is not clearing them automatically for you).

In addition, a learning rate of 0.5 is very high for the Adam optimiser - it is very common to leave it at the default value because Adam is an adaptive optimiser that will adjust step sizes depending on gradients seen so far.

Here is a working version of your code:

import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):

        super(Net,self).__init__();

        self.fc1 = nn.Linear(in_features=2,out_features=3)
        self.fc2 = nn.Linear(in_features=3,out_features=6)
        self.out = nn.Linear(in_features=6,out_features=1)

    def forward(self,x):
        x = self.fc1(x)
        x = torch.relu(x)
        x = self.fc2(x)
        x = torch.relu(x)
        x = self.out(x)
        return x

data = torch.tensor([[2,4],[3,6],[3,3],[4,4],[100,5]],dtype=torch.float)
values = torch.tensor([[8],[18],[9],[16],[500]],dtype=torch.float)

lossfun = torch.nn.MSELoss()
model=Net()
optim = torch.optim.Adam(model.parameters(),lr=0.001)

for epoch in range(20000):
    optim.zero_grad()
    y_pred=model.forward(data)
    loss = lossfun(y_pred,values)
    if (epoch % 1000 == 0):
        print(loss.item())
    loss.backward()
    optim.step()

This version can be tweaked to quite easily reach 0 loss for your data set.

This has not in really learned how to multiply two values. The approximation to multiplying will be very weak as there is very little data. However, playing with some very basic data and a simple NN is a first step towards understanding details like this . . .

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.