I am very new to this pytorch and neural networks.I am stuck in training one model since last 1 week. My model paramters are not getting updated after each epoch. Also,
list(model.paramteres()[-1].grad) returns None
This is the code I wrote.
for epoch in range(10): running_loss = 0.0 print(list(net.parameters()).grad ) outputs = net(inputs.float()) a=torch.max(outputs,1).indices a=a.float() labels=labels.float() print(list(net.parameters()).grad ) loss = criterion(a,labels.squeeze(1)) loss = torch.tensor(loss, requires_grad = True) print(list(net.parameters()).grad ) optimizer.zero_grad() loss.backward() optimizer.step() print(list(net.parameters()).grad ) print('Finished Training')
also, this is the optimizer I have used,
import torch.optim as optim criterion = nn.BCELoss() optimizer = optim.Adam(net.parameters(), lr=0.001,weight_decay=0.0001) from torch.autograd import Variable
Basically, what I did is I tried to train the model on only one batch of batch size=16 in order to see if my weights are getting updated or not.And I found that my loss, and the weights all remain same for 100 iterations.I really can't figure out why is this happening.
I also have another question as well. My dataset contains grayscale images.And I need to perform a binary classification on them. This is the model I have made.It takes as an input tensor of shape (N,1,64,64) where N is the batch_size.
import torch import torch.nn as nn import torch.nn.functional as F class Net(nn.Module): def __init__(self): super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 16, 5) self.conv2 = nn.Conv2d(16, 32, 7) self.dropout1 = nn.Dropout2d(0.25) self.dropout2 = nn.Dropout2d(0.5) self.fc1 = nn.Linear(4608,128) self.fc2 = nn.Linear(128,16) self.fc3 = nn.Linear(16, 2) def forward(self, x): # Max pooling over a (2, 2) window x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) # If the size is a square, you can specify with a single number x = F.max_pool2d(F.relu(self.conv2(x)), 2) x = self.dropout1(x) # x = F.max_pool2d(F.relu(self.conv3(x)), 2) x = torch.flatten(x, 1) # flatten all dimensions except the batch dimension x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) x = self.dropout2(x) x =self.fc3(x) #print(x,torch.max(x)) return x net = Net() net = net.float() print(net)
This model returns a tensor of shape (N,2) with float type values in the range of [-1,1]. So, I took the index of the element with maximum value using
torch.max(outputs,1).indices and my labels were negative and positive. So, I used the encoding :
positive->1 .So,finally what goes to my
loss=criterion() is : the output that is the index with maximum value and the label with the encoding as specified above.I wanted to confirm if this is right or not.Since,this is the first time I am building a model and doing binary classification.I also tried not to use
torch.tensor(loss,requires-grad=True) but commenting this line of code brought up errors. I am actually not able to figure out why my weights are not getting updated and why the model is not getting trained. I do have a doubt that how to mark a label as 0 and the other as 1.Specially when ,they are compared to the indices of an array. Like instead of labelling positive as 0 if I mark it 1.Will that cause any problem?