0
$\begingroup$

I've been trying to construct resnet50 architecture from scratch using pytorch for classification. After construction I've run training job on CIFAR10 torchvision dataset, in 20 epochs with lr of 0.01 Ive achieved 62% accuracy on validation dataset. Afterwards I've run training job using torchvision.models.resnet50 model on the same dataset which results in slight increase in accuracy, but after searching I've found multiple articles saying that resnet50 achieving accuracy of 90+% on fifth training epoch.
Training code

for i in range(num_epochs):
  train_loss = 0
  correct = 0
  total = 0
  
  net.train()
  for j, (image, label) in tqdm(enumerate(trainloader)):
    image = image.to(device)
    label = label.to(device)
    optimizer.zero_grad()
    out = net(image)
    loss = criterion(out, label)
    loss.backward()
    optimizer.step()
    train_loss += loss.item()
    _, predicted = out.max(1)
    total +=  label.size(0)
    correct += predicted.eq(label).sum().item()
  print('Loss: %.3f | Acc: %.3f%% (%d/%d)'% (train_loss/(j+1), 100.*correct/total, correct, total))

  test_loss = 0
  total = 0
  correct = 0
  net.eval()
  for l, (image, label) in tqdm(enumerate(testloader)):
    image = image.to(device)
    label = label.to(device)
    out = net(image)
    loss = criterion(out, label)

    test_loss += loss.item()
    _, predicted = out.max(1)
    total += label.size(0)
    correct += predicted.eq(label).sum().item()
  print('Loss: %.3f | Acc: %.3f%% (%d/%d)'% (test_loss/(l+1), 100.*correct/total, correct, total))
    

Model code, optimiser and criterion codes:

net = models.resnet50(num_classes = 10, pretrained = False)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=learning_rate,momentum=0.9, weight_decay=5e-4)

Datasetloading and transformation code:

    transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])



trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

If OHE of dataset labels could be the case? if not please guide be what's could cause low accuracy results

$\endgroup$
1
  • 1
    $\begingroup$ It would be also very informative, both for you and us if you could present learning curves, this could be helpful to estimate under or over fitting. $\endgroup$
    – GKozinski
    Commented Sep 15, 2022 at 19:56

1 Answer 1

2
$\begingroup$

The articles you mention likely meant pretrained resnet50. It can get to 85%+ accuracy in 5 epochs with Adam and 1e-3 learning rate indeed. You'd need to replace the last layer or use the timm wrapper to utilize this though.

However, resnet50 is not a perfect choice for images this tiny. Something like resnet18 or even smaller would perform better and faster with no pretraining.

On a minor note, you probably meant enumerate(tqdm(trainloader))

$\endgroup$
1
  • $\begingroup$ enumerate(tqdm(trainloader)) - you are right thank you! And yeah, I've use pertained resnet which gave me higher accuracy, btw how do you know that resnet50 is not suitable for 'tiny' images in cifar(I assume tiny you meant low resolution)? $\endgroup$
    – qvuer7
    Commented Sep 17, 2022 at 12:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .