How is a ResNet-50 used for deep feature extraction?

Question

I'm trying to implement the vehicle re-identification model described in https://arxiv.org/pdf/2004.06271.pdf.

My question focuses on Section 3.2 of the paper, which uses a ResNet-50 for deep feature extraction in order to generate discriminative features which can be used to compare images of vehicles by Euclidean distance for re-identification. It takes a 256x256x3 image as input.

My understanding of ResNet-50 is that its output is of the shape N, where N is the number of classes which an input image could be, and ground truth labels take the form of a one-hot encoding where the '1' value represents the node in the output layer which is associated with the given class.

I am therefore confused by the usage of ResNet-50 in a re-identification task in which the goal is to generate an array of discriminative features which can be compared by Euclidean distance. There is no discrete set of N classes, as the model should work on any of the infinite number of vehicles in the world.

What is the ground truth label in a ResNet-50 in the context of a re-identification task?

Answer 1

The authors use so-called embeddings, it's a form to represent the images in some meaningful vector form.

The procedure to get embedding as follows. First, keep in mind most of the popular convolutional net architectures starts with convolutional layers and then have few fully connected layers. Then do the following.

1. Train the full network with one-hot encoded labels as usual
2. Take away the last fully connected layer and use the values on the previous layer as a representation.

In the case of resnet50, you will get a 2048-value float vector. The property of a neural network is that semantically close images usually have close representation on the last layers and you could use euclidian distance to measure the similarity of images in some sense (not really, but it's another long discussion)

I don't know the paper, but I glimpsed through it. You could see in formula 4 the $x$ is embedding representation. Loss at formula 5 is cross-entropy after applying the last linear layer ($Wx + b$) and then softmax.

As a side note, you could skip the first step completely and use pretrained weights as a starting point. I.e. in pytorch you could take pretrained on ImageNet classification weights as follows

import torchvision.models as models
resnet = models.resnet50(pretrained=True)