I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-pooling) as feature vectors.
To configure the model accordingly i do:
from tensorflow.keras.models import Model from tensorflow.keras.applications import VGG16 base_model = VGG16(include_top=False) # Cut off the fully-connected-layers model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_conv3').output) # Discard the last max-pooling-layer model.layers[-1].activation = None # Change activation-function of last layer from Relu to Linear
and i get a feature-tensor of shape
(1, 14, 14, 512) for the default input-shape of
(224, 224, 3). So far, this looks the way i would want it to be. However, the authors state:
The VGG-M [equivalent to VGG16 in the context of the paper] convolutional features are extracted as the output of the last convolutional layer, directly from the linear filters excluding ReLU and max pooling, which yields a field of 512-dimensional descriptor vectors
Now, i have stated above, that the number 512 is part of my output-shape. However i thought, that it means that i get back 512 individual image-patches of size 14x14! The only way i could think of, that would get me 512-dimensional descriptor vectors would be something like this:
features = model.predict(img) feature_vectors =  for i in range(features.shape): for j in range(features.shape): feature_vectors.append(features[0, i, j, :]) feature_vectors = np.array(feature_vectors)
But then i would slice through all of the existing image-patches!
Did the authors mean to do just that? Is this a common practice anyone her has used before? All the tutorials or blogposts i found online just
flatten() the output-tensor and add it to the database of existing feature-vectors.
The authors also state, that:
...local descriptors are extracted at multiple scales, obtained by rescaling the image by factors $2^s, s=−3,−2.5,\dots,1.5$ (but, for efficiency, discarding scales that would make the image larger than $1024^2$ pixels).
I can totally extract images at different scales, by specifying the
input_shape, when instantiating the VGG16-model. My method of getting 512-dimensional feature-vectors stated above would also work in that case, even though the output-tensor could be much larger (i.e. a shape of
(1, 64, 64, 512)) in case of a bigger image.
Is this the way to do feature-extraction at multiple scales?