0
$\begingroup$

I am trying to classify whether or not a specific object is in panoramic photos. The issue is, a panoramic photo can be any width, so the input to my neural network can't be fixed in that dimension.

I've been using RNNs (QRNNs to be specific, as I am trying to reduce the number of parameters as much as possible), but they always learn where the object usually is in the image, and then have a really hard time classifying an image with the object in a different place.

I'm looking for something similar to CNNs, where it doesn't have a spatial dependance (or in this case, a temporal dependance?), but it can't have a fixed input width.

Any ideas?

$\endgroup$
6
  • $\begingroup$ pretty sure you can just use a bunch Conv layers, then do something like a global average pooling and some dense layer for the classification $\endgroup$
    – Alberto
    Aug 31, 2023 at 14:34
  • $\begingroup$ As I said, I don't know the input size of my network, so conv layers are out. $\endgroup$
    – user55176
    Aug 31, 2023 at 14:46
  • 1
    $\begingroup$ You are probably missing something on how convolution works, as they do not at all rely on input size (width and height wise), you can definitely train convolutional layers without any knowledge of the input size of each image $\endgroup$
    – Alberto
    Aug 31, 2023 at 16:14
  • $\begingroup$ CNNs cannot be trained with variable input sizes unless you use padding, which won't work in my case. $\endgroup$
    – user55176
    Aug 31, 2023 at 16:19
  • $\begingroup$ Again, yes, you can, convolution does not depend on the width and heigh of your images, please try and you will see that it's indeed the case, of if you want to be more theoretical, check the definition of convolution, and you will see that they indeed can. $\endgroup$
    – Alberto
    Aug 31, 2023 at 17:11

1 Answer 1

2
$\begingroup$

Listen, this is not an answer to your question, but it seems that you are missing the whole point of convolution.

Simplified explanation: Convolution is just a weighted sum of the neighbors of a pixel

You see how this is not dependent on the size of the image?
Take a 3x3 filter, on a NxM image, apply convolution, and you will get a (N-2)x(M-2) image as output

Now take a TxS image, and apply the same filter over it, what you get (T-2)x(S-2)

you see now that you can apply a convolutional layer to any arbitrary sized image?

You still don't believe me? take this code, and you will see that you can input two images of different sizes to this neural network and it won't complain:

network = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, kernel_size=3, activation=tf.nn.leaky_relu, padding="SAME"),
    tf.keras.layers.Conv2D(32, kernel_size=3, activation=tf.nn.leaky_relu, padding="SAME"),
    tf.keras.layers.Conv2D(32, kernel_size=3, activation=tf.nn.leaky_relu, padding="SAME"),
    tf.keras.layers.Conv2D(32, kernel_size=3, activation=tf.nn.leaky_relu, padding="SAME"),
    tf.keras.layers.GlobalMaxPooling2D(),
    tf.keras.layers.Dense(NUM_CLASSES, activation="softmax"),
])

you're welcome :-)

$\endgroup$
4
  • $\begingroup$ You just proved me right. Applying a 3x3 kernel to a TxS image gives you a (T - 2)x(S - 2) output, which is the MATHEMATICAL DEFINITION of dependance. Regardless, this is not an answer to the question. Please stop. $\endgroup$
    – user55176
    Sep 1, 2023 at 1:16
  • 1
    $\begingroup$ This actually is an answer to the question (if not, it's not entirely clear what the real question is: indeed the title of the question doesn't seem to match the body of the question too clearly). @Ghull Yes, conv layers depend on the input size, in the sense that their output size depends on the input size. Bigger input size --> bigger output size. But that's great, that means they can handle different input sizes, which is exactly what you need. Just need to make sure to, after conv layers, have something (like pooling) that reduces everything to a single, fixed dim (num. classes). $\endgroup$
    – Dennis Soemers
    Sep 1, 2023 at 9:49
  • $\begingroup$ @Ghull seems like somebody agrees with me, the number of parameters of convolution is not dependent of the input size, so it's said not to be "dependent from input" as a fully connected layer for example. You can perfectly use the code I provided to solve your problem without any problem $\endgroup$
    – Alberto
    Sep 1, 2023 at 13:43
  • $\begingroup$ @DennisSoemers thank you $\endgroup$
    – Alberto
    Sep 1, 2023 at 13:44

You must log in to answer this question.