0
$\begingroup$

How to deal with videos where the frame sizes are not the same frame to frame?

For example this video moves up and down and when it does, the video part of the screen has a different amount of pixels vertically.

How to deal with different frame sizes in a CNN?

Improve this question
$\endgroup$
1
  • $\begingroup$ ,Don't you think your question lacks something there? or it's me who isn't analyzing critically $\endgroup$ – quintumnia Apr 28 '17 at 10:41
1
$\begingroup$

Well, the easiest way in order to make different frame sizes work with a convolutional neural network, is to process them using a different operations such as scaling, and or cropping in order to make them the same size. I do not know of any way currently where different frame sizes can be inputted directly into the neural network, as in order to do that you would need to almost completely change the neural network architecture. If the frames have a small difference in size, you may be able to get away with scaling the frame to the dimensions of the largest frame. This also depends upon what kind of problem you are trying to solve. While this method would work if you are trying to classify an image into a couple categories, it would not work if your are trying to do tasks such as object location. If you are doing one of the latter tasks, you should be able to pad the image with blank pixels in order to make the frame you are trying to process the dimensions of the largest one. This would work quite well, especially for object location detection. Finally, another solution is to crop the larger images to the size of the smallest one. While this will work just as well as padding, if you need any image data from the very edges of the frame, this will not work very well. So in conclusion, there are three different methods you could apply. Me personally, I would go with padding because it does not remove any important image data, and it will work very well for tasks such as object location detection. If you are doing a classification task though, scaling would work also. Finally, you could crop the images, but this would have to be evaluated for your specific task.

Improve this answer
$\endgroup$
3
  • $\begingroup$ If you pad many of the training images, will the network learn to look for padding? $\endgroup$ – user3731622 Nov 9 '18 at 2:00
  • 1
    $\begingroup$ @user3731622 it depends upon how your training data is structured. If you have enough training data, and one class does not have a specific padding size to it, then you should not have any issues. $\endgroup$ – Aiden Grossman Nov 9 '18 at 3:35
  • $\begingroup$ Thanks. I've just come across Global Average Pooling (GAP). Instead of dense layers after CNN, I seem to recall someone suggesting GAP could be used to handle variable input image sizes for CNN. I'm not sure how this works yet, but thought I would share. Here is one useful link $\endgroup$ – user3731622 Nov 9 '18 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.