Well, the easiest way in order to make different frame sizes work with a convolutional neural network, is to process them using a different operations such as scaling, and or cropping in order to make them the same size. I do not know of any way currently where different frame sizes can be inputted directly into the neural network, as in order to do that you would need to almost completely change the neural network architecture. If the frames have a small difference in size, you may be able to get away with scaling the frame to the dimensions of the largest frame. This also depends upon what kind of problem you are trying to solve. While this method would work if you are trying to classify an image into a couple categories, it would not work if your are trying to do tasks such as object location. If you are doing one of the latter tasks, you should be able to pad the image with blank pixels in order to make the frame you are trying to process the dimensions of the largest one. This would work quite well, especially for object location detection. Finally, another solution is to crop the larger images to the size of the smallest one. While this will work just as well as padding, if you need any image data from the very edges of the frame, this will not work very well. So in conclusion, there are three different methods you could apply. Me personally, I would go with padding because it does not remove any important image data, and it will work very well for tasks such as object location detection. If you are doing a classification task though, scaling would work also. Finally, you could crop the images, but this would have to be evaluated for your specific task.