Since you are looking at a single iteration and expect a meaningful change my guess is that you aren't training for long enough. Q-learning can take very long, for many environments it takes millions of iterations.
Context of the question
This is a link to the text cited in the question.
It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is "What is the difference between SeparableConv2D and Conv2D layers". This answer points to this excellent article by Chi-Feng Wang:
A Basic Introduction to Separable Convolutions
One of the suggestions in the accepted answer was SSD.
On their website, SSD mentioned a competitor, faster_rcnn.
faster_rcnn was deprecated in favor of Detectron.
Detectron was deprecated in favor of Detectron2.
Long live detectron2.
It looks pretty cool and powerful:
Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content?
The filters (aka kernels) are the learnable parameters of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-...
Since this a classification problem you will use a CNN preferably. Then you need to fix an architecture of the CNN like VGGNet or Resnet or Le-net. You can find details on architectures here- Neural Network Architecures. As a beginner you can use VGG 16. You can read about the architecure here- Medium.com blog on VGG 16.
which tools/tutorials i should look ...
Well probably the response is that previous approach was a little naive.
I managed to fave some interesting result with this kernel that allow me to have an accuracy of 0.969 and a validation accuracy of 0.931.
Model I used is based on ResNet50 with the following additional layers ( and the last one just for binary classification ):
After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a "Big Task agnostic CNN" to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read Big Self-Supervised Models are Strong Semi-Supervised Learners by Ting Chen, ...
No, transfer learning cannot be applied "between" different architectures, as transfer learning is the practice of taking a neural network that has already been trained on one task and retraining it on another task with the same input modality, which means that only the weights (and other trainable parameters) of the network change during transfer ...