No, the images do not need to be same. You can use different images for downstream task however you need to do some changes in model definition while loading model state_dict as CNN architecture used for pretext task 'relative patch location', assume Alexnet will expect 2 patches as input.
Heatmap in the sense of Corner net is the heatmap of the pooled corner values. As discussed in the paper, there is a corner pooling operation that gives you the vector values for a pixel point being a corner(it may or not). The output from the corner pooling is CxHxW. Then to generate a heat map you train the network similar to the Grad-CAM method. Training ...
This is a whole sub-field of reinforcement learning known as model-based reinforcement learning. The idea in model based RL is to learn the mapping from current state/action to next state in order to facilitate learning good policies.
If you are dealing with images as inputs I would recommend checking out the Dreamer papers. The most recent being this one.
The work was done through the following:
Extract feature points using a Detector
Extract the descriptor for these feature points
Make matching using similarity measure between two descriptors from two meshes.
I would look at table 1 of the original paper. While you're reading the alogorithm, try to really focus on Step 2 when you get to it.
In summary, each feature is used to train it's own classifier. So in your example, the calculated features X1, X2, ... Xn you describe coorespond to apply some set of feature transforms f_1, f_2, ... f_n to a single image. ...