The output of randomly initialised neural networks will be random. I am not sure if I understand your question correctly but I would like to try and explain what OpenAI is trying to achieve with their RND Distillation approach.
Any re-enforcement or supervised "learning" algorithm such as a neural network (NN) with back-propagation, genetic algorithm, partial swarm etc requires a fitness function. The fitness function "directs" the algorithm to a solution.
The basic idea is that the algorithm is randomly initialised. The algorithm is run, and then parts of the algorithm are changed. In this case the weights of the NN.
A number of algorithms exist to update the weights, the most common back propagation / gradient decent as used in the paper. To update the weights the output of the NN is compared to the optimum output. Sometimes the optimum is known. Other times it is not. In the case of a game the optimum output is not known. It is assumed that the more points gained the better the NN does. In this case the fitness function or policy is the sum of points gained when playing the game. But gradient descent needs an error (i.e difference between output and expected output), which does not exist when playing a game, because you don't know what the points should be. However the NN could play one round of the game score some points, then run the NN again on the next move to determine a prediction of future points. The difference between the points gained and the prediction is then the error, which gradient descent can use. [Hope this makes sense]
This is fine for games like pong, a NN could play the game pong randomly and score point at each prediction, which will direct gradient descent into updating the weights to maximise points.
However, games such as Montezuma’s Revenge, requires a lot of complex actions before points are gained. If the points stay 0, how does gradient descent determine if the weights should be decreased or increased?
Instead, the authors initialise a second random neural network which remains the same for the duration of training. As mentioned above the output of a randomly initialised neural network is random.
The output of the actual NN, and world state is then passed through the random NN (RND), which will return a random value. The NN is then required to not only determine a game action but also determine the result of the RND.
The policy function is extended from just points to also predict the result of the RND. The greater the prediction error the more bonus points are allocated to the policy function. This encourages gradient descent to change the weights of the NN to explore unseen parts of the game as this increases the prediction error of the RND which increases points. At the same time gradient descent must update the weights of the NN to predict the RND.
I hope this answers your question.
In summary randomly initialised NN include multilayered perceptrons or convolutional networks will behave randomly with respect to their inputs. The RND network is randomly set once at the star of training, and used to basically remind the NN undergoing training that it has done this before do not do this again - i.e. rewards the NN for doing something new (exploration).
