Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
I'm training an AutoEncoder-like network to take a face embedding, encode it, decode it, and then I calculate the loss between the input and output embedding. The input embedding is calculated by passing an image through a pre-trained frozen network. Is there a difference between pre-computing the embeddings from the frozen network before training and computing them during training to have gradients?