I'm intrigued by the idea of employing a separate neural network (which I'll refer to as the "loss network") to compute the loss for a primary network based on its inputs and outputs. The mechanism is as follows:

Primary Network:

  • Input: Any data sample (e.g., image, text).
  • Output: Predicted value or classification.

Loss Network (Evaluator):

  • Input: Inputs provided to the primary network and its corresponding outputs.
  • Output: Scalar loss value that evaluates the quality or suitability of the primary network's output given its input.

The computed loss would then guide the training of the primary network.

My questions are:

  1. Has there been research or experimentation with such an architecture in the deep learning community?
  2. If so, can anyone point me to relevant literature or papers that discuss this approach and its applications?
  3. Are there known challenges or limitations associated with this method?

I'm keen to understand if this concept has been explored and any insights the community might have. Thanks in advance!

  • $\begingroup$ This is somewhat similar to Generative Adversarial Networks where a generator network tries to fool a discriminator network that it's generations are "real'. $\endgroup$
    – SpiderRico
    Commented Oct 31, 2023 at 0:45


You must log in to answer this question.

Browse other questions tagged .