It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. However, with AI being applied to so many fields, can we be sure that there exists a loss function for every task, such that upon reaching its global optima, it guarantees reasonably good/human-level performance in the task?


There is an existence of a loss/reward function for any task that can be evaluated, but this does not mean that function generalizes to any model.

In your prompt you mention "human-level performance", this assumes a metric such as accuracy, auc, precision, winning %, etc that humans were evaluated on for some task. Since it is possible to model any function with a boundless resource budget we can then model a function that reaches any form of theoretical optimum for that metric and some given domain.

Ofcourse there is issues with what I wrote above in an applicable sense:

  1. Infinite resource budget doesn't exist.
  2. Not all metrics can be easily calculated.
  3. We usually subset a domain to "train", but to find one that truly encompasses a domain is usually not feasible and requires clever data usage and handling.

So in practice we make simplifying assumptions that handle the above issues. Examples include finite-parameter deep learning models designed for efficient learning, differential metrics such as distance, and data augmentation. In doing so, it is possible to achieve cases where the global min/max over its parameter space performs poorly. Its the researcher/engineers job though to make this not the case.

  • $\begingroup$ Thank you for your answer. I would like to clarify on a few points: 1. this does not mean that function generalizes to any model - Do you mean that not all models may be suited for this function? 2. possible to model any function - Can everything be expressed as a function? To give an example, I'm working on a navigation task, the network is learning a function that maps from image to action, but can we be sure that such a function exist, one that generates the correct action for all images? $\endgroup$
    – tangolin
    Jan 11 at 3:52
  • $\begingroup$ adding to it outliers would be hard to determine as you club number of task together which could affect all your performance metrics $\endgroup$ Jan 11 at 9:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.