It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. However, with AI being applied to so many fields, can we be sure that there exists a loss function for every task, such that upon reaching its global optima, it guarantees reasonably good/human-level performance in the task?
There is an existence of a loss/reward function for any task that can be evaluated, but this does not mean that function generalizes to any model.
In your prompt you mention "human-level performance", this assumes a metric such as accuracy, auc, precision, winning %, etc that humans were evaluated on for some task. Since it is possible to model any function with a boundless resource budget we can then model a function that reaches any form of theoretical optimum for that metric and some given domain.
Ofcourse there is issues with what I wrote above in an applicable sense:
- Infinite resource budget doesn't exist.
- Not all metrics can be easily calculated.
- We usually subset a domain to "train", but to find one that truly encompasses a domain is usually not feasible and requires clever data usage and handling.
So in practice we make simplifying assumptions that handle the above issues. Examples include finite-parameter deep learning models designed for efficient learning, differential metrics such as distance, and data augmentation. In doing so, it is possible to achieve cases where the global min/max over its parameter space performs poorly. Its the researcher/engineers job though to make this not the case.