It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. However, with AI being applied to so many fields, can we be sure that there exists a loss function for every task, such that upon reaching its global optima, it guarantees reasonably good/human-level performance in the task?
There is an existence of a loss/reward function for any task that can be evaluated, but this does not mean that function generalizes to any model.
In your prompt you mention "human-level performance", this assumes a metric such as accuracy, auc, precision, winning %, etc that humans were evaluated on for some task. Since it is possible to model any function with a boundless resource budget we can then model a function that reaches any form of theoretical optimum for that metric and some given domain.
Ofcourse there is issues with what I wrote above in an applicable sense:
- Infinite resource budget doesn't exist.
- Not all metrics can be easily calculated.
- We usually subset a domain to "train", but to find one that truly encompasses a domain is usually not feasible and requires clever data usage and handling.
So in practice we make simplifying assumptions that handle the above issues. Examples include finite-parameter deep learning models designed for efficient learning, differential metrics such as distance, and data augmentation. In doing so, it is possible to achieve cases where the global min/max over its parameter space performs poorly. Its the researcher/engineers job though to make this not the case.
As a more direct answer to your question as stated in the question title, finding a global minimum means we have found the optimal solution for that model. This doesn't mean that the model necessarily works particularly well. Hence the saying from George Box, "All models are wrong, some models are useful."
In other words, while you may have found an optimal state for that model, the model may not be optimal or even acceptable for the task. (Note that that phrase can be interpreted in different ways, and this is not the only way to apply it.)