This is an important subfield within multi-task learning, called gradient combination. Here is a list of about a dozen recent approaches: https://github.com/Manchery/awesome-multi-task-learning#loss--gradient-strategy
In particular, this paper is a good starting place: https://arxiv.org/abs/2001.06782
A brief summary of the work done on this topic: It is often better to analytically combine the gradients, than to directly combine the losses and then get one gradient. Many ways have been proposed to do this, some involving gradient norms, some involving dynamic rescaling of gradients, some involving geometric analysis, and some involving game theory.
I should also note: there is an entirely different approach to multi objective optimization called Pareto Optimality, where instead of simply combining the losses in one particular way, we analyze a set of different ways of combining the objectives and their trade-offs. For more info, this is a good starting point: https://en.wikipedia.org/wiki/Pareto_front