5
votes
Does ChatGPT use different transformers for different downstream tasks?
It is just one huge model which performs autoregressive text generation.
The ability to perform a wide variety of task, defined at inference time is called in-context learning and was introduced in ...
- 400
2
votes
Accepted
How should I incorporate numerical and categorical data as part of the inputs to the U-net for semantic segmentation?
What you want to do is called multi-task learning. Here's what you do:
Create a second Input.
Attach it to 1D CNN (2-3 layers), so it aggregates this tabular information.
Concatenate this feature ...
- 868
2
votes
Instead of accumulating the gradient, can we accumulate loss values?
Accumulating the loss like that doesn't improve the memory requirements, because the memory consumption depends on the size of your computational graph. In other words, each time you add a term to the ...
- 1,748
1
vote
What is the difference between multi-label and multi-task classification?
Is there any fundamental difference between the two?
The difference is in the names:
Multi task means that we are learning more than a single task, i.e. the labels we have will be used to compute ...
- 5,428
1
vote
How to deal with losses on different scales in multi-task learning?
I am currently working on a similar problem. I think your approach is good. As for setting the parameter lambda, since you are using deep neural networks, you can make it a learnable parameter, ...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible