Skip to main content

Questions tagged [multi-task-learning]

For questions related to multi-task learning (MTL), which is a subfield of machine learning in which multiple learning tasks are solved at the same time, while exploiting commonalities and differences across tasks. MTL can result in improved learning efficiency and prediction accuracy for the task-specific models, when compared to training the models separately.

Filter by
Sorted by
Tagged with
1 vote
1 answer
25 views

Multi-task objective sometimes improve single-task performance, but is this true when fine tuning?

It is known that multitask objectives in neural networks sometimes have the effect of improving the performance of the neural network for each of the tasks individually (versus training the same ...
Alexander Soare's user avatar
0 votes
0 answers
17 views

Why is my multi-task-learning model not working?

I'm writing Python code to predict fetal head circumference using regression and classification together in a single model. The model will train to classify a fetal head image into a range (e.g., 50–...
NiStack's user avatar
0 votes
0 answers
34 views

Multi-Task VAE, Decoder Activation Functions?

I'm working on a Multi-Task VAE with one Encoder and two Decoders. The input consists of a vector with parameters which describe a design of a fluid system. The goal is to reconstruct the parameters ...
tekay's user avatar
  • 1
0 votes
0 answers
95 views

How to generate original training videos based on existing videoset?

I am a software engineer who is quickly ramping up on AI tech, but am nevertheless very new to the sector. A collegue has an extensive collection of training videos, the vertical is wheelchair seating ...
lukabloomrox's user avatar
3 votes
1 answer
1k views

Does ChatGPT use different transformers for different downstream tasks?

What I find hard to figure out is whether ChatGPT guesses from the prompt the downstream NLP task to be performed - text summary, text generation, question-answering, doing logic or arithmetic, ...
Hans-Peter Stricker's user avatar
0 votes
0 answers
45 views

References for the theory of pretraining and unsupervised learning to improve subsequent supervised learning

I am not sure if the title of this post uses the correct terminology, so suggestions are welcome. I have been following a lot of the ideas of using Pre-training methods on neural networks, to improve ...
krishnab's user avatar
  • 207
1 vote
1 answer
499 views

Instead of accumulating the gradient, can we accumulate loss values?

I have read and used Gradient Accumulation as a method to handle large batch size on smaller memory restrictions. It is described as following: ...
LSM's user avatar
  • 11
2 votes
1 answer
130 views

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
ChargeShivers's user avatar
2 votes
1 answer
3k views

What is the difference between multi-label and multi-task classification?

I am working on a data-set that has multiple labels associated with it (not necessarily independent of each other). During my development, I am confused if I should consider it as a multi-class ...
Payal Mohapatra's user avatar
2 votes
0 answers
90 views

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs? ...
user366312's user avatar
2 votes
0 answers
46 views

Is optimizing weighted sum multi objective tasks considered a multi-task learning?

I have two sequence prediction tasks, finding $\vec{\pi} \in \Pi$ and $\vec{\psi} \in \Psi$. Each sequence has its own objective function, i.e. $f_1(\vec{\pi})$ and $f_2(\vec{\psi})$. The input for ...
Sanyou's user avatar
  • 165
0 votes
1 answer
391 views

How should I incorporate numerical and categorical data as part of the inputs to the U-net for semantic segmentation?

I am using a U-Net to segment cancer cells in images of patients' arms. I would like to add patient data to it in order to see if it is possible to enhance the segmentation (patient data comes in the ...
Skyris's user avatar
  • 115
7 votes
1 answer
2k views

How to deal with losses on different scales in multi-task learning?

Say I'm training a model for multiple tasks by trying to minimize sum of losses $L_1 + L_2$ via gradient descent. If these losses are on a different scale, the one whose range is greater will dominate ...
SpiderRico's user avatar
  • 1,000
1 vote
0 answers
39 views

How do I format task features with a one-hot task identification vector to ensure separate weight matrices for each task in multi-task RL?

I am on Lecture 2 of Stanford CS330 Multi-Task and Meta-learning, and on slide 10, the professor describes using a one-hot input vector to represent the task, and she also explained that there would ...
iamPres's user avatar
  • 116