New answers tagged deep-learning
1
vote
Do GANs have constant running time?
The question is a bit ill defined... usually when we want some bound on the running time, we have to say with respect to what
For example:
sorting is O(nlogn) wrt the size of the input
Transformer is ...
- 1,308
1
vote
how to determine the number of units for dense layer for transfer learning?
In adition to @Alberto answer,
Start by the same number of layer and units you removed from original model.
If the problem is the same, this will be probably the best solution.
After that, if the ...
- 111
2
votes
how to determine the number of units for dense layer for transfer learning?
Not only the units but also the number of layers... you can reason over something like "how complex is your task", but usually we resort to grid search over some educated guesses (like 2/3 ...
- 1,308
1
vote
Accepted
The training process of a conditional GAN
I assume you mean how to label the image and class inputs since the discriminator can reasonably output either "real" or "fake" labels for either of those inputs, and you generally ...
- 30.2k
2
votes
Are foundation models something fundamentally new? Is there a proper definition?
A foundation model is any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks
from "On the ...
- 231
3
votes
Can mini-batches for stochastic gradient be balanced but not representative of the training data?
Though you've already accepted an answer, from your comments it seems you're still somewhat unclear and need an example. Take your example of (binary) classification, usually balanced training data ...
- 940
3
votes
Accepted
Can mini-batches for stochastic gradient be balanced but not representative of the training data?
The problem is, what mathematical definition do you give to "balanced" and "representative"?
Balanced usually means that classes are uniformly distributed, but then you are ...
- 1,308
0
votes
My model is only improving when learning rate is 1. Should I be worried?
Learning rate must be below 1 because if it were above 1 the output would only depend on the last input of the last training epoch , which cant be correct.
- 1
0
votes
My model is only improving when learning rate is 1. Should I be worried?
From an optimization POV, there is no problem, indeed, given a loss $L$ for which a good learning is $10^{-4}$, you can build a second loss equivalent which has $1$ as good learning rate, just by ...
- 1,308
1
vote
My model is only improving when learning rate is 1. Should I be worried?
A learning rate of 1 is indeed atypical in many machine learning contexts, as it may cause the model to converge too quickly and overshoot the global minimum.
Generally, a smaller learning rate within ...
- 11
1
vote
What causes my loss curve to consistently oscillate when training an LLM?
Based on the colab notebook that you used, the periodic oscillation of the training loss does not seem to originate from the learning rate scheduler which is set to "constant".
However, I ...
- 111
2
votes
the best choice to reduce a features vector
Feature selection -- the case in which the features are highly correlated is the prototypical case in which you want to select a subset of independent features that allows for an equal performance. ...
- 31
1
vote
Accepted
What is the potential issue of nested neural networks
Isn't this just a very short recurrent neural network? Same issues apply, although they are less severe since you aren't applying as many recurrent iterations. Once you start "nesting" them ...
- 737
1
vote
Accepted
In the conditional GAN (cGAN) architecture, why does the discriminator need conditional variable?
Because otherwise there is no conditioning... consider the case where you condition the generator but not the discriminator: given an image and a label, the generator proposes an image, which will be ...
- 1,308
3
votes
Accepted
Regression loss conditioned by the ground-truth values
Your suggestion should work to focus the ML more on larger angle examples.
You may want to try a slightly simpler approach of weighting the loss (or the resulting gradient) by a factor depending on ...
- 30.2k
0
votes
Why does Stable Diffusion use VAE instead of AE?
Stabilizing diffusion uses variational autoencoders (VAEs) instead of autoencoders (AEs) because VAEs allow for the generation of continuously distributed representations in the latent space, which ...
- 1
0
votes
Why does Stable Diffusion use VAE instead of AE?
To my knowledge, when it comes to stable diffusion, the VQ-VAE is the commonly used method. This differs slightly from vanilla VAE which assumes the encoded features to be a normal distribution and ...
- 1
1
vote
Accepted
Inquiry on Combining Two Neural Networks for unsupervised training: Has This Been Researched?
Based on my knowledge, there has not been an architecture that covers all the points you have described. Significant development has occurred in creating networks that utilize subnetworks with ...
- 453
Top 50 recent answers are included