All Questions
718 questions
1
vote
1
answer
24
views
How do neural scaling laws explain the improvements from LSTMs to Transformer based models
I was reading about a study on neural scaling laws from 2017 and they noted this as a summary. From Hestness, Joel; Narang, Sharan; Ardalani, Newsha; Diamos, Gregory; Jun, Heewoo; Kianinejad, Hassan; ...
- 279
0
votes
0
answers
36
views
How to properly train a GAN model?
I tried to train a ViT-GAN model (from this repo) on my database, where i have images as input and output. The input image is a PNG map of a path planning problem. Red channel is obstacle map, green ...
0
votes
0
answers
37
views
When should you use a transformer and when LSTM, GRU and other Neural Networks?
There is a lot of information on the Internet that the transformer is better than RNN in everything, but is it true?
Examples:
«What if I need to translate words?»
«Generate text, images?»
«Play a ...
2
votes
2
answers
110
views
Are there any metrics suitable for evaluating conditional GANs in the context of medical imaging?
I am currently training a conditional GAN for breast cancer images with three classes: benign, normal, and malignant. While researching evaluation methods, I found that GANs are commonly assessed ...
0
votes
0
answers
20
views
How to train the simplest GAN? Uniform 1D noise to learn a Gaussian
I am starting a PhD, and we are trying to understand the simplest GAN so we can later use it for more complex goals. We want a GAN to learn to approximate a gaussian distribution from a 1D uniform ...
- 1
1
vote
1
answer
35
views
Does anyone use Statistical Energy to monitor generative AI training?
Statistical Energy (Szekely & Rizzo, 2013 or Aslan & Zech, 2005) can be used as a statistical test of whether two distributions are the same or different. It works particularly well on high ...
- 111
0
votes
0
answers
20
views
GAN performance is good when using train seed but bad for unseen seed random latent space vector
I trained GAN with fixed random seed such as 42, the model good at generating high quality training real samples.
When the model is predicted using unseen seed random latent space, the generated ...
0
votes
0
answers
21
views
How to build my own text-to-image or image-to-image with GAN or StableDiffusion and run locally?
When I say "build" is not from scratch of course, is get a model pre-trained, such like Stable Diffusion, and "train again" with another kind of dataset or change parameters to a ...
- 845
0
votes
2
answers
55
views
Is there any actual difference between these 2 definitions of a state value function?
The definition of the value function in TRPO paper is
\begin{align}
V_\pi(s_t) &= \mathbb{E}_{a_t,s_{t+1},\ldots} \left[ \sum_{l=0}^{\infty} \gamma^l r(s_{t+l}) \right], \\[10pt]
a_t &\sim \pi(...
- 139
1
vote
1
answer
20
views
"Window Pane" pattern when training CycleGAN
I've implemented a CycleGAN in Pytorch for style transfer between images. However, I've noticed during training that a distinct "window pane" pattern emerges, regardless of how I tweak ...
- 121
0
votes
1
answer
114
views
WGAN-GP loss never converging, terrible image quality
I'm a beginner in Deep learning area. I've been working on my WGAN-GP model to generate some radar images, and I'm using my own dataset which is relatively very small(1309 images with 128*128 pixels). ...
- 1
0
votes
1
answer
108
views
Does Machine Learning focus on discriminative AI while Deep Learning also focus on generative AI?
I know that Deep Learning is subset of Machine learning
But is it correct that classical ML algorithms mainly focus on implementing Discriminative AI while DL algorithms implement both Generative AI ...
- 181
1
vote
1
answer
25
views
Image Augmentation for Leaf Disease Detection: Training or Testing?
I am working on a leaf disease detection project and evaluating different strategies for augmenting the existing dataset to improve model performance. However, I am facing some confusion. Should I ...
- 43
0
votes
1
answer
30
views
Can a GAN Produce Different Inception Scores with the Same Dataset and Noise?
If the dataset, shuffle, and noise are all kept the same, is it possible for the same GAN to give different Inception Scores each time?
1
vote
1
answer
59
views
How long do i need to train a really DEEP network?
When we are training a really DEEP and complicated network (CYCLEGAN, VQ-GAN, VQ-VAE2), how to estimate required time (to be accurate "training steps") for training process?
Because from the ...
- 39
1
vote
0
answers
51
views
Why generative models produce mesh structure at the beginning?
I don't really understand the reason of this. I have listed the outputs of different models below.
Gan (source: self made simple GAN for CIFAR10)
Vq-vae (source: link)
What i mean is illustrated ...
- 39
1
vote
0
answers
62
views
How exactly do you backpropagate the gradient penalty in WGAN-GP?
I am trying to implement WGANs from scratch.
The loss function for the critic is given by :
which i implement in my code as L = average(real output) - average(fake output) + lambda*GP.
For ...
0
votes
2
answers
107
views
What do we mean by "AI is correlated"?
From Wikipedia
Causal AI is a technique in artificial intelligence that builds a causal model and can thereby make inferences using causality rather than just correlation. One practical use for ...
- 117
0
votes
1
answer
23
views
What is $z|y$ in Conditional Adversarial Nets?
I am currently going through Conditional Adversarial Nets (CGANs) and the modified objective function of the two-player minimax game is stated as follows:
$$\min_G \max_D V(D, G)=\mathbb E_{x\sim p_{...
0
votes
1
answer
98
views
How to interpret this training
I'm still learning ml/ai and I'm running a training where the curves look like this.
I was told that this looks good by some and that it doesn't look good by others... But none told me exactly why, I ...
- 103
0
votes
1
answer
24
views
Question about the redundance in DCGAN training
I don't understand the necessity of the redundance in the training of DCGAN. So a classical DSGAN training procedure is like this:
My questions:
Can I remove step (7)-(8) and and reuse the fake ...
- 101
2
votes
2
answers
556
views
Is it easier to use back-propagation or genetic algorithms to teach an artificial intelligence?
I am making a very simple neural network for a school project, and I would like to know what the best and easiest way to "teach" a neural network would be. From what I know, backpropagation ...
- 23
1
vote
0
answers
18
views
re-use D(fake) for optimizing both, G and D when training GANs
When training GANs, I can do this:
pseudo code
opt_g = Optimizer(G.params)
opt_d = Optimizer(D.params)
...
- 111
0
votes
0
answers
21
views
Are there leaderboards/tables/stats that compare inference times between close-sourced LLMs such as GPT 3.5/4 and Claude?
https://huggingface.co/spaces/optimum/llm-perf-leaderboard is great to compare inference times between LLMs but it misses close-sourced LLMs such as GPT 3.5/4 and Claude.
- 3,267
1
vote
1
answer
107
views
Why do you sample twice from the generator during a GAN training step?
Most basic GAN algorithm definitions I found go like this:
Generate
Train discriminator
Generate
Train generator
Like this one: GAN pseudocode
If I'm not misunderstanding, we sample twice from the ...
- 113
1
vote
2
answers
1k
views
What is the difference between densenet and resnet?
Is the only difference between the two how the skip connection is combined? Resnet combines skip connections through addition and Densenet through concatenating.
The Densenet paper appears to be ...
- 233
2
votes
1
answer
319
views
Why are the generator and discriminator designed differently in this example?
Why are the generator and discriminator designed differently in the example My First GAN of the coursera course: Build Basic Generative Adversarial Networks (GANs)?
Why didn't we use the same set of ...
- 393
0
votes
0
answers
81
views
What are the differences between Inception Score and Fréchet Inception Distance?
From the articles I've read about image generation using GANs, the Inception Score measures two things simultaneously: the variety of images (diversity) and the distinct quality of each image. Does ...
user77925
0
votes
0
answers
62
views
StyleGAN 2 multiplies loss components with zero, why?
I found a rather odd piece of code in a 3.8k star repo of the well known StyleGAN 2 paper.
In the loss function they use the following expression:
...
- 111
0
votes
1
answer
834
views
Why does the latent space in Stable Diffusion have a shape of 64x64x3?
Since the encoding is performed by a Variational Autoencoder, the VAE encoder must output some mean and log variance that we can ...
5
votes
1
answer
797
views
How can the discriminator determine the sample is fake or real?
Based on the articles I've read, the discriminator can identify whether a sample is fake or real. However, the articles don't clarify the conditions used to determine if a sample is fake or real. I ...
user79662
2
votes
1
answer
152
views
What are meaning of parameters $\theta$ in this context?
I'm reading the article about generative model from Open AI, here is the section where they explain them:
Our network is a function with parameters $\theta$, and tweaking these parameters will tweak ...
user79662
1
vote
1
answer
98
views
Is there any advantage of genetic algorithm (or programming) over Neural Networks? [closed]
I am planning to switch from neural networks to genetic algorithms (GA) and programming (GP).
One of the main hassles of working with neural networks is that it requires a large amount of training ...
- 339
1
vote
1
answer
198
views
Do I understand the technology of AI upscaling of films and cartoons correctly?
With the help of artificial intelligence, it is possible to increase the resolution of images that are initially low resolution, bringing it to ultra-high resolution. Also, initially static images are ...
- 239
0
votes
2
answers
61
views
Should I define my problem as image segmentation or detection?
I have a problem and have to decide wether it's an object detection or object segmentation problem. I want to use Yolov8 for training. We already have hundrets of images but they aren't labeled yet. ...
- 113
1
vote
1
answer
1k
views
When to use Pruning, Quantization , Distillation and others when optimizing speed
I want to understand how to optimize models for inference speed and am seeking some advice and best practices for the same.
I am a little bit aware of the concepts of pruning, quantization, and ...
- 789
4
votes
2
answers
3k
views
What are the differences between seq2seq and encoder-decoder architectures?
I've read many tutorials online that use both words interchangeably. When I search and find that they are the same, why not just use one word since they have the same definition?
user78615
1
vote
1
answer
133
views
Why are these two implementations of the $\epsilon$-greedy policy different?
According to the book Reinforcement Learning An Introduction, the epsilon greedy policy can generally implemented as:
$$
\pi(a|s) =
\begin{cases}
\frac{\epsilon}{|A|} + 1 - \epsilon & \text{if } ...
- 195
2
votes
1
answer
541
views
What are the similarities between Q-learning and Value Iteration?
This is the explanation of value iteration in our notes where you keep applying bellman optimality equation till it stops changing and then acting greedily wrt the value function gives the optimal ...
- 23
2
votes
0
answers
246
views
cGAN: Discriminator loss going to zero while Generator's going always up but the result is very good
I have a Conditional Generative Adversarial Network for Quantum State Tomography. The metrics I am monitoring during the training process are the losses and the Fidelity (the degree of similarity ...
- 23
1
vote
1
answer
115
views
Do GANs have constant running time?
After the model is trained, you just need to input random noise and the generator will output an image, does this mean GANs have constant running time ? I'm asking about both naïve GAN and variants of ...
user78202
1
vote
2
answers
260
views
The training process of a conditional GAN
For example, consider a dataset like MNIST. I give the conditional vector to produce only the number $7$ for both the generator and discriminator. In the following scenarios, what will the ...
user77925
0
votes
1
answer
387
views
In the conditional GAN (cGAN) architecture, why does the discriminator need conditional variable?
I'm reading about conditional GAN (cGAN) architecture, what I know is that the generator creates images combining both noise vector and conditional variable, the noise vector brings in random elements ...
user77925
0
votes
1
answer
1k
views
Generator loss not decreasing while training GAN
I’ve been attempting to create a basic GAN to generate images using this database of flowers (https://www.robots.ox.ac.uk/~vgg/data/flowers/102/).
I’ve spent a few days on this, and largely based my ...
1
vote
1
answer
48
views
Is there validation data in K-fold cross-validation?
We know that in machine learning the dataset is divided into 3 parts: training data, validation data and test data.
On the other hand, K-fold cross-validation is defined as follows:
the dataset is ...
- 1,115
0
votes
1
answer
39
views
Do WGAN gradients require multi-variable calculus?
The generator tries to maximise this function D(G(z)). That much I understand.
But how can the critic maximise D(x) - D(G(z)). ...
- 13
0
votes
1
answer
80
views
How to deal with sparse 1D data with WGAN-gp
I want to modify WGAN-gp so that it will work on very sparse 1D data (FFT of Gauss signal). Can you suggest any methods or papers that will be helpful?
I have a working WGAN-gp, but it is not ...
1
vote
1
answer
177
views
How the generator loss works in a GAN
I've been reading about GANs so I can implement a simple image generator. It seems that the loss for the generator is given by the following equation: ...
- 13
0
votes
1
answer
248
views
What is the difference between Machine Learning model, algorithm and hypothesis?
I'm fairly new to Machine Learning field and still to grasp the basics, so this question may seem very stupid, but what is the difference between Machine Learning model, algorithm and hypothesis?
Like ...
0
votes
0
answers
91
views
How to generate quality synthetic images of human subjects to be used for training stable diffusion
I'm trying to generate some synthetic images of humans, that are "good enough" to be used to fine-tune a stable diffusion model, but i'm not sure if this is possible.
I have experimented ...
- 101