# Tag Info

0

I'd like to add some details to the Neil Slater's answer. In order to generate data, we want to find some unknown distribution. Since we do not know anything about a real distribution, we can approximate it using GAN. It was shown that optimizing the loss function of the original GAN is equivalent to minimizing Jensen-Shannon divergence between the real ...

1

SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178. $n$ is the number of instances and $d$ is the number of dimensions. I'm assuming that you have more instances than dimensions giving a complexity of $O(nd^2)$. Hopefully this ...

0

Why I have to set to real these fake images and what fake images are these? You set them to "real" label for the discriminator when training the generator, because that is the goal of the generator, to produce an output of 1 (probability of being a real image) when tested. Usually you will generate a new batch of generated images for this step in ...

1

Yes, this method of training a model is commonly known as online learning and specific learning algorithms have been designed for this purpose, such as, Stochastic Gradient Descent(SGD). As opposed to Batch Gradient descent, which computes gradients over the entire training set at each step, the SGD algorithm computes gradients for individual samples and ...

1

I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps. You can see how bi-directional RNN works from this video from Andrew NG. I got the image below from that video: For more clarity: So if you know how to backprop through a simple RNN, you should be able to do so for bi-...

0

You can calculate the memory requirement analytically, but it's still not going to beat physical test in practice as there are so many unknown variables in the system which can takes the GPU memory. Maybe tensorflow will decide to store the gradients, then you have to take into account the memory usage of it also. The way I do it is by setting the GPU memory ...

1

You might be able to glean what you want from Chapter 13 or Sutton & Barto's Reinforcement Learning: An Introduction, which deals with policy gradient algorithms, and includes pseudocode for a variety of agents based on linear approximation using softmax regression. From your description, you appear to be using - or should consider - softmax regression ...

1

The neural network will learn what we teach it, for example with that image only, after finish training, your model will hard to recognize humans with dark skin, glasses, big eyes, etc, the features that two annotated targets don't have. If your data is big enough, and contain all the feature of humans face, the result should be good. If not, I recommend a ...

1

If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven dataset distribution, we can observe those jumps/irregularities in the training curve. And yes, those jumps do mean it might've found something significant in ...

2

There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger rate will cover a broader range, while the increasingly lower rate then produces a less 'erratic' climb towards a maximum. If you only use a low rate, you risk ...

0

Check out Figure 6 in this paper: PyTorch Distributed: Experiences on Accelerating Data Parallel Training It breaks down the latency of the forward pass, the backward pass, the communication step, and the optimization step for running both ResNet50 and BERT on a NVIDIA Tesla V100 GPUs. From measuring the pixels in the figure, I estimated the times for the ...

-1

Normally you only have two classes along with a threshold probability. It's how systems like YOLO work.

Top 50 recent answers are included