New answers tagged deep-learning
2
The main benefit of deep learning is that you don't have to manually design features.
Classic Machine Learning algorithms always include the Feature engineering step, whereas neural networks are able to crate features automatically during learning. The classic example is CNN. In the first layer it creates simple features that representing lines, the last ...
2
Assume the image can contain objects of class $C_1 \dots C_c$. Assume a set of additional inputs that has a meaning of questions as "contains the image a C_i or C_j or ... ?".
The main problem for the system is classify the image in classes $C_i$. Second problem is answer the implicit question proposed by the remainder inputs.
Thus, better combine ...
1
Vanishing gradient is: as the gradient starts to flow from the end of the network (right side of the network) to the start of the network (left side of the network), it will be multiplied by numbers less than 1 and gradually it will become weaker and weaker and when it arrives to the first layers, it's so weak that makes almost no change in initial layers ...
1
If you use RNNs, then I think the solution is to use padding (zero padding) with max sequence length (that is the max number of words in a text) in order to tell your model to skip the zeros when possible. In that way, your model will try to learn a good representation of your input with fixed size. If you do not know this dimension, a solution may be to ...
-1
You might want to look at an encoder-decoder sequence to sequence model. This model allows you to input and output data with variable length.
1
The state space is certainly continuous, assuming that you can somehow feed that AI exact coordinates. You may have to resort to CNNs if you do not have access to this information. For the action space, you should consider how the game actually plays. Since you use a mouse to simply show the direction, you could use (x,y) positions of the mouse as an action, ...
2
is it common to deal with weights and biases in everyday tasks or in most of the cases existing algorithms do it well?
No; and it is no coincidence that you will not be able to find any reference to such a practice in any course or tutorial about neural networks. Such a practice would require a whole additional level of (business/SME) know-how in order to ...
1
I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps.
You can see how bi-directional RNN works from this video from Andrew NG. I got the image below from that video:
For more clarity:
So if you know how to backprop through a simple RNN, you should be able to do so for bi-...
0
You can calculate the memory requirement analytically, but it's still not going to beat physical test in practice as there are so many unknown variables in the system which can takes the GPU memory. Maybe tensorflow will decide to store the gradients, then you have to take into account the memory usage of it also.
The way I do it is by setting the GPU memory ...
0
I only have one good news... There is nothing wrong with your code. Neural networks tend to do that. Especially with a really complex function.
Increasing the amount of neurons will not decrease how the error is distributed.
There are better loss functions for each case but is not a really effective solution.
Neural networks are really good managing noise. ...
1
In a simple linear model of the form $y = \beta_0 + \beta_1 x $ we can see that increasing $x$ by a unit will increase the prediction on $y$ by $\beta_1$. Here we can completely determine what the effect on the models prediction will be by increasing $x$. With more complex models such as neural networks it is much more difficult to tell due to all the ...
-1
ML is full with things that supposed to work better (in theory).
Sigmoid function seems better than ReLu.
L1 seems way better than L2.
Spikes neural network seem to be better than standard neural network.
A shallow neural network with a lot of neurons has more parameters than a deep one with the same amount of neurons. So, in theory, has to be more ...
1
The problem you state is a well known problem, and it is called "keyword spotting" os KWS. If you add a wake up word before it (like "hey google/siri"), you can also use "voice command" system to alleviate the problem.
There are two kind of KWS systems: those which develop to detect a hard coded set of keywords, and those who ...
1
? This means that there are not promising versions of this algorithm fro regression until 2012.
After your question, I have found one of the survey research paper which is done or ensemple methods for regression. This table also extracted from this paper.
Read this paper, it will help you a lot more
This one is latest paper published on object detection with ...
0
In rehearsal, you do not necessarily train with all old training data, but you can just use some of it [1], which you add to your current (or new) training data.
In batch learning, at every epoch, you typically train with all training data, every step with a different batch (or subset) of the training data; so, if you have $N$ training examples and your ...
0
Check out Figure 6 in this paper: PyTorch Distributed: Experiences on Accelerating Data Parallel Training
It breaks down the latency of the forward pass, the backward pass, the communication step, and the optimization step for running both ResNet50 and BERT on a NVIDIA Tesla V100 GPUs.
From measuring the pixels in the figure, I estimated the times for the ...
0
What are examples of under-parametrization and over-parametrization in machine learning? [duplicate]
We have a parametric model and non-parametric models
a learning model that summarize data with a set of parameters of fixed size(independent of the number of training exmample) is called a parametric modeland if it couldn't do that we say non parametric model. The non-parametric model is good when you have a lot of data and no prior knowledge and when you ...
1
You are right. If you don't continuously train the neural network after you have deployed it, there is no way it can continuously learn or be updated with more information. You need to program the neural network to learn even after it has been deployed. There is no such thing as a neural network that decides what it does without a human deciding first what ...
1
So if I understand correctly, you're proposing to use a neutral net with $N$ input units (let's say data is in $\mathbf{R}^N$), 1 hidden unit, and whatever the necessary output needs to be.
Let's say we try to do this. Then each unit of the output layer is responsible for computing its output based on a single scalar input. So it's like as if you're saying ...
1
I think the best thing to use here is a form of "structured prediction". Our "target" is a sequence of operations. The framework of structured prediction allows us to chain together as many filters as we want.
With a neural network of fixed architecture, you would have to make sure you have enough space for all the filters you might need.
0
First of all, Keep in mind that maths operations aren’t the only thing that contribute to performance. Memory bandwidth can also be a factor.
And most importantly, we want to capture as much area as we can in lowest possible number of operations. So in 3x3 kernel case, we can capture 9 cells in one shot, but with 3x1 followed by 1x3, we have to compute 6 ...
Top 50 recent answers are included
