New answers tagged deep-learning
0
To emphasize (and this is not emphasized in this answer), in the case of neural networks, the biases or, more precisely, the connections (or weights) between biases and other neurons are also learnable parameters, so the back-propagation algorithm calculates a gradient of the loss function that contains the partial derivatives with respect to the connections ...
1
In a simple feed-forward network, each artificial neuron has a separate bias value. This allows for greater flexibility for the output layer function than if each neuron had to use a single whole-layer bias. Although not an absolute requirement, without this arrangement it may become very hard to approximate some functions. Moving from a bias vector to a ...
1
Mathematical Exploration
let $\Theta^+$ be the pseudo-inverse of $\Theta$.
Recall, that if a vector $\boldsymbol v \in R(\Theta)$ (ie in the row space) then $\boldsymbol v = \Theta^+\Theta\boldsymbol v$. That is, so long as we select a vector that is in the rowspace of $\Theta$ then we can reconstruct it with full fidelity using the pseudo inverse. Thus, ...
3
The method you propose is already known, its basically a numerical approximation to the gradient. It is not used to train neural networks because its well... an approximation. You still need to do two forward passes to get an approximation, which introduces noise and might make the training process fail.
Using backpropagation to compute the gradient is an ...
1
What's the input to the Generator?
In the basic implementation of GANs, the Generator only takes in a vector of random variables. This might seem strange, but after training, the generator can transform this input noise into an image resembling those of the training set.
How does it work?
It is trained along with its counterpart the Discriminator, whose ...
1
I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it worked.
So think about your input guys
0
It is quite common in DQN to instead of having the neural network represent function $f(s,a) = \hat{q}(s,a,\theta)$ directly, it actually represents $f(s)= [\hat{q}(s,1,\theta), \hat{q}(s,2,\theta), \hat{q}(s,3,\theta) . . . \hat{q}(s,N_a,\theta)]$ where $N_a$ is the maximum action, and the input the current state. That is what is going on here. It is ...
2
This question is very broad, so let me attempt to answer it using my own background in time series analysis.
As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account?
Well, an LSTM model is not ...
0
From what I understood in a classifier a common method is that you sample a mini-batch, calculate the loss for every example, calculate the average loss over the whole batch and adjust the weights w.r.t to average loss? (Please correct me if I'm wrong)
You are wrong.
The weights are adjusted w.r.t. to average gradient, and this must be calculated using ...
0
When you one-hot-encode your labels with $p_i \in \{0,1\}$ you get $p_i = 0$ iff $i$ is not correct and, equivalently, $p_i =1$ iff $i$ is correct.
Hence, $p_i \log(q_i) = 0 \log(q_i) = 0 $ for all classes except the "truth" and $p_i \log(q_i) = 1 \log(q_i) = \log(q_i) $ for the correct prediction.
Therefore, your loss reduces to:
$$
H(p,q) = - \sum p_i \...
3
I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS.
It's usually called the loss function (and, in general, objective function) and often denoted as $\mathcal{L}$ or $L$ (or something like that, i.e. it is not really important how you denote it). The specific function used as a ...
3
Welcome to AI Stack exchange!
You're right, as the network is initialised randomly, the resultant function is essentially impossible to get your head around. This is because most of the time the network has >4 dimensions (4 can be graphed with some effort and a lot of color), and as such is literally beyond human comprehension via graphing.
So what do we ...
1
Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models?
Of course, the model that achieves state-of-the-art performance depends on the problem, available datasets, etc., so a comprehensive comparison between traditional ML models and deep neural networks is not ...
0
Ideally, yes. Ideally, because the network should be fed with the words of an entire book (wich vary around 100k words). With an hypotetical amount of processing power, you should be able to just train the NN with like thousands of books.
It might be possible to be trained with quantum computers.... who knows...
For smaller stories, I think that the major ...
0
You would need to perform some kind of speech-to-text to get the audio transcription with the corresponding synchronization wrt the audio. Then search in the transcription.
You could use DSAlign by mozilla
1
If is a truly a random number, and you could guess each of the next successive five in sequence, then you could win the lottery consistently.
This is one of the first tasks many people try to do when first learning machine learning. If the lottery is truly a random physical process with fair, i.e., balanced ping pong balls, then you cannot predict which ...
2
The paper you are citing is the paper that introduced the cascaded convolution neural network. In fact, in this paper, the authors say
To realize 3DDFA, we propose to combine two achievements in recent years, namely, Cascaded Regression and the Convolutional Neural Network (CNN). This combination requires the introduction of a new input feature which ...
4
There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may perform badly in general (but those functions may still work in specific cases)
If you are using gradient descent as a training method, then differentiability ...
0
To my knowledge the deployment model (that you will test on underwater images) as inference will not have a negative effect. Yet drawings may even help differentiate some classes at training and inference. Provided that you won't use drawings in inference, adding them in training phase will not necessarily hurt the accuracy. Note that a drawing of a ...
4
This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...
3
No. Different batch sizes mean different gradients (check stochastic gradient descent concept you will get how loss calculated) are calculated in each step, and thus the gradient descent will likely end up in different places in parameter space.
In addition, how this is actually parallelized might make a difference, including the order of operations and ...
0
But because the inputs have to have a fixed length
Do they? Why? The go-to strategy would be to use an RNN (possibly with LSTM or GRUs, but probably not necessary) and train it to process input sequentially and output the final classification of the paragraph. This has the advantage of being able to take into account word order and constellations, as ...
2
Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original numbers back. However summing up a sequence of word vectors may work depending on your task. You should also normalize the values, or just use average value.
For ...
4
For the precision metric for example you have:
$$
Precision = \frac{TP}{TP+FP},
$$
with TP = True Positive and FP = False Positive.
Imagine you have the following values:
Image 1: $TP = 2, FP = 3$
Image 2: $TP = 1, FP = 4$
Image 3: $TP = 3, FP = 0$
The precision scores as you calculated will be:
Image 1: $2/5$
Image 2: $1/5$
Image 3: $1$
Your average ...
2
In short: It is easy to quantify information, but it is not easy to quantify its usefulness
I'm not sure how exactly you are looking to formalise your experiment, but it might be helpful to consider these points:
There is no such thing as an absolute measure of information. The amount of information contained in some dataset is dependent on the underlying ...
0
CNNs, LSTMs, GRUs and transformers are or use artificial neural networks. The expression computational intelligence (CI) is often used interchangeably with artificial intelligence (AI). CI can also refer to a subfield or superfield of AI where biology is often an inspiration. See What is Computational Intelligence and what could it become? by Włodzisław Duch....
1
Getting the intent of the sentence is not an easy task. To get you started on what to do, have a look on word vectors. You can also download pre-trained word2vec models. They help in getting similarity of words and reasoning with words. To get the intent of a sentence, you can use LSTM.
Fun fact most NLP algorithms strip away punctuation with is sufficient ...
1
First of all, there are multiple factors on how well models will work. Amount of data, source of data, hyperparameters, model type, training time etc... All of these will affect the accuracy. However, no classifier will work best in general. It all depends on the different factors, and not one can satisfy all, at least for now.
For improving the accuracy, ...
1
The accuracy depends on various factors. Might not always be the algorithm. For example a cleaner data with a poor algorithm might still give better results and vice versa.
What are the preprocessing techniques you are using? This preprocessing techniques article is a good starting point for html data. And by vectorising I assume you mean word2vec, use a ...
1
In reallity any continous function on a compact can be approximated by a neural network having one hidden layer with a finite number of neurones (This is the Universal Approximation Theorem). Thus you only need one hidden layer to approximate the multiplication on a compact, note that you need to apply a non linear activation on the hidden layer to do this.
2
As you know, an LSTM language model takes in the past word and tries to predict the new one and continue over a loop. A sentence is divided into tokens and depending on different method, the tokens are divided differently. Some model maybe character based models which simply uses each character as input and output. In this case you can treat punctuation as ...
0
Let $n=C*K_w*K_h$. Then you should only need $n$ filters. Not $2^n$ to keep all the information. If you just used the rows of the identity matrix as your filters than your convolution would just be making an exact copy so it definitely wouldn't be throwing away information. On the other hand, there will be a max pooling operation. To simplify the question ...
3
Take the sentence that was generated by your LSTM and feed it back into the LSTM as input. Then the LSTM will generate the next sentence. So the LSTM is using it's previous output as it's input. That's what makes it recursive. The intial word is just your base case. Also you should consider using GPT2 by open AI to do this. It's pretty impressive. https://...
1
Here's a link to my answer on CV Stack Exchange, where I have mentioned about latent spaces and some deep learning models that learn these representations: https://stats.stackexchange.com/questions/442352/what-is-a-latent-space/442360#442360
In short, deep learning models for Domain Adaptation, Computer Vision, Natural Language Processing, Recommendation ...
2
In your case, $L$ is the loss (or cost) function, which can be, for example, the mean squared error (MSE) or the cross-entropy, depending on the problem you want to solve. Given one training example $(\mathbf{x}_i, y_i) \in D$, where $\mathbf{x}_i \in \mathbb{R}^d$ is the input (for example, an image) and $y_i \in \mathbb{R}$ can either be a label (aka class)...
2
Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead.
A true positive is when you predict a car in a place and there is a car in that place.
A predicted positive is every car you ...
1
See Deep Metric Learning Beyond Binary Supervision in CVPR 2019
0
Definition and Explaination
For how Batch Normalization works exactly, I'll suggest you to read the following papers:
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
How Does Batch Normalization Help Optimization?
The recent interpretation on How BN works is that it can reduce the high-order effect as mentioned ...
0
Right now im trying cyclegan dcgan implementation on this (paired pictures from danbooru dressed to nude. cycle_gan model). I ll report text result in comparison to pix2pix here asap. But maybe 5-7 days will require.
Top 50 recent answers are included
