New answers tagged deep-learning
0
Hey i am working on my Bachelor thesis at the moment and use UNET in combination with a GAN for image segmentation. I spend the last 5 months on that, so on my tests, the new approach of januar 2020, called Multires-UNET is quite a good choice for more texture orientated segmentation. I use the current github implementation. Its quite nice, maybe you notice ...
1
I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options.
I see a little thing that might be making your model a little worse, though:
You're concatenating "relu" with "sigmoid".
Placing two different nature values in the same array may make it more difficult ...
1
Batch size and epochs are independent parameters - they serve very different purposes. Your main question as I understand it (and for general, non-library specific consumption) is what is an epoch and how is the data used for each epoch?
Simply put, an epoch is a single iteration though the training data. Each and every sample from your training dataset ...
0
ok so let me explain in my word how i understood this process:
i know that one sample mean one row, therefore if we have data with size(177,3), that means we have 177 sample. because we have divided X and y into training and test, therefore we have following pairs (X_train,y_train) and (X_test, y_test)
now about batch size, if we have let say 177 ...
3
Usually, when talking about regularization for neural networks there are 3 main types:
L1, L2 and dropout. All affect the gradient descent procedure.
L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of ...
0
I am training a model and i am getting test results greater than train results.
You don't give us too many details, but most probably it's underfitting.
What could be the reason behind this?
Underfitting is often a result of an excessively simple model.
Too much regularization techniques were used.
Is this acceptable to get the RMSE greater in test ...
2
How to estimate the accuracy upper limit of any CNN model over a computer vision classification task
There is no easy rule for this. You can use transfer learning to select a model that works well on image classification. However the accuracy you achieve will be highly dependent on your training set. If your training set is "similar" in quantity and quality to what was used for the accuracy achieved by the transfer learning model in some application you ...
0
RMSE stands for Root Mean Squared Error. As the name suggests, it is calculated by taking the square root over the mean of the squared errors of individual points.
It is normal for the test error to be higher than the train error and in most cases, the test error will be greater than the train error.
1
The main thing to keep in mind when designing a reinforcement learning agent is that you need to develop an interactive environment in which the agent can learn and define the possible moves the agent can make. In your case, the environment is the memory cards.
Next you need to define how the agent can interact with the environment, that is choosing a card ...
4
There are certainly things like this.
I'd say a strong example is layered learning approaches, descended from Peter Stone's work.
A programming language is essentially a collection of useful shorthands for assembly-level instructions. Ultimately, everything you do in a programming language eventually gets executed in assembly. So making a programming ...
2
Short Answer: Yes
Visually:
if you see the image from wikipedia, it shown that ReLU (the blue line) is non-Linear (the line is not straight, it turns in 0). You can also check "visual" definition of linear function in wikipedia:
"In calculus and related areas, a linear function is a function whose graph is a straight line"
Mathematically:
Linear ...
2
It is really simple.
In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you use gradient descent to adjust the network weights to minimize the cost. Then you repeat this process until you get a satisfactory level of accuracy. For example, ...
0
"Naturally, this will affect the gradient through the network." this statement is only partially true, let's see why by starting explaining the real aim of batch normalisation.
As the title of the paper suggest, the aim of batch normalisation is to decrease training time by reducing covariance shift. What is covariance shift? We can conceive it as the ...
0
The VAE uses the ELBO loss, which is composed of the KL term and the likelihood term. The ELBO loss is a lower bound on the evidence of your data, so if you maximize the ELBO you also maximize the evidence of the given data, which is what you indirectly want to do, i.e. you want the probability of your given data (i.e. the data in your dataset) to be high (...
1
From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on "unstructured" data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet).
...
2
It's perfectly reasonable to apply 'traditional' Deep Learning approaches to try and learn an adjacency matrix (a matrix is just a vector of vectors, which can be flattened into a single output vector) but you might need a lot of training data as N gets larger.
Your outputs could certainly have the form of an adjacency matrix, as you describe. Whether it's ...
11
This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i.e., you do not expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit ...
1
There are several ways you can do this.
One is to input both images in input, so it can be a 2 input system or an input with 6 channels.
As you suggested in 1st point, you can make 2 networks, connect them at the end and add another layer for final classification or use outputs from both and train another classifier (like Gradient bosting). You can look ...
1
This sounds to me like a use case for a chatbot. You would have different intents reflecting the types of user queries that your system can respond to. The intent matching can be done by pattern matching, machine learning (classification), or a combination of the two (hybrid). You can then use the chatbot to ask clarification questions or elicit more ...
2
ResNet is an architecture for object recognition and you may use it to do your classification task. Fast RCNN may improve your results but is a more difficult architecture to implement. If you want to go in this direction the best place to start is the arxiv paper of the Fast R-CNN (arxiv.org/abs/1504.08083). If I am not wrong, there is an implementation ...
1
I think that making some draws might help.
Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector I with length f (number of features), a hidden layer H which does not have a fixed size, and output O of length c (number of classes). Then we have 3 extra vectors than usual:
a vector U they ...
2
So as you're probably already aware of, CBOW and Skip-gram are just mirrored versions of each other. CBOW is trained to predict a single word from a fixed window size of context words, whereas Skip-gram does the opposite, and tries to predict several context words from a single input word.
Intuitively, the first task is much simpler, this implies a much ...
3
Neural networks seem to have a great deal of difficulty handling adversarial input, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them.
This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be ...
0
End to end means deep learning is the only thing that is used.
Many people have doubts on its viability though, I certainly do. I wouldn't trust an end-to-end DL based self driving car.
1
In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of translation invariance)
For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face.
Operations like max-pooling may also ...
2
Large scale route optimization problems.
The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: https://arxiv.org/abs/1802.04240v2.
However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.
15
In our deep learning lecture, we discussed the following example (from Unmasking Clever Hans predictors and assessing what machines really learn (2019) by Lapuschkin et al.).
Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the ...
3
I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points.
One example that I've had the opportunity to work on is stance classification 1. In many datasets, the best F score achievable is around 70%....
3
A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data.
See e.g. this article The Unlearnable Checkerboard Pattern (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.
4
This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other ...
3
According to Wikipedia:
A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process.
Answer to your question:
To build any neural network ...
19
In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as the universal approximation theorem (UAT), but that should not be called universal, given that there are a lot more discontinuous functions than continuous ones,...
28
Here's a snippet from an article by Gary Marcus
In particular, they showed that standard deep learning nets often fall
apart when confronted with common stimuli rotated in three dimensional
space into unusual positions, like the top right corner of this
figure, in which a schoolbus is mistaken for a snowplow:
.
.
.
Mistaking an ...
0
Validation dataset: 600 * 24 = 14400
Means that you are augmenting the validation set, right? For an experiment, you can do that and it might take validation accuracy more than train accuracy?
The idea of augmentation in only valid for the training set and you should not change the validation set or test set.
You can try without the augmentation in the ...
0
Assuming you pass through the entire validation dataset, this can't be due to shuffling since you still compute the loss/accuracy over the entire dataset, so order does not really matter here. It is more likely that you have a significantly smaller or less representative validation dataset, e.g., distribution of the validation dataset can be skewed towards ...
0
There is a general idea in the field of NLP that there is a mapping between embeddings in different langauges. Figure 1 explains this.
In Figure 1. we have the embedding of English words and Spanish words, and we see that their exists a mapping between the manifolds associated to this two languages, i.e. Spanish manifold is a distorted image of the English ...
1
Embeddings generated by transformers like Bert or XLM-R are fundamentally different from embeddings learned through language models like GloVe or Word2Vec.
The latter are static, i.e. they are just dictionaries containing a vocabulary with n-dimensional vectors associated to each word. Because of this they can be plotted through PCA and the distance between ...
2
$\nabla_{\theta_{i-1}} \theta_{i-1} = \mathbf{I}$ in a similar way that $\frac{d f}{dx} = 1$ for $f(x) = x$. Strictly speaking, $\mathbf{I}$ should be a vector of $1s$ with the same dimensionality as $\theta_{i-1}$, but they are probably abusing notation here and putting such a vector as the diagonal elements of a matrix. Alternatively (actually, the most ...
1
No, it appears to take large amounts of screenshots to generate the images to be used for training.
Don't waste your money. I would say you could try training on colab if you can upload the training data.
2
It might be that your dataset of images is to small. Your discriminative network might hardlearn these images at which point your generative network can only produce good images if it copies the same images of your dataset.
2
Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable?
It's not so much that answers are subjective, but you are performing an experiment, and this should be driven by results. If you can find something published about a similar environment that might help you narrow down your choices.
That said, ...
1
First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is to take terminology, notation and definitions with a grain of salt. However, in this case, although it may sound confusing to you, all of the current ...
Top 50 recent answers are included
