Tag Info

28

Here's a snippet from an article by Gary Marcus In particular, they showed that standard deep learning nets often fall apart when confronted with common stimuli rotated in three dimensional space into unusual positions, like the top right corner of this figure, in which a schoolbus is mistaken for a snowplow: . . . Mistaking an ...

19

In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as the universal approximation theorem (UAT), but that should not be called universal, given that there are a lot more discontinuous functions than continuous ones,...

15

In our deep learning lecture, we discussed the following example (from Unmasking Clever Hans predictors and assessing what machines really learn (2019) by Lapuschkin et al.). Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the ...

11

This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i.e., you do not expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit ...

4

This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other ...

4

There are certainly things like this. I'd say a strong example is layered learning approaches, descended from Peter Stone's work. A programming language is essentially a collection of useful shorthands for assembly-level instructions. Ultimately, everything you do in a programming language eventually gets executed in assembly. So making a programming ...

3

Usually, when talking about regularization for neural networks there are 3 main types: L1, L2 and dropout. All affect the gradient descent procedure. L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of ...

3

I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points. One example that I've had the opportunity to work on is stance classification 1. In many datasets, the best F score achievable is around 70%....

3

A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data. See e.g. this article The Unlearnable Checkerboard Pattern (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.

3

According to Wikipedia: A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form, the data-generating process. Answer to your question: To build any neural network ...

3

Neural networks seem to have a great deal of difficulty handling adversarial input, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them. This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be ...

2

It might be that your dataset of images is to small. Your discriminative network might hardlearn these images at which point your generative network can only produce good images if it copies the same images of your dataset.

2

Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable? It's not so much that answers are subjective, but you are performing an experiment, and this should be driven by results. If you can find something published about a similar environment that might help you narrow down your choices. That said, ...

2

$\nabla_{\theta_{i-1}} \theta_{i-1} = \mathbf{I}$ in a similar way that $\frac{d f}{dx} = 1$ for $f(x) = x$. Strictly speaking, $\mathbf{I}$ should be a vector of $1s$ with the same dimensionality as $\theta_{i-1}$, but they are probably abusing notation here and putting such a vector as the diagonal elements of a matrix. Alternatively (actually, the most ...

2

Large scale route optimization problems. The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: https://arxiv.org/abs/1802.04240v2. However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.

2

So as you're probably already aware of, CBOW and Skip-gram are just mirrored versions of each other. CBOW is trained to predict a single word from a fixed window size of context words, whereas Skip-gram does the opposite, and tries to predict several context words from a single input word. Intuitively, the first task is much simpler, this implies a much ...

2

ResNet is an architecture for object recognition and you may use it to do your classification task. Fast RCNN may improve your results but is a more difficult architecture to implement. If you want to go in this direction the best place to start is the arxiv paper of the Fast R-CNN (arxiv.org/abs/1504.08083). If I am not wrong, there is an implementation ...

2

It's perfectly reasonable to apply 'traditional' Deep Learning approaches to try and learn an adjacency matrix (a matrix is just a vector of vectors, which can be flattened into a single output vector) but you might need a lot of training data as N gets larger. Your outputs could certainly have the form of an adjacency matrix, as you describe. Whether it's ...

2

It is really simple. In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you use gradient descent to adjust the network weights to minimize the cost. Then you repeat this process until you get a satisfactory level of accuracy. For example, ...

2

There is no easy rule for this. You can use transfer learning to select a model that works well on image classification. However the accuracy you achieve will be highly dependent on your training set. If your training set is "similar" in quantity and quality to what was used for the accuracy achieved by the transfer learning model in some application you ...

2

Short Answer: Yes Visually: if you see the image from wikipedia, it shown that ReLU (the blue line) is non-Linear (the line is not straight, it turns in 0). You can also check "visual" definition of linear function in wikipedia: "In calculus and related areas, a linear function is a function whose graph is a straight line" Mathematically: Linear ...

1

The main thing to keep in mind when designing a reinforcement learning agent is that you need to develop an interactive environment in which the agent can learn and define the possible moves the agent can make. In your case, the environment is the memory cards. Next you need to define how the agent can interact with the environment, that is choosing a card ...

1

There are several ways you can do this. One is to input both images in input, so it can be a 2 input system or an input with 6 channels. As you suggested in 1st point, you can make 2 networks, connect them at the end and add another layer for final classification or use outputs from both and train another classifier (like Gradient bosting). You can look ...

1

This sounds to me like a use case for a chatbot. You would have different intents reflecting the types of user queries that your system can respond to. The intent matching can be done by pattern matching, machine learning (classification), or a combination of the two (hybrid). You can then use the chatbot to ask clarification questions or elicit more ...

1

I think that making some draws might help. Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector I with length f (number of features), a hidden layer H which does not have a fixed size, and output O of length c (number of classes). Then we have 3 extra vectors than usual: a vector U they ...

1

From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on "unstructured" data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet). ...

1

In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of translation invariance) For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face. Operations like max-pooling may also ...

1

I'm not sure it's possible to help much because this is an experimental question. I'm afraid the only answer comes with testing many different options. I see a little thing that might be making your model a little worse, though: You're concatenating "relu" with "sigmoid". Placing two different nature values in the same array may make it more difficult ...

1

No, it appears to take large amounts of screenshots to generate the images to be used for training. Don't waste your money. I would say you could try training on colab if you can upload the training data.

1

First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is to take terminology, notation and definitions with a grain of salt. However, in this case, although it may sound confusing to you, all of the current ...

Only top voted, non community-wiki answers of a minimum length are eligible