# Tag Info

4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...

4

This sounds like a problem that might be solvable with a LSTM-DQN approach, as described in Language Understanding for Text-based Games using Deep Reinforcement Learning by Narasimhan et al., 2015, and then extended to a domain very similar to your problem in Deep Reinforcement Learning for Syntactic Error Repair in Student Programs by Gupta et al., 2019. ...

4

Introduction The paper Generalization in Deep Learning provides a good overview (in section 2) of several results regarding the concept of generalisation in deep learning. I will try to describe one of the results (which is based on concepts from computational or statistical learning theory, so you should expect a technical answer), but I will first ...

3

There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time. Many networks have been tried for text generation, with notable examples such as Markov chain. However RNN have been proven to work the best and is dominating the field of language modelling (...

3

tl;dr The whole point of gradient descent is to assess the contribution of each parameter towards the loss. This information is uncovered through the gradient of the loss w.r.t each parameter. A deeper look... Suppose we have a NN with parameters $w_{i}, \; i={1, 2, ...}$. This NN makes some predictions, which we compare to the actual targets and compute ...

3

For the first question, RMSE and Euclidean distance have no difference, not that i know of. For the second question, you only need the common loss function for normal tasks. MSE is a common loss function used in linear regression tasks as well as loss function similar in nature like the RMSE. For classification tasks, Cross Entropy Loss is preferred. For ...

3

When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a guess on what is in the picture and then you click if it got it right or not. If you continue to input images and lie to it it will obviously get worse and ...

2

I'm working on a similar problem. I'm using a 2D point cloud of an object, for example, X and Y coordinates for height, and with that more simple data set I will train a regression model (currently working on that). In my opinion, this approach with dissecting complex point cloud into cross sections that contain wanted dimension and feeding that to the model ...

2

Your code suggests a likely problem here: It looks like you are training a very deep neural network with sigmoidal activation functions at every layer. The sigmoid has the property that its derivative (S*(1-S)) will be extremely small when the activation function's value is close to 0 or close to 1. In fact, the largest it can be is about 0.25. The ...

2

In the paper Exploiting Open-Endedness to Solve Problems Through the Search for Novelty (2008), by Joel Lehman and Kenneth O. Stanley, which introduced the novelty search approach, it is written Thus this paper introduces the novelty search algorithm, which searches with no objective other than continually finding novel behaviors in the search space. and ...

2

Another explanation of deep learning as an end-to-end framework is in deep learning, pre-processing or feature extraction steps are not necessary. So it only uses a single processing step, which is to train the deep learning model. In other traditional machine learning methods, some separated feature extraction steps usually required. For example in image ...

2

These are big areas, so here is a brief description of the differences: Game theory is concerned with studying solutions for 'games', which are basically a set of decisions leading to certain outcomes. In game theory you look at strategies to achieve the best outcome for a given participant. One classic example (which isn't really a game in the traditional ...

2

If the task involves only apples, orange and peaches, you should use method 1. As the number of classes is small, the network cannot generalize well to all classes. As a side note, you should start with the pretrained weights of YOLO v3 as some classes of YOLO v3 may be fruits, which can help your model converge faster. If the number of classes is large, ...

2

It is explained in this CrossValidated post. Top1 accuracy means the best guess (class with highest probability) is the correct result 58.9% of the time, while top5 accuracy means the correct result is in the top 5 best guesses (5 classes with highest probabilities) 87.7% of the time.

2

I try to answer the things I know for sure: One effect of bigger images is the increasing computation time due to more pixels (input to your training) 4.Grayscaling reduces the information, which might decrease training time, but also model performance (accuracy, precision, recall). What I have seen is that grayscaling is used in for example face detection ...

2

Recurrent Neural Networks (RNNs) have been applied to generate text. In this blog post you will find a couple of interesting text examples (the author also has made his code available on github), e.g. their Shakespeare-like texts generated by an RNN: PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd ...

2

The most distinct words of a language are usually the function words (the, and, of, with,...); other lexical items are often (at least partly) shared between languages that had come in contact with each other. So looking for function words is usually the best way to identify the language in a given text. This can be done by having a list of function words ...

1

1) your input should be so that you describe your entire environment. This could be done by 8 (length)* 8 (height)* 3 (either empty space, opponent chip or your chip) = 192 input neurons. you can just import a image of the current boardstate (which is width pixels * height pixels input neurons), but this means you task the neural network with learning to ...

1

If we seek proven working source code to plug into a GPLv2-licence compatible solution, we should at least consider autotrace. Its source code is open for review. It can be tested against the example images we have and, if it works fine, called by our GPLv2 software. We can even use the calling code in Inkscape's plug-in image tracing implementation as a ...

1

I think you may have a class imbalance problem here, if I am reading your output correctly. You have 20,000 negative examples, but only 8000 positive ones, and you are minimizing binary cross entropy without re-weighting the examples, so your model can achieve a low-ish loss just by consistently outputing a value close to 0. This forms a local optima in the ...

1

This is relevant when you have two or more neural networks serving as components to a larger architecture. Training this architecture in an end-to-end manner means simultaneously training all components (i.e. training it as a single network). The best example I can think of are image captioning architectures. These usually comprise of two networks: a CNN ...

1

logp seen in code is actually logit p which has this story behind: Given a probability p, the corresponding odds are calculated as p / (1 – p). For example if p=0.75, the odds are 3 to 1: 0.75/0.25 = 3. The logit function is simply the logarithm of the odds: logit(x) = log(x / (1 – x)). Sigmoid near logp is like follows: The inverse of the logit ...

1

The key here is think up strategies. If we define this as examining, creating a hypothesis, and testing it as strategizing then yes AI has the ability to strategize. It can examine other users' games, quantifies actions that correlated with victory then test if it gains victory by doing those actions. Strategy by definition is: a plan of action or policy ...

1

The first neural network machine was the stochastic neural analog reinforcement calculator (SNARC), built in the 1950s. As you can see, it's pretty old. After that, there were several advances regarding backpropagation and the vanishing gradient problem. However, the ideas itself are not novel. Simply put, we have the data and processing power today that we ...

1

The reason for robot like speech may be because tacotron uses griffin lim for vocoder, which cannot reproduce sound with perfection, often introducing robot like sound artifects. A vocoder is a network that transforms a transform a spectrogram image back to speech waveform. Tacotron and many other speech generation neural network uses CNN to generate ...

1

There is a platform called Flow, sounds like its what you're looking for https://github.com/flow-project/flow

1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ...

1

Lets start with question 1) how does JS-divergence handles zeros? by definition: \begin{align} D_{JS}(p||q) &= \frac{1}{2}[D_{KL}(p||\frac{p+q}{2}) + D_{KL}(q||\frac{p+q}{2})] \\ &= \frac{1}{2}\sum_{x\in\Omega} [p(x)log(\frac{2 p(x)}{p(x)+q(x)}) + q(x)log(\frac{2 q(x)}{p(x)+q(x)})] \end{align} Where $\Omega$ is the union of the domains of $p$ and ...

Only top voted, non community-wiki answers of a minimum length are eligible