4

I'll cover both L2 regularized loss, as well as Mean-Squared Error (MSE): MSE: L2 loss is continuously-differentiable across any domain, unlike L1 loss. This makes training more stable and allows for gradient-based optimization, as opposed to combinatorial optimization. Using L2 loss (without any regularization) corresponds to the Ordinary Least Squares ...


3

Depends on perspective. On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works such as Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2):215 – 250, 2002. Michael Bowling....


3

In general, it is definetely very computationally expensive, so the exhaustive search is not perfromed in pratice, however, there are some recent approaches for determining, whether the architecture is fine, without performing the training - by looking at the covariance matrix after forwarding the data, for example, in a recent paper - https://arxiv.org/abs/...


3

I don't think people generally do use neural nets for grid world. As long as the state and action spaces are small enough, you should be able to store Q values in a table like you suggested. Neural nets come in handy when the state space is very large (or even continuous), so you can't afford to store a table of Q values. Also, neural nets have the ability ...


2

You should not use augmented data in the validation nor in the test set. Validation and test set are purely used for hyperparameter tuning and estimating the final performance, i.e. estimating the generalization error. These two data sets should be as close as possible to other data, which you could have acquired, but you actually haven not, i.e. your true ...


2

In DQN that was presented in the original paper the update target for the Q-Network is $\left(r_t + \max_aQ(s_{t+1},a;\theta^-) - Q(s_t,a_t; \theta)\right)^2$ were $\theta^-$ is some old version of the parameters that gets updated every $C$ updates, and the Q-Network with these parameters is the target network. If you didn't use this target network, i.e. if ...


2

They can indeed. Although generally they are kept to images because at the moment, they are the best at that, but not the best in other areas that you might consider. GANs can be used for audio generation, with many examples such as GANsynth and GAN voice generation. But each of these tasks are outperformed by other methods. With music generation, WaveNet is ...


2

Currently both ReLU and ELUs are the most popular activation functions (AF) used in neural nets (NNs). This is because they eliminate the vanishing gradient problem that causes major problems in the training process and degrades the accuracy and performance of NN models. Also these AFs, more specifically ReLU, are very fast learning AF which make them even ...


2

Am I correct in this understanding that with the increasing complexity of problems, tabular RL methods are getting obsolete? Individual problems don't get any more complex, but the scope of solvable environments increases due to research and discovery of better or more apt methods. Using deep RL methods with large neural nets can be a lot less efficient for ...


2

ELU does not suffer from from dying neurons issue, unlike ReLU. While ELU can help you to acheive a better accuracy, it is a slower than ReLU because of its non linearity in its negative range. Choosing a right activation function totally depends on situations but you need to also consider other similar types of activation function such as leaky ReLU. Check ...


2

You could sequentially pass in each element of your sequential data and save the hidden and cell states in a separate buffer. In a typical LSTM implementation, you input the entire sequence and the hidden and cell states are propagated internally. In the end, the final hidden and cell states returned as the output. This works if your input is all the same ...


1

A few years ago, deep learning was a buzz word, but now is de-facto a standard term or expression, and it's widely used in all research papers, although deep learning is almost never defined rigorously (but this doesn't seem to be a big problem!). From my experience (this is not just an opinion, of course!), after having read so many papers on the topic, ...


1

I'll answer in a couple of stages. I feel somewhat lost as to what the input for the NN should look like. Your choices boil down to two options, each with their own multitude of variants: Vector Representation: Your input is a vector of the same size as your vocabulary where the elements represent the tokens in the input example. The most basic version of ...


1

Yes, a category "no person" or "random image" would make sense. Binary classification is only helpful if you know that your input always belongs to one or the other category, for example by pre-filtering the inputs.


1

From what I understood, you want to be able to determine whether the input to your classifier is a valid picture or not. Where: Valid picture: image of a person wearing or not wearing a seatbelt Not valid picture: unrelated images (say a kitchen picture) or noise, or a black image (no input at all) For that you could build a Bayesian model from your ...


1

The policy doesn't change over time. That is, the values will change, otherwise we would not be learning anything, but our rules for action selection don't. I.e. we always take action according to the distribution postulated to our current estimate of the policy $\pi_\theta(a|s)$, we don't suddenly start taking $\max_a \pi_\theta(a|s)$, which would be a true ...


1

Assuming that you have access to the training data set, you could use an autoencoder network to predict what features f4, f5, f6 'could be' for the test data set. The way to do this is to train the autoencoder on the training data set with features f1, f2, f3 as inputs, and then use f1,f2,f3,f4,f5,f6 as the output of the network. The autoencoder then ...


1

How to fix the network above to auto-classify XOR data, in unsupervised manner? This cannot be done, except accidentally. Unsupervised learning cannot replace or emulate supervised learning. As a thought experiment, consider why you would expect the network to discover XOR, when simply considering outputs rounded to binary, you could equally find AND, OR, ...


1

Generally, order matters. A (trained) Neural Network (NN) is just a mathematical function trained on taking some given input and producing the corresponding output. So, if you train a certain node on producing large output if (and only if) an animal is present in a picture (for example), but later you give it the numeric evidence for a car being present in ...


1

When it talks to other domains such as image or music, using transformer will always face a problem of sequence length limitation. To the best of my knowledge, the bottleneck of self-attention which uses a $n^2$ matrix quite limits transformer being applied to other domains. For example, a 32x32 pixel image, means a sequence of 1024 tokens. OpenAI did some ...


1

In general it's better to not use sigmoid function in any hidden layer. There are many other great options such as ReLU and ELU. However, if for any reason you have to use sigmoid-like function, then go with Tanh function, at least it has ~0 mean.


Only top voted, non community-wiki answers of a minimum length are eligible