# Tag Info

5

In reinforcement learning, exploration has a specific meaning, which is in contrast with the meaning of exploitation, hence the so-called exploration-exploitation dilemma (or trade-off). You explore when you decide to visit states that you have not yet visited or to take actions you have not yet taken. On the other hand, you exploit when you decide to take ...

3

Personally, I would choose the following two as the most important: epsilon: When using an epsilon-greedy policy, epsilon determines how often the agent should explore and how often it should exploit. Balancing exploration and exploitation is crucial for the success of the learning agent. Too little exploration might not teach anything to the agent and too ...

3

In principle, yes, you can also evolve the genetic algorithm (or, in general, evolutionary algorithm), i.e. you can evolve its operations (such as the mutation and cross-over) and hyper-parameters (such as the size of the population or mutation rate). For example, you could use genetic programming to evolve the cross-over operation of a genetic algorithm. ...

3

In general, it is definitely very computationally expensive, so an exhaustive search is not performed in practice. However, there are some recent approaches for determining whether the architecture is "fine" without training the neural network first - by looking at the covariance matrix after forwarding the data, for example, in a recent paper ...

2

The crossover rate, $p_c \in [0, 1]$, is a hyper-parameter that controls the rate at which solutions are subjected to crossover. So, the higher $p_c$, the more crossovers you perform, so the more diversity (in terms of solutions/chromosomes) you may introduce in the population. Typical values of $p_c$ are in the range $[0.5, 1.0]$. For example, in this ...

2

You should read this study https://arxiv.org/abs/2006.05990 which does some empirical study on this question, specifically for on-policy, continuous action space DRL. It suggests that discount factor and learning rate are the two most important parameters to tune, followed by the width of the policy/value functions. That study also reports that it's very ...

2

Efficiently integrating HPO frameworks into an existing project is non-trivial. Most common datasets/tasks already have established architectures/hyperparameters/etc. and require only a few additional tuning parameters. In this case, the benefits (assuming they exist) brought by Bayesian HPO techniques lack behind development time (simplicity), and this is ...

1

What does it mean when ϵ=0 and ϵ=1? If ϵ=1, does it mean that the agent explores randomly? If this intuition is right, then it will not learn anything - right? On the other hand, if I set ϵ=0, does this imply that the agent doesn't explore? You are correct, when ϵ=1 the agent acts randomly. When ϵ=0, the agent always takes the current greedy actions. Both ...

1

I no longer really use validation that much, but rather only training and testing. Why? Because I mostly follow Ron Kohavi's (Stanford Univ) approach to CV. I have done a lot of validation but it seemed to be overkill, essentially causing me to ask why I have this very small-sampled parameter watch on the side from which I am supposed to learn from. You ...

1

In your case the most probable explanation would be the case of overfitting. The model with too many hidden layers have lots of parameters. By means of all these parameters the model is remembering stuff from the training data itself instead of generalizing by learning the useful patterns. As a rule of thumb if you increase the number of hidden layers more ...

1

This is from my own experience with (Vanilla) GANs, so it might not translate exactly to your application, but maybe it gives some orientation. your learning rate seems quite high. I've quite frequently found that 1e-5 is a good value for me. The training might take longer but will probably be more stable. have you tried using dropout? It's a good ...

1

Create two different optimizers and split the subnets' parameters into either with different lrs. You will have to call optimizer1.step(), optimizer2.step() with a single backward() call

1

Here is what I discovered empirically, trial and error. Since tuning the parameters are going to be environment specific, I'll lay out mine to give a better understanding of what I found to work for my case. Hopefully someone with better understanding of the algorithm will weigh in: Environment: A 2D map where an agent controls a simulated PC mouse pad and ...

1

I would generally assume that parameter tuning is the process of finding the combination of hyperparameters (e.g., population size, crossover and mutation operators and rates, etc.) that yield the best performance on your problem. When you're thinking about the way that performance varies with parameter choice, this is the "what". What is the best ...

1

I finally grasped the concept of word embedding. Thanks to @nbro, after reading the 2 articles s/he recommended What Are Word Embeddings for Text? and Word embeddings the 1st article gives me a good idea about the big picture of the Word Embeddings; whereas the 2nd article is actually the one which clears my mind. I am an visual person, I understand ...

1

The specific term you are looking for is "word embedding" and not just "embedding". How to numerically represent textual data? Neural networks (typically) require as inputs (and produce as outputs) numerical data (i.e. numbers, vectors, matrices, or higher-dimensional arrays). So, when processing textual data, we first need to encode (or ...

1

It is hard to tell what exactly is better because these are hyperparameters. However, the sigmoid activation function is closer to biological neurons. In the paper below, Bengio demonstrates why ReLU activation functions are better for hidden layers. In summary, they increase the sparsity of calculations (matrix in each layer shod multiply to its relative ...

1

As you said yourself, it is a hyperparameter. Hence, no one (even you) can say what is the ideal update frequency. You have to test and try. Having said that, remember one thing the target NN should mimic the actual network as closely as possible. Hence if you update it after a long number runs, then I think you will start losing the accuracy. On the ...

1

There is no singular best set of hyperparameters. Even more, there is no real search algorithm for hyperparameters. You can do a grid search, but this obviously will take some time. Most people either do that or will try to handpick their parameters. A few other things to note: Initializing your weights at [10^9,10^8] doesn't seem right to me. They should be ...

1

There is a hardware based reasoning. Matrix multiplication is one of the central computations in deep learning. SIMD operations in CPUs happen in batch sizes, which are powers of 2. Here is a good reference about speeding up neural networks on CPUs by leveraging SIMD instructions: Improving the speed of neural networks on CPUs You will notice batch sizes ...

Only top voted, non community-wiki answers of a minimum length are eligible