All Questions
Tagged with hyperparameters or hyper-parameters
18 questions with no upvoted or accepted answers
4
votes
0
answers
3k
views
Optimal episode length in reinforcement learning
I have a custom environment for stock trading where an episode can be as long as 2000-3000 steps. I've run several experiments with td3 and sac algorithms, average reward per episode flattens after ...
- 361
3
votes
0
answers
191
views
How can I do hyperparameter optimization for a CNN-LSTM neural network?
I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
- 31
3
votes
0
answers
36
views
How to organize model training hyperparameters
I am working on multiple deep learning projects, most of them in the area of computer vision. For many of them I create multiple models, try different approaches, use various model architectures. And ...
- 165
3
votes
0
answers
41
views
Which hyper-parameters are considered in neural architecture search?
I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
- 73
2
votes
0
answers
56
views
Is there an optimal number of species for NEAT?
Is there an optimal number of species for NEAT?
Since too low and too high is bad, I am thinking about adjusting the threshold of the distance function at runtime in order to have the number of ...
- 51
2
votes
0
answers
71
views
Given a 2-layer GCN, can we choose the dimensions of the 2nd weight matrix, such that this architecture has the same capacity as a 1-layer GCN?
This might be more of a question about nested function classes:
For $k$ class node classification in a graph with $n$ nodes, and $d$ feature vector.
I want to compare
Architecture I: the GCN model of ...
- 121
2
votes
0
answers
271
views
How will the filter size affect the transpose convolution operation?
After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling.
How will the ...
- 51
2
votes
0
answers
73
views
Which CNN hyper-parameters are most sensitive to centered versus off centered data?
Which hyper-parameters of a convolutional neural network are likely to be the most sensitive to depending on whether the training (and test and inference) data involves only accurately centered images ...
- 121
2
votes
0
answers
218
views
What is the correct dimension of mu/logvar and z in the VAE?
I'm having a problem to understand the needed dimensions of a VAE, especially for mu, logvar and ...
- 21
2
votes
0
answers
39
views
What is the benefit of scaling the hyperparameter C of an SVM?
Please read the following page of the Sklearn documentation.
The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled ...
- 141
1
vote
0
answers
18
views
Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering
I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
- 11
1
vote
0
answers
49
views
How to make my neural networks designs more robust
Whenever, I design a neural network to solve a novel problem (requires a novel loss function i.e. not image classification) it always ends up being hypersensitive to batch size and learning rate.
...
- 366
1
vote
0
answers
21
views
Hyperparameters for Reproducing the Results of IRGAN on MovieLens 1M
I am trying to reproduce results reported for IRGAN (information retrieval GAN) on the MovieLens 1M dataset. The results I want to reproduce and their sources are listed in the table below.
Model
...
- 111
1
vote
0
answers
238
views
How does noise input size affect fake image generation with GANs?
In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
- 793
1
vote
0
answers
179
views
Should we start with a small batch-size and increase during training to improve sample efficiency?
Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
- 461
1
vote
0
answers
125
views
Why is the number of neurons used in various neural networks power of 2?
I have noticed that almost all tutorials take the number of neurons as a power of 2. Is there any proper mathematical and well-proven reason for that?
If you sometimes change it to some other odd ...
- 164
1
vote
0
answers
614
views
Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?
As far as I understand, RL is a process that can be divided into 2 stages:
Exploring a wide range of paths (acting randomly)
Refining the current optimal paths (revolving around actions with a so-...
- 280
0
votes
2
answers
221
views
How to ensure that the ES-HyperNEAT algorithm generates an ANN in the substrate?
I'm trying to implement the ES-HyperNEAT algorithm using the original paper, as well as the pseudocode provided in the official user page. Occasionally, the algorithm would be unable to generate a ...
- 1