Questions tagged [hyper-parameters]

For questions related to the hyper-parameters of AI models and algorithms, which are parameters that are set before the learning process begins. For example, the number of hidden layers in a feed-forward neural network is usually a hyper-parameter.

21 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4 votes
0 answers
3k views

Optimal episode length in reinforcement learning

I have a custom environment for stock trading where an episode can be as long as 2000-3000 steps. I've run several experiments with td3 and sac algorithms, average reward per episode flattens after ...
Mika's user avatar
  • 341
3 votes
0 answers
165 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
Yining's user avatar
  • 31
3 votes
0 answers
35 views

How to organize model training hyperparameters

I am working on multiple deep learning projects, most of them in the area of computer vision. For many of them I create multiple models, try different approaches, use various model architectures. And ...
Matthias's user avatar
  • 165
3 votes
0 answers
40 views

Which hyper-parameters are considered in neural architecture search?

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
cocojambo's user avatar
2 votes
1 answer
1k views

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input? For example, we may want to learn how to reconstruct a relatively low-...
James Arten's user avatar
2 votes
0 answers
51 views

Is there an optimal number of species for NEAT?

Is there an optimal number of species for NEAT? Since too low and too high is bad, I am thinking about adjusting the threshold of the distance function at runtime in order to have the number of ...
IAmUser's user avatar
  • 51
2 votes
0 answers
63 views

Given a 2-layer GCN, can we choose the dimensions of the 2nd weight matrix, such that this architecture has the same capacity as a 1-layer GCN?

This might be more of a question about nested function classes: For $k$ class node classification in a graph with $n$ nodes, and $d$ feature vector. I want to compare Architecture I: the GCN model of ...
Tinatim's user avatar
  • 121
2 votes
0 answers
244 views

How will the filter size affect the transpose convolution operation?

After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling. How will the ...
Vortex's user avatar
  • 51
2 votes
0 answers
72 views

Which CNN hyper-parameters are most sensitive to centered versus off centered data?

Which hyper-parameters of a convolutional neural network are likely to be the most sensitive to depending on whether the training (and test and inference) data involves only accurately centered images ...
hotpaw2's user avatar
  • 121
2 votes
0 answers
205 views

What is the correct dimension of mu/logvar and z in the VAE?

I'm having a problem to understand the needed dimensions of a VAE, especially for mu, logvar and ...
Shoorty's user avatar
  • 21
2 votes
0 answers
37 views

What is the benefit of scaling the hyperparameter C of an SVM?

Please read the following page of the Sklearn documentation. The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled ...
zwithouta's user avatar
  • 141
1 vote
0 answers
17 views

Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering

I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
Prithvi's user avatar
  • 11
1 vote
0 answers
31 views

How to make my neural networks designs more robust

Whenever, I design a neural network to solve a novel problem (requires a novel loss function i.e. not image classification) it always ends up being hypersensitive to batch size and learning rate. ...
Tom Huntington's user avatar
1 vote
0 answers
21 views

Hyperparameters for Reproducing the Results of IRGAN on MovieLens 1M

I am trying to reproduce results reported for IRGAN (information retrieval GAN) on the MovieLens 1M dataset. The results I want to reproduce and their sources are listed in the table below. Model ...
Lukas's user avatar
  • 111
1 vote
0 answers
212 views

How does noise input size affect fake image generation with GANs?

In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
mark mark's user avatar
  • 763
1 vote
0 answers
164 views

Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
Ray Walker's user avatar
1 vote
0 answers
106 views

Why is the number of neurons used in various neural networks power of 2?

I have noticed that almost all tutorials take the number of neurons as a power of 2. Is there any proper mathematical and well-proven reason for that? If you sometimes change it to some other odd ...
neel g's user avatar
  • 146
1 vote
0 answers
594 views

Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?

As far as I understand, RL is a process that can be divided into 2 stages: Exploring a wide range of paths (acting randomly) Refining the current optimal paths (revolving around actions with a so-...
Kari's user avatar
  • 270
0 votes
0 answers
12 views

Incorporating HiPlot and Keras

I just started to learn about Keras and train some models, and I came across HiPlot which is used for tuning hyperparameters. I was wondering if HiPlot can also be used to see what parameters would ...
Kasiopea's user avatar
0 votes
0 answers
37 views

Which of these 3 mutation rates is the best in terms of performance?

I am need some comments since I am conducting experiments with 3 different mutation rates and hesitate to choose the best one. I ...
LearningLogic's user avatar
0 votes
2 answers
183 views

How to ensure that the ES-HyperNEAT algorithm generates an ANN in the substrate?

I'm trying to implement the ES-HyperNEAT algorithm using the original paper, as well as the pseudocode provided in the official user page. Occasionally, the algorithm would be unable to generate a ...
SirBob's user avatar
  • 1