Questions tagged [hyper-parameters]

For questions related to the hyper-parameters of AI models and algorithms, which are parameters that are set before the learning process begins. For example, the number of hidden layers in a feed-forward neural network is usually a hyper-parameter.

19 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
52 views

How can I do hyperparameter optimization for a CNN-LSTM neural network?

I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not ...
3
votes
0answers
21 views

How to organize model training hyperparameters

I am working on multiple deep learning projects, most of them in the area of computer vision. For many of them I create multiple models, try different approaches, use various model architectures. And ...
2
votes
0answers
25 views

How will the filter size affect the transpose convolution operation?

After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling. How will the ...
2
votes
0answers
31 views

Which CNN hyper-parameters are most sensitive to centered versus off centered data?

Which hyper-parameters of a convolutional neural network are likely to be the most sensitive to depending on whether the training (and test and inference) data involves only accurately centered images ...
2
votes
0answers
33 views

What is the benefit of scaling the hyperparameter C of an SVM?

Please read the following page of the Sklearn documentation. The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled ...
1
vote
0answers
8 views

Is there an optimal number of species for NEAT?

Is there an optimal number of species for NEAT? Since too low and too high is bad, I am thinking about adjusting the threshold of the distance function at runtime in order to have the number of ...
1
vote
0answers
14 views

Hyperparameters for Reproducing the Results of IRGAN on MovieLens 1M

I am trying to reproduce results reported for IRGAN (information retrieval GAN) on the MovieLens 1M dataset. The results I want to reproduce and their sources are listed in the table below. Model ...
1
vote
0answers
19 views

How does noise input size affect fake image generation with GANs?

In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
1
vote
0answers
35 views

Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
1
vote
0answers
46 views

Why is the number of neurons used in various neural networks power of 2?

I have noticed that almost all tutorials take the number of neurons as a power of 2. Is there any proper mathematical and well-proven reason for that? If you sometimes change it to some other odd ...
1
vote
0answers
328 views

Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?

As far as I understand, RL is a process that can be divided into 2 stages: Exploring a wide range of paths (acting randomly) Refining the current optimal paths (revolving around actions with a so-...
0
votes
0answers
16 views

How many singular vectors do we need to calculate for SVD?

In the geometrical interpretation of SVD, the data points that we have need to be imagined as points in high dimensional space (say $d$-dimensional space). But we need to find a hyperplane in $k-$...
0
votes
0answers
15 views

How to change number of trained layers in object detection TensorFlow models?

Training custom object detection models with TensorFlow usually means a transfer learning of pre-trained models and, if I understand it correctly, it means only training the few last layers, with ...
0
votes
1answer
24 views

How to ensure that the ES-HyperNEAT algorithm generates an ANN in the substrate?

I'm trying to implement the ES-HyperNEAT algorithm using the original paper, as well as the pseudocode provided in the official user page. Occasionally, the algorithm would be unable to generate a ...
0
votes
0answers
37 views

Optimal episode length in reinforcement learning

I have a custom environment for stock trading where an episode can be as long as 2000-3000 steps. I've run several experiments with td3 and sac algorithms, average reward per episode flattens after ...
0
votes
0answers
26 views

Is there any way to determine/estimate the number of rounds for the whole Federated Learning process?

In Federated Learning (FL) the process ends until the model converges or reached certain accuracy. My question: Is there any way to determine/estimate the number of rounds for the whole process?
0
votes
0answers
166 views

How to determine the number of fully connected layers for a convolutional neural network?

How many fully connected layers should be added to a convolutional neural network? Does it depend on input size to the fully connected layer? If so, how do we decide? What if the input size of the ...
0
votes
0answers
44 views

How can I determine the k-NN's hyper-parameters and their range that significantly influence the outcome of classification?

Algorithm: Classification by k-nearest neighbors with Euclidean distance (neighbors.KNeighborsClassifier). Determine the important hyperparameters (2 maximum) that can significantly influence ...
0
votes
0answers
43 views

How to optimize my GAN generator and discriminator models' structures?

I'm using Tensorflow to feed a DCGAN 3000 320x320 colored images of cars. The goal is to generate new cars. I've been training on Google Colab for the past 10 hours or so. I guess I can expect results ...