Questions tagged [hyper-parameters]
For questions related to the hyper-parameters of AI models and algorithms, which are parameters that are set before the learning process begins. For example, the number of hidden layers in a feed-forward neural network is usually a hyper-parameter.
93
questions
0
votes
1
answer
45
views
GPT beam search length (number of tokens)
Background: I'm currently trying to use GPT to give me numerical scores, and looking for tips on prompt design, see my previous StackExchange post.
To craft good prompts it seems important to have a ...
0
votes
1
answer
35
views
Patterns binary classification - model doesn't overfit
I am working on a very basic binary classification problem. For each set of four float numbers $(x,y,z,w)$, I want to check if they fall or not into one category.
I have written a model with 3 dense ...
- 103
1
vote
1
answer
143
views
What's the architecture and size of neural-network-based reward models as used in reinforcement learning by human feedback
My rough understanding of RLHF as used for ChatGPT in a nutshell is this:
A reward model is trained using comparisons of different responses
to the same prompt. Human trainers rank these responses ...
0
votes
0
answers
9
views
Incorporating HiPlot and Keras
I just started to learn about Keras and train some models, and I came across HiPlot which is used for tuning hyperparameters. I was wondering if HiPlot can also be used to see what parameters would ...
- 1
0
votes
1
answer
78
views
How to approach a toy classification problem using a neural network?
The toy problem:
50 unique numbers are randomly selected from number 0 to 99.
If number 1 appears in the selection but number 2 doesn't, the selection is labelled as "1".
If number 2 ...
- 1
0
votes
0
answers
73
views
How to increase exploration in IMPALA?
I am using the RLLib library for applying IMPALA algorithm to my environment. I want to know which variables in https://docs.ray.io/en/latest/_modules/ray/rllib/algorithms/impala/impala.html#...
- 113
0
votes
1
answer
48
views
Do different architectures really make a difference or is it just a matter of the training process?
I was wondering which influence different architectures for deep learning truly have on the performance. Of course, substantial changes in the paradigms we use when building neural networks (such as ...
- 121
0
votes
0
answers
21
views
Is it a problem if the target network and main network are the same?
I am currently working with this code that I found on Github: this is a DQN with Graph Neural Network (GNN) on TSP problem.
So, in this architecture, the target network and the main network are the ...
1
vote
1
answer
83
views
Are there any guidelines on picking hyperparameters for Deep Reinforcement Learning?
I am trying to learn machine learning from Andrew NG's Machine learning specialization on Coursera. In the chapter about reinforcement learning Andrew NG said that if you do not select correct ...
0
votes
0
answers
36
views
Which of these 3 mutation rates is the best in terms of performance?
I am need some comments since I am conducting experiments with 3 different mutation rates and hesitate to choose the best one.
I ...
- 101
3
votes
1
answer
1k
views
What's the relationship between number of heads and embedding dimension in Transformers?
I am reading the book: Natural Language Processing with Transformers. It has the following paragraph
Although head_dim does not have to be smaller than the number of embedding dimensions of the ...
- 588
0
votes
2
answers
294
views
What should I do if my validation score is good, but my test score is bad?
I've trained my artificial neural network, and, as per standard practice, I've picked out the one neural network throughout training that did the best on my validation dataset. That is, the neural ...
- 103
1
vote
0
answers
23
views
How to make my neural networks designs more robust
Whenever, I design a neural network to solve a novel problem (requires a novel loss function i.e. not image classification) it always ends up being hypersensitive to batch size and learning rate.
...
- 316
0
votes
1
answer
126
views
Which paper describes the effect of learning_starts in Reinforcement Learning?
I have seen many popular RL libraries have a learning_start parameter. This allows the agent to collect enough experiences before training on the replay_buffer. However, I am unable to find the paper ...
- 588
1
vote
1
answer
218
views
Is the described Q-table considered large?
I never saw any rule of thumb as to what size is said as large for a q-table but I have a Q-table with like 2500 entries. Is it considered large for a tabular approach? Anyone from experience can ...
1
vote
2
answers
2k
views
How many training steps does it usually take to train an RL model?
This is my model average rewards as follow image.
How to tell if it is undertrained or not convergent? How many training steps does it usually take to train an RL model?
And I'm using PPO to train.
- 283
3
votes
1
answer
396
views
Why is a large replay buffer inefficient?
Open AI spin up says
... the replay buffer should be large enough to contain a wide range
of experiences, but it may not always be good to keep everything. If
you only use the very-most recent data, ...
- 131
2
votes
1
answer
378
views
For continuing tasks, is the choice of episode length completely arbitrary?
Let's say I'm training a reinforcement learning agent to act in some environment that perpetually continues to give the agent opportunities to earn rewards, and there is no cap on the score and there ...
- 342
0
votes
1
answer
197
views
Why should data batches in a neural network have an equal size?
Why should data batches in a neural network have an equal size?
I have seen some recent research works on making the batch size dynamic, but still, I can't find an answer to my question.
- 123
2
votes
1
answer
989
views
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
For example, we may want to learn how to reconstruct a relatively low-...
- 297
0
votes
3
answers
130
views
What can I infer if my model is converging extremely fast?
I am running a model with fixed hyperparameters. To my surprise/shock, the model converged extremely fast with the least loss possible.
I want to know the causes of this phenomenon. I have the ...
- 3,581
0
votes
2
answers
199
views
Is it true that batch size of form $2^k$ gives better results?
I am confused among the following in selecting the batch size for my model.
#1: powers of 2
I generally see that batch sizes are in powers of two: 32, 64, 128, 256.
#2: maximum GPU
Suppose my GPU ...
- 3,581
2
votes
1
answer
95
views
What components of reinforcement learning influence the result the most?
I'm working on my thesis concerning a reinforcement learning problem and am trying to prioritise my time on different components of it:
Formalising the agent environment (like the design of state-, ...
- 115
3
votes
2
answers
427
views
When can I call an entity a hyperparameter?
As per my knowledge, any entity that is learnable by a training algorithm can be called a parameter. Weights of a neural network are called parameters because of this reason only.
But I have doubts ...
- 3,581
0
votes
2
answers
1k
views
Is there any relationship between the batch size and the number of epochs?
I am currently running a program with a batch size of 17 instead of batch size 32. The benchmark results are obtained at a batch size of 32 with the number of epochs 700.
Now I am running with batch ...
- 3,581
0
votes
2
answers
103
views
How to reduce the number of clusters produced by the Markov Clustering Algorithm?
I have used the Markov Clustering Algorithm (MCL) to cluster tweets, based on their similarity. However, I got a too high number of clusters, and most of the clusters have only one tweet. Any ...
0
votes
2
answers
107
views
Why data required for hyperparameter tuning is considered as an additional data?
Any parametric model may have parameters as well as hyperparameters. Learning algorithm deals with parameters and hyperparameters should be dealt outside learning algorithm. Consider the following ...
- 3,581
3
votes
1
answer
384
views
Why is the input layer of a neural network usually not counted?
I came across the following statement from the caption of figure 7.8 from the textbook Neural Networks and Neural Language Models
the input layer is usually not counted when enumerating layers
Why ...
- 3,581
7
votes
3
answers
14k
views
How to determine the embedding size?
When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden ...
- 877
1
vote
1
answer
167
views
How many singular vectors do we need to calculate for SVD?
In the geometrical interpretation of SVD, the data points that we have need to be imagined as points in high dimensional space (say $d$-dimensional space). But we need to find a hyperplane in $k-$...
- 3,581
0
votes
2
answers
166
views
How to ensure that the ES-HyperNEAT algorithm generates an ANN in the substrate?
I'm trying to implement the ES-HyperNEAT algorithm using the original paper, as well as the pseudocode provided in the official user page. Occasionally, the algorithm would be unable to generate a ...
- 1
3
votes
0
answers
2k
views
Optimal episode length in reinforcement learning
I have a custom environment for stock trading where an episode can be as long as 2000-3000 steps. I've run several experiments with td3 and sac algorithms, average reward per episode flattens after ...
- 331
5
votes
2
answers
3k
views
What are the best hyper-parameters to tune in reinforcement learning?
Obviously, this is somewhat subjective, but what hyper-parameters typically have the most significant impact on an RL agent's ability to learn? For example, the replay buffer size, learning rate, ...
- 243
2
votes
0
answers
41
views
Is there an optimal number of species for NEAT?
Is there an optimal number of species for NEAT?
Since too low and too high is bad, I am thinking about adjusting the threshold of the distance function at runtime in order to have the number of ...
- 51
1
vote
0
answers
21
views
Hyperparameters for Reproducing the Results of IRGAN on MovieLens 1M
I am trying to reproduce results reported for IRGAN (information retrieval GAN) on the MovieLens 1M dataset. The results I want to reproduce and their sources are listed in the table below.
Model
...
- 111
3
votes
1
answer
101
views
What is the most statistically acceptable method for tuning neural network hyperparameters on very small datasets?
Neural networks are usually evaluated by dividing a dataset into three splits:
training,
validation, and
test
The idea is that critical hyperparameters of the network such as the number of epochs ...
- 401
2
votes
1
answer
615
views
Is it a good idea to use different width and height of the kernel in a CNN?
I always see that the width and height of the kernel are the same. But is it a good idea to use different numbers?
Recently I tried to use GoogLeNet (which expects images to be 224x224) on my images (...
user40943
2
votes
0
answers
61
views
Given a 2-layer GCN, can we choose the dimensions of the 2nd weight matrix, such that this architecture has the same capacity as a 1-layer GCN?
This might be more of a question about nested function classes:
For $k$ class node classification in a graph with $n$ nodes, and $d$ feature vector.
I want to compare
Architecture I: the GCN model of ...
- 121
2
votes
0
answers
207
views
How will the filter size affect the transpose convolution operation?
After a series of convolutions, I am up-sampling a compressed representation, I was curious what is the methodology I should follow to choose an optimum kernel size for up-sampling.
How will the ...
- 51
1
vote
0
answers
174
views
How does noise input size affect fake image generation with GANs?
In Generative Adversarial Networks, the Generator takes noise vector as input and feeds it forward to create an image. The noise vector consists of random numbers sampled from the normal distribution. ...
- 753
1
vote
1
answer
122
views
For episodic tasks with an absorbing state, why can't we both have $\gamma=1$ and $T= \infty$ in the definition of the return?
For episodic tasks with an absorbing state, why can't $\gamma=1$ and $T= \infty$?
In Sutton and Barto's book, they say that, for episodic tasks with absorbing states that becomes an infinite sequence, ...
- 717
2
votes
1
answer
3k
views
What should the value of epsilon be in the Q-learning?
I am trying to understand Reinforcement Learning and already explored different Youtube videos, blog posts, and Wikipedia articles.
What I don't understand is the impact of $\epsilon$. What value ...
- 253
1
vote
2
answers
584
views
Why is an embedding of dimension 400 enough to represent 70000 words?
I am learning PyTorch on Udacity. In lesson 8, section 11: Training the Model, the instructor writes:
Then I have my embedding and hidden dimension. The embedding dimension is just a smaller ...
- 181
1
vote
1
answer
226
views
Should the range and initial values of weights and biases be adjusted to fit input and output data?
As a routine (in typical everyday tasks) of a data scientist, should they usually decide about weights and biases range and initial values as a function of which data they are planning to insert as ...
- 181
2
votes
1
answer
131
views
Which hyperparameters in neural network are accesible to users adjustment
I am new to Neural Networks and my questions are still very basic.
I know that most of neural networks allow and even ask user to chose hyper-parameters like:
amount of hidden layers
amount of ...
- 181
1
vote
1
answer
98
views
Why one unit in the layers of neural network is not enough?
In a deep connected network, when every unit gets all the input features(X) so it has one parameter for every feature and every unit tweaks its parameters for loss optimization. What if we use only ...
- 111
3
votes
1
answer
897
views
What is the intuition behind the number of filters/channels for each convolutional layer?
After having chosen the number of layers for a convolutional neural network, we must also choose the number of filters/channels for each convolutional layer.
The intuition behind the filter's spatial ...
- 33
1
vote
1
answer
2k
views
What is the best activation function for the embedding layer in a deep auto-encoder?
I am designing a deep autoencoder for graph embedding (exactly node embedding) following this paper SDNE. In the original paper, they used the sigmoid activation for all hidden layers in the ...
- 27
1
vote
1
answer
2k
views
How to determine the number of hidden layers and units of a deep auto-encoder?
I am using a deep autoencoder for my problem. However, the way I choose the number of hidden layers and hidden units in a hidden layer is still based on my feeling.
The size of the model that ...
- 27
3
votes
1
answer
2k
views
How should I choose the target's update frequency in DQN?
I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we ...