# Tag Info

Accepted

### How to train a sample weight model for another ML model?

Sample Reweighting There's actually been a good amount of work on sample reweighting, namely learning a set of weights for each datapoint in your training data. However, the goal of these algorithms ...
• 1,289

### When to use Pruning, Quantization , Distillation and others when optimizing speed

There are many survey papers on the topic of efficient deep learning. In general the decisions for pruning and quantisation are very dependant on your target hardware and downstream task. For example: ...
• 141

### Applicability of Holland's Schema Theorem to Genetic Algorithms with Non-Binary Individual Representations

It's been a while since I've been active in the research community, and it's possible that this is a controversial opinion, but I suspect it's still true that the Schema Theorem is not really used ...
• 611
1 vote

as reported in the original paper: You can see that $v^t$ is a (corrected) exponential moving average of the second moment of the gradient This doesn't mean that the gradient is penalized if it has ...
• 2,293
1 vote

### Can you explain the Hinton's comment "Rprop is equivalent to using the gradient, but also dividing by the size of the gradient"?

"Rprop is equivalent to using the gradient" means Rprop fundamentally relies on information about the sign of the gradient of each weight to determine the direction of weight updates. Like ...
• 2,277
1 vote
Accepted

### The SOTA of derivative-free optimization

The problem is not the input size but the model size. Indeed, derivative-free/zero-order optimization methods usually tend to estimate a descent direction that correlates with some notion of local ...
• 2,293
1 vote

### How to prevent update a pretrained model if a model is optimized with backpropagation?

You can freeze a model's parameters in pytorch using the following code: for param in encoder_model.parameters(): param.requires_grad = False or you can simply ...
1 vote

### Maximize a scoring function within the latent space of a generative model

I doubt that you can condition training on a scoring function, if such scoring is not used during training. Now, assuming your generator is differentiable, it's already been done the "n-step ...
• 2,293
1 vote

### Maximize a scoring function within the latent space of a generative model

For the first suggestion, are you suggesting doing gradient descent with objective $-S$? If so (and $S$ is differentiable), that's definitely possible. I would suggest looking at the PULSE paper. They ...
• 1,289
1 vote
Accepted

### Best way to generate fitness landscape when using higher dimensional data

I understood your question as being "I have a fitness function based on 8 parameters" how can I display that graphically. Visualising a multi-dimensional landscape is a hard problem. If you ...
• 223
1 vote

### When training a CNN, what are the hyperparameters to tune first?

It depends a lot on what type of architecture you are using. However, most of the standard architectures are quite stable and there is no need for much hypreparameter tuning. Choose whether you want ...
• 815

Only top scored, non community-wiki answers of a minimum length are eligible