Let's say I have a function $f(p_{i,t})$ and want to optimize $p_{i,t}$ so that I minimize a certain loss. Here, $p_{i,t}$ is the individual $i$ at iteration $t$. Now, $p_{i,t}$ can live anywhere between 0 and $N$, where $N$ is a potentially large number.

The update rule for $p$ (or mutation operator) is given by

$$p_{i,t+1} = p_{i,t} + \eta \cdot \text{rand(-1, 1)}$$

I would like to know how to choose the step size $\eta$. Should it be fixed or variable? Should it be proportional to the parameter's magnitude? Is there any good reason to choose one over the other?



You must log in to answer this question.

Browse other questions tagged .