Considering a black box optimization problem on non-linear, non-convex function where we want to minimize an objective function.

One way to assess the quality of an optimizer is to look at the best solution it finds. However that doesn't give us any information on how much of the parameter space the optimizer had to explore to come up with these solutions.

Therefore I was wondering if there are metrics quantifying how much of the parameter space is explored ?

  • 2
    $\begingroup$ Number of iterations is kind of measure of parameter space explored. If you want something more advanced you first have to specify strictly what do you mean by "explored". You have to specify type of coverage associated with set of parameter vectors. $\endgroup$ – mirror2image Sep 23 '19 at 18:32

It is somewhat common to measure the number of operations involved in the search. This is only really useful if you're doing scientific work, because the computational cost of measuring it accurately is quite high. For example, if you were using a GA that used tournament-based fitness selection, and wanted to compare it to one that used round-robin selection, counting the number of evaluations of individuals would be a good measure of total computational effort.

In Genetic Programming, it is fairly common to build the measurement of the total op count into the interpreter you write for the programs. You can then compare that directly to the number of evals times the length of the genomes for something like a GA.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.