What are the advantages and disadvantages of using meta-heuristic algorithms on optimization problems? Simply, why do we use meta-heuristic algorithms, like PSO, over traditional mathematical techniques, such as linear, non-linear and dynamic programming?

I actually have a good understanding of meta-heuristic algorithms and I know how they work. For example, one advantage of this kind of algorithms is that they can find an optimal solution in a reasonable time.

However, my lack of knowledge about other methods and techniques brought this question to my mind.

  • $\begingroup$ The only reason is that it has some areas where it is effective, problems like size of parts in a car for optimum performance (paraphrased from a well known user in this website), other than that its quite useless in my opinion. $\endgroup$ – user9947 Jan 27 '20 at 12:33
  • $\begingroup$ @DuttaA I am not sure what you mean by "size of parts in a car for optimum performance", but I think you're quite incorrect when you say that meta-heuristics are useless. I know meta-heuristics are currently used for the vehicle routing problem (and similar combinatorial problems). $\endgroup$ – nbro Jan 27 '20 at 12:42
  • $\begingroup$ @nbro what I meant to say is multiple features affecting some parameter which does not have a very good loss function defined so that it can be optimised by a NN. Those Meta algos are very less and far in-between. I think you are probably talking about ACO? There are many other Meta algos which are redundant, repititive and quite useless IMO. $\endgroup$ – user9947 Jan 27 '20 at 13:06
  • $\begingroup$ @DuttaA You're saying that certain meta-heuristics are redundant, but this statement can be applied also to neural networks or any other field/methods. For example, I could say that the perceptron is redundant or that a 1-hidden layer MLP is redundant given an LSTM for a sequence modeling task. $\endgroup$ – nbro Jan 27 '20 at 13:31
  • $\begingroup$ @nbro many well known stupid Meta algos has been given different names, these are not just architectural changes which will work well on different problems. The main non redundant algos are PSO, genetic, ACO and maybe 1 animal algo. The rest are all stupid variations for this stuff. I can't cite the source but apparently elseiver or some other journal had to ban the use of paper named 'some random animal' optimisation algo. $\endgroup$ – user9947 Jan 27 '20 at 13:48

Meta-heuristics are particularly suited for combinatorial optimization problems, given that, although they are not usually guaranteed to find the optimal global solution, they can often find a sufficiently good solution in a decent amount of time. So, they are an alternative to exhaustive search, which would take exponential time. For example, ant colony optimization algorithms have been used to approximately (or exactly, in the case of small or medium-size instances) solve the travelling salesman problem, whose decision version is an NP-complete problem (which means that, unless P=NP, there is no polynomial-time solution to solve it).

Meta-heuristics can also be easily applied to many problems, given that they are not problem-specific. For example, in the case of genetic algorithms, you just need to encode the possible solutions, but, in principle, you can apply genetic algorithms to a wide range of problems, although they may not always be the best solution to each of these problems. Moreover, as opposed to gradient-based optimisation algorithms, there's no need for the gradient of the objective function. For instance, in the case of genetic algorithms, you just need a way of evaluating the solutions (e.g. the fitness or the novelty).

Meta-heuristics often incorporate some form of randomness in order to escape from local minima. Ant-colony optimization algorithms or simulated annealing are two good examples of this approach.

If you are still interested in meta-heuristics, the book Clever Algorithms: Nature-Inspired Programming Recipes (by Jason Brownlee) is a very good resource for learning about them. There's also a Github repository with the implementation of the algorithms described in this book.

  • $\begingroup$ I was already preparing this answer before the last edit to the question, which indicates that the OP is already aware of an advantage of meta-heuristics that I mention in this answer. $\endgroup$ – nbro Jan 27 '20 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.