I have 10 variables as like below

$V_1=1$, $V_2=2$, $V_3=3$, $V_4=4$, $V_5=5$, $V_6=6$, $V_7=7$, $V_8=8$, $V_9=9$ and $V_{10}=10$

Note: Each variable can have any value

Now I want to select the best 3 variables combination as like below

$V_1V_3V_4$ or $V_{10}V_1V_7$ or $V_5V_3V_9$ etc.

The best combination is nothing but the sum of the values of 3 variables in the combination.


Combination 1($V_1V_2V_3$) : 1+2+3 $\Rightarrow$ 6

Combination 2($V_8V_9V_{10}$) : 8+9+10 $\Rightarrow$ 27

In the above example Combination 2($V_8V_9V_{10}$) has the highest sum value. So the Combination 2($V_8V_9V_{10}$) is the best combination here.

Like this, if I have a large number of variables means which machine learning algorithm selects the best combination in all the sense.

Suggest me the best machine learning algorithm for selecting the best variable combinations. Thanks in advance.

  • 1
    $\begingroup$ If you exactly know the rule to pick these 3, why using a machine learning approach? You could just find the 3 biggest values. Even if the rule was other one, if you know how to score these, you could just make all the combinations posible (which are 120), and find out the best one $\endgroup$
    – freesoul
    Mar 6 '19 at 15:58

What you have is an instance of a local search or black-box optimization problem.

There are many AI algorithms to do this, though as commenters have noted, with only 10 binary variables, you're happier to just look at all $2^{10}$ of them.

If you had, say, 60, then looking at them all is no longer a good idea ($2^{60}$ is very large).

The simplest BBO technique is called hill climbing. It looks like this:

  1. Pick a random subset of the variables to be true.
  2. For each variable, determine the effect of flipping that variable from true to false in the current assignment.
  3. Select the 'flip' that leads to the largest value of your function.
  4. If the 'flip' has a larger value of your function than the current assignment, GOTO 2.
  5. Otherwise, return the current assignment.

This amounts to: 'Pick an assignment at random, then keep making small changes that improve it until you can't anymore. Claim that this is the best solution.'

Clearly, this algorithm will not recover the globally best solution with certainty, but you could run it many times, and eventually, it will find it with high probability.

A complete algorithm, that will always uncover the global optimum, is simulated annealing. This algorithm has a temperature parameter, usually denoted T, which changes according to a temperature schedule. If you start T large 'enough', and decrease it slowly 'enough', then it will find the global optimum. Unfortunately, 'enough' depends a lot on the shape of the function you are optimizing, which you generally cannot observe. There are some common guesses for T, like multiplying it by, e.g. 0.9999 at each step in the search, that seem to work okay in practice.

Both of these algorithms are widely available in packages.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.