0
$\begingroup$

I have a background in OR and I am new to AI. I know the basics and I currently try to understand the article "Learning Combinatorial Optimization Algorithms over Graphs".

So far I understand the overall procedure, however, part $x$ of EQ (3), is hard to grasp (I reformulated the formula in a more general way that does not require knowledge about the article)

$$\mu = relu(\theta_1 a + \theta_2 b + \theta_3 \underbrace{\sum_{i \in I} relu(\theta_4 c_i)}_{x})$$

with $\theta_1 \in \mathbb{R}^p, \theta_2,\theta_3 \in \mathbb{R}^{p\times p}, \theta_4 \in \mathbb{R}^p$ and $I$ being a set of varying cardinality for each input to be evaluated and $a,b,c_i$ being given inputs.

The authors claim to train the network end to end. If we replace $x$ by some input value (without $\theta_4$) I would be able to follow the idea.

However as far as I understand $x$, $relu(\theta_4 c_i)$ is a little NN on its own with input $c_i$ and output of length $p$. Sure, we can compute the values and sum them up and use them as input, however, I do not understand how the end to end training will work for such a setup where basically two networks exist.

I would be more than thankful for help on this topic.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.