I have a background in OR and I am new to AI. I know the basics and I currently try to understand the article "Learning Combinatorial Optimization Algorithms over Graphs".

So far I understand the overall procedure, however, part $x$ of EQ (3), is hard to grasp (I reformulated the formula in a more general way that does not require knowledge about the article)

$$\mu = relu(\theta_1 a + \theta_2 b + \theta_3 \underbrace{\sum_{i \in I} relu(\theta_4 c_i)}_{x})$$

with $\theta_1 \in \mathbb{R}^p, \theta_2,\theta_3 \in \mathbb{R}^{p\times p}, \theta_4 \in \mathbb{R}^p$ and $I$ being a set of varying cardinality for each input to be evaluated and $a,b,c_i$ being given inputs.

The authors claim to train the network end to end. If we replace $x$ by some input value (without $\theta_4$) I would be able to follow the idea.

However as far as I understand $x$, $relu(\theta_4 c_i)$ is a little NN on its own with input $c_i$ and output of length $p$. Sure, we can compute the values and sum them up and use them as input, however, I do not understand how the end to end training will work for such a setup where basically two networks exist.

I would be more than thankful for help on this topic.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.