I have a question regarding features representation for graph convolutional neural network.

For my case, all nodes have a different number of features, and for now, I don't really understand how should I work with these constraints. I can not just reduce the number of features or add meaningless features in order to make the number of features on each node the same - because it will add to much extra noise to the network.

Are there any ways to solve this problem? How should I construct the feature matrix?

I'll appreciate any help and if you have any links to papers that solve this problem.

  • $\begingroup$ Hi and welcome to AI SE! The addition of dummy values isn't that uncommon in deep learning. For example, when doing sequence processing, one may require all sequences to have the same length. In that case, the sequences may be padded with dummy values. (Anyway, although I am familiar with GDL and graph NNs, I am not qualified enough to give an answer to your question). $\endgroup$ – nbro Feb 6 '20 at 14:53
  • $\begingroup$ Hi @nbro and thanks for your answer. May I ask you, how to choose the dummy values in this case? Do you have any idea of methods that can be used, so that I don't broke completely the distribution? $\endgroup$ – PasDeSence Feb 6 '20 at 14:58
  • 1
    $\begingroup$ Honestly, I should not suggest something that may not be appropriate. But maybe try some dummy values that are consistent with your representation of the features, train the networks, see the results, and iterate (i.e try other ideas). For example, whenever there are missing features, you could have zeros, which, when multiplied around, will have the effect of zeroing contributions. I hope someone else will provide more help, but GDL is a quite new area, so many techniques have yet to be developed. It's also been a while I've not read a GDL paper, so I am a bit rusty :) $\endgroup$ – nbro Feb 6 '20 at 15:09

The simplest way I could come with is to pad with 0 each feature which is not present. You said that you're going to add too much noise to the network, but I don't see the problem (please correct me if I'm wrong). For example we have two nodes, the first one has only 2 features with the 3rd one missing and the second node has all features X=[[1,2,0], [3,4,5]]. Now we can project the nodes to a hidden representation (pretty common). I'm going to use a weight matrix of W=[[1], [1], [1]]. The output of XW will be [[3], [12]]. Now let's add a new feature to the second node X=[[1,2,0, 0], [3,4,5,6]] and apply the same transformation W=[[1], [1], [1], [1]] the output will be [[3], [18]] you can see that the first node is not affected by the number of missing features.

Another way you could achieve this if you don't want to use the projection could be using a mask. For example give the same example above we could create a mask M=[[1,1,0], [1,1,1]] where each entry represents if a specific feature is present in a specific node. Now usually a GCN layer is defined as H=f(AHW) where A is the adjacency matrix. We could change the propagation rule to H=f(AH*MW) where * is the pointwise multiplication. Like this if a node is missing a feature it can not "access" information from others that are having that feature.

  • 1
    $\begingroup$ Hello @razvanc92, thank you. I like the masking idea, thank you for the suggestion. $\endgroup$ – PasDeSence Feb 6 '20 at 15:19

My immediate suggestion would be to zero-fill the missing values, but I recalled the below comment suggesting a more sophisticated method:

Karim: How to deal with different size of feature vectors?

Nabila: That's a problem I'm actually working on. I've seen that you can create separate networks for each type of node feature, and sort of project them - so train them separately, and project them to the same size.

Or you can do concatenation so you don't have to worry about that, but at some point they all need to be the same size to do classification at the end.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.