14 votes

How could we build a neural network that is invariant to permutations of the inputs?

Here is a few that might be what you are looking for: Deep Sets, https://papers.nips.cc/paper/6931-deep-sets.pdf BRUNO: A Deep Recurrent Model for Exchangeable Data, https://arxiv.org/pdf/1802.07535....
elgehelge's user avatar
  • 241
13 votes
Accepted

How could we build a neural network that is invariant to permutations of the inputs?

Traditionally, due to the way the network is structured, each input has a set of weights, that are connected to more inputs. If the inputs switch, the output will too. Approach 1 However, you can ...
BlueMoon93's user avatar
5 votes

How could we build a neural network that is invariant to permutations of the inputs?

I have implemented Permutational Layer here using Keras: https://github.com/offchan42/superkeras/blob/master/permutational_layer.py You can call the ...
offchan's user avatar
  • 325
4 votes

How to design a neural network that gets the author name of a piece of art as input?

The most straightforward approach I would recommend would be the one-hot encoding solution without a feature for ''other author''. If you use drop-out during training, the network should learn how to ...
Dennis Soemers's user avatar
  • 10.3k
4 votes
Accepted

How to design a neural network when the number of inputs is variable?

The best option in your case would probably be zero-padding or padding up. This is simply zeroing out inputs for cases in which there is no data. It's done a lot on the borders of images for CNNs. ...
hisairnessag3's user avatar
3 votes

How to constraint the output value of a neural network?

There are many ways of constraining the network's output. Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by ...
BlueMoon93's user avatar
3 votes
Accepted

How would I go about creating a neural network that outputs a non-binary number?

First of all, sigmoid does not output 0 or 1, it outputs any real number in the range between 0 and 1. Furthermore, neural networks don't usually output binary values, unless the output layer uses the ...
Mr. Eivind's user avatar
3 votes
Accepted

How do I design a neural network that breaks a 5-letter word into its corresponding syllables?

I would highly recommend modeling things differently with regard to how letters are presented to the model. While the problem is more natural, perhaps, for a Convolutional or Recurrent Neural Network, ...
Gilad Tsur's user avatar
3 votes

How to design a neural network that gets the author name of a piece of art as input?

I would try to find some proxy features about the author, as opposed to encode the identity of the author. Likely good features of an author include averages of other features about the work (such as ...
Neil Slater's user avatar
  • 32.1k
2 votes

How could we build a neural network that is invariant to permutations of the inputs?

So, a practical application of this with a lot of research is in the deep lidar processing community. In that world, you have to do a lot of computation on point clouds which are completely unordered....
juicedatom's user avatar
2 votes
Accepted

Do all neurons in a layer have the same activation function?

From here: Using other activation functions don’t provide significant improvement in performance and tweaking them doesn’t provide any big improvement. So as per simplicity we use same activation ...
Recessive's user avatar
  • 1,396
1 vote

Are there metrics for image complexity for informing neural network design?

In the realm of computer vision and machine learning, "task" complexity is more important than image complexity. This is one of the major reasons that research has not formally described any ...
Arun Aniyan's user avatar
1 vote
Accepted

How to output an integer/discrete number n with a single output neuron?

Predicting the correct amount of repetitions for an action sounds like a regression task. Turning it into a classification task using a model with n output nodes will lead to several drawbacks, the ...
Edoardo Guerriero's user avatar
1 vote
Accepted

Number of LSTM layers needed to learn a certain number of sequences

From my personal experience, the units hyperparam in LSTM is not necessary to be the same as max sequence length. Add more units to have the loss curve dive faster. ...
Dan D.'s user avatar
  • 1,283
1 vote

Which neural network should I use to approximate a specific but unknown function?

If the concept class specified is $$f(x, y) = k \, \sin(2 \pi f_x x) \, sin(2 \pi f_y y) \\ \land 0 < x < 1 \\ \land 0 < y < 1 \; \text{,}$$ and the optimum fit to example data is ...
Douglas Daseeco's user avatar
1 vote

How to design a neural network when the number of inputs is variable?

I think, the proposed in the other answer CNN and RNN is a bad choice for this particular problem. The input is the unordered sequence of the features, corresponding to each runner, so the input is ...
spiridon_the_sun_rotator's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible