I built a three layer neural network (first is 1D Convolutional and the remaining two are Linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively the probability of failure or success. The NN is trained on a simulation.

The simulation goes this way: it takes 5 angles in radians and calculates the vector sum of 5 vectors having x as module and alpha as angles (taken from the input). It returns 1 if the vector sum has module greater than y, or 0 if it is less than y.

My intention is to be able to tell sequences of radians that will generate vectors with a sum greater than y in module from the ones which won't.

Which would be the best configuration to achieve this? Is the configuration I set up (1D conv layer + 2 Linear Layers) efficient? If so, would it be easy to find the right size for the convolution? Or should I just remove it?

I noticed that if I change the order of the input angles the output of the simulation will be the same. Is there a particular configuration you should use when dealing with these cases?

Thank you all!

  • $\begingroup$ None of the layouts will fit to the problem description. The bottleneck isn't the arrangement of the neurons, but it's the preprocessing step in which the radians angle are converted into neural network representation. If the problem “Encoding Angle Data for Neural Network” remains unsolved, the best neural network layout in the world can't reduce the backpropagation training error. $\endgroup$ – Manuel Rodriguez Oct 8 '19 at 9:08

I'm not completely sure I understand your simulation. What I think you are doing is:

  1. Generate 5 angles specified in radians (Are these always normalized to within (0, $2*\pi$)?).
  2. Interpret each angle as a unit vector in a 2d space.
  3. Add the unit vectors together, yielding a vector that lies somewhere inside a circle with radius 5.
  4. Ask whether the summed vector is more or less than a distance $y$ from the origin of the circle.

If you're doing that, your problem looks like trying to learn a separation of two concentric rings, which is a well known benchmarking problem for classification.

I am reasonably certain you can learn this pattern with several layers of ReLU neurons. I'm not certain that convolutional layers will help you much here. The main patterns I'd expect the network to learn are:

  • perhaps 2-3 layers to learn whether the point lies far away from the origin in each of several different directions.
  • 1 layer to learn where the decision boundary is in each of 4 directions away from the origin.
  • 1 layer to inclusive-OR the 4 decision boundaries together.

My guess is that this is fairly easy to learn with 4 layers of 8 ReLU neurons, or something like it.

  • $\begingroup$ You got the question exactly right. I was not thinking about a unit vector but scaling the unit vector for my custom steps instead, anyway the procedure stays the same. It is very inspiring for me to see you actually taking a guess about the meaning of each layer! Also, very interesting idea of ring separation, a well known problem, which is what I hoped I would find! $\endgroup$ – Genoma Oct 9 '19 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.