I'm wondering if there is a NN that can achieve the following task:

Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\mathbf{v}\|$.

The dimension $d$ can be fixed, say $2$.

To achieve this, it seems to me that we need to use NN to do three functions: square, square-root, and division. But I don't know if a NN can do all of these.

  • $\begingroup$ Vectors are used for regression tasks, tasks involving definite mathematical operations is a big no no due to the unbounded nature of it. If you notice, a particular NN has a maximum value which it can output for a given dataset e.g if we assume all positive weights then the maximum magnitude of input will produce max magnitude of output and that is the bound, if inputs beyond that is provided at testing the outputs will simply be incorrect. If you train NN on a large enough range for this problem it's possible that the NN outputs reasonable correct values given the input is within the range. $\endgroup$ – DuttaA Sep 24 '19 at 16:57
  • $\begingroup$ Thus you see NNs are not really useful for maths operations. For example a square root graph maybe approximated by a straight line (within a range). If you have 2 straight lines you can approximate it better by breaking the range in 2 parts and assigning each range to 1 line. Like this a large enough NN can approximate a range, but it's inefficient. intmath.com/integration/5-trapezoidal-rule.php Check the pic $\endgroup$ – DuttaA Sep 24 '19 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.