Can neural networks deal with unbounded numbers as inputs?

Question

I want to train an ANN. The problem is that the input features are completely unbounded (There are no boundaries as maximum and minimum for them).

For example, the following input vectors $(42, 54354354)$ and $(0.4, 47239847329479324732984732947)$ are both valid.

I know RNNs that can add up input neurons, which are pretty similar to my case, but the number of the digits was limited in all of the implementations.

Is there a way to implement an ANN that can add up the input numbers of any magnitude?

Answer 1

Depending on the activation functions in the first layer, this is not a real problem, many functions such as $\tanh(x)$ and $\sigma(x)$ (sigmoid) have asymptotic upper and lower bounds, so the huge input values swamp all other inputs in the corresponding neurons, but the output is well-behaved.

However, a conversion before the input layer may be appropriate for some kind of data. For example, you might want to take the $\log(x)$ of the input values such that the initial weight networks work with the relation of the values instead of their absolute magnitude. This may be useful for audio data such as speech or music and many other measurements of signals that may be attenuated before the measurement.

If your numbers don't actually represent magnitudes of some kind but arbitrary sequences of digits, you should use a network that can deal with sequences, of course.