Method to compute the sum when the activation is a continuous function?

Question

Background

My understanding is the input neurons seem to seem to compute a weighted sum moving from one layer to another.

$$ \sum_i a_i w_i = a'_{k} $$

But to compute this weighted sum the sum must be discrete. Is there any known method to compute the sum when the activation is a continuous function? Is the below formula of any consequence problems in artificial intelligence? Can anyone give a specific problem where it might be useful?

My Method

Let $b_r = \sum_{d \mid r} a_d\mu(\frac{m}{d})$. We prove that if the $b_r$'s are small enough, the result is true (where $\mu$ is the mobius function).

Claim: If $\lim_{n \to \infty} \frac{\log^2(n)}{n}\sum_{r=1}^n |b_r| = 0$ and $f$ is smooth, then $$\lim_{k \to \infty} \lim_{n \to \infty} \sum_{r=1}^n a_rf\left(\frac{kr}{n}\right)\frac{k}{n} = \left(\lim_{s \to 1} \frac{1}{\zeta(s)}\sum_{r=1}^\infty \frac{a_r}{r^s}\right)\int_0^\infty f(x)dx.$$

I will not go into the proof of this over but for those who are interested: https://math.stackexchange.com/questions/2888976/a-rough-proof-for-infinitesimals I will merely state what the formula means:

Consider we have a curve $f(x)$ now if one wishes to perform a weighted sum in the limiting case of this function.

Consider the curve $f(x)$. Then splitting it to $k/n= h$ intervals then adding the first strip ($d_1$ times): $ f(h) \cdot d_1$. Then the second strip ($d_2$ times) $ f(2h) \cdot d_2$ times ... And so on . Hence. $d_r$ can be thought of as the weight at $f(rh)$.

Answer 1

In terms of the normal use cases for machine learning, the equation does not have much utility, because:

Consider we have a curve $f(x)$ now if one wishes to . . .

In most AI problems, we don't usually have such a curve as input that can be treated analytically. For instance, there is no such input curve to describe a natural image received by a sensor.

In the vast majority of cases in AI problems, the form of inputs - whether it is images, text, mapping data, robotic telemetry, is going to be multi-dimensional discrete samples from highly complex functions where we don't know the analytical form and can only construct approximations from a set of basis functions. The resulting combination of basis functions could be treated as continuous, integrated etc, but as it would have been constructed from discrete data, the end result would be a lot of computation to end up with something probably less accurate than working direct with the discrete samples. In a lot of cases, the raw data is discrete by definition (e.g. whether someone clicked on a link or replied to a message), so the form of $f(x)$ would be discrete by definition of the problem, and calculus not really applicable.

There might be some interesting use cases in analog signal processing. Using your formula or a variation of it for instance it should be possible to create an analog neural network learning system - a robotic brain that worked with continuous signals and only contained analog components. Such a system would have some interesting advantages - speed of processing, probably low power consumption compared to digital approach, but the accuracy and precision might be lower. E.g. imagine a bot that could steer towards/away from light sources (compared to a digital one that might recognise the faces of people and steer towards/away from them). I would not be at all surprised to find that someone had done just that already, although I am not sure how to search for it.