# How to use information on a function to design a neural network learning that function?

I have a function $$g$$ that takes a vector $$x$$ of size $$n$$ and an integer $$k$$ in $$1, \ldots, n$$. I know this function is of the form $$g(x,k) = G\left(\sum_{i=1}^k f(x_{i})\right),$$ where $$f$$ and $$G$$ are some unknown functions.

I want to train a neural network that estimates the function $$g$$. What is the best way to proceed given the structural information I have about the function?

Of course, I can train the neural network without taking the specific structure into account, but I believe this is not the best way. I can also estimate $$n$$ neural networks for $$g(x,1), \ldots, g(x,n)$$, but I guess this is not the best way either. Is there a good / standard way to proceed?

• Do you have any information about f and/or G? Jul 5, 2022 at 15:08

A first observation is that $$f(x)$$ is shared across all the $$n$$ dimensions of $$x$$ so you can approximate it using a fully connected neural network with a 1-d output. You can think of it as a block in your architecture. To get a variable number of $$f(x_i)$$ you just run the whole network on each $$x_i$$ up to $$k$$. I expect this won't be hard to parallelize.
The other function $$G$$ doesn't really need to know anything about $$k$$, it just requires the sum of $$f(x_i)$$ which is nothing but a scalar. Hence, to approximate $$G$$ you just need another fully connected neural network with 1-d input and 1-d output layers.
As for how big the middle layers should be, this depends on many factors and foremost on your data and how complex the functions $$f$$ and $$G$$ are. You'd better experiment a bit on both cases.