1
$\begingroup$

I have a function $g$ that takes a vector $x$ of size $n$ and an integer $k$ in $1, \ldots, n$. I know this function is of the form $$g(x,k) = G\left(\sum_{i=1}^k f(x_{i})\right),$$ where $f$ and $G$ are some unknown functions.

I want to train a neural network that estimates the function $g$. What is the best way to proceed given the structural information I have about the function?

Of course, I can train the neural network without taking the specific structure into account, but I believe this is not the best way. I can also estimate $n$ neural networks for $g(x,1), \ldots, g(x,n)$, but I guess this is not the best way either. Is there a good / standard way to proceed?

$\endgroup$
1
  • $\begingroup$ Do you have any information about f and/or G? $\endgroup$
    – sfotiadis
    Jul 5, 2022 at 15:08

1 Answer 1

0
$\begingroup$

A first observation is that $f(x)$ is shared across all the $n$ dimensions of $x$ so you can approximate it using a fully connected neural network with a 1-d output. You can think of it as a block in your architecture. To get a variable number of $f(x_i)$ you just run the whole network on each $x_i$ up to $k$. I expect this won't be hard to parallelize.

The other function $G$ doesn't really need to know anything about $k$, it just requires the sum of $f(x_i)$ which is nothing but a scalar. Hence, to approximate $G$ you just need another fully connected neural network with 1-d input and 1-d output layers.

As for how big the middle layers should be, this depends on many factors and foremost on your data and how complex the functions $f$ and $G$ are. You'd better experiment a bit on both cases.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .