How to make NN distinguish between two types of functions (data)?

Question

I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict are the following:

1. a linear function with negative slope
2. a significantly steeper exponential function.

Example inputs for each would look like

1. [10, 9, 8, 7]
2. [11, 1, 0.1, 0]

Sometimes my neural network predicts well, such as the following: Linear

Other times my neural network completely messes up and predicts the exponential decay instead: Poor Prediction on Linear

As you can imagine, this makes the final result a very poor one.

I'm using a NN architecture of NCF because I'm setting up from a matrix setting. That is, the arrays I have generated come from a noisy matrix setting of low rank.

Any suggestions to improve the result? When I only use one function (e.g. just linear or just exponential) I get superb results. However, mixing both those functions together randomly causes my neural network to perform poorly, as described in my question.

Answer 1

How about dividing the problem?

You can first train a classification model that predicts the type of function (linear or exponential).

Then you can use your seperately trained nn depending on the classification output.

P.S.

I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple problem that can be achieved using traditional ML approaches.

Answer 2

As DKDK said, Indeed one could fit both linear and exponential function and see which one has smaller residual, without using any complex AI.

But OTOH this could be a great toy-problem for learning about neural networks. You could have a network with these parts:

• A network with a final sigmoid activation, which predicts whether the function is linear or not. Let's call its prediction C for "class".
• An additional "data path" within the same network, which predicts linear functions. Let's call its prediction L for "linear".
• An additional "data path" within the same network, which predicts exponential functions. Let's call its prediction E for "exponential".
• There is just a single input but two outputs. The first output is C and the second one is C * L + (1 - C) * E.

Then you can train the whole network end-to-end, but this model could output a mix of 20% linear and 80% exponential values. If this is a problem then I'd use three separate networks as DKDK suggested.

Edit: Now that I read the question again I noticed that all your linear functions have a negative slope? Well surely they will be easy to identify by comparing the first and last element of the list. Then this is just a normal regression problem.