0
$\begingroup$

I have a textual dataset that has a set of real numbers as labels: L={0.0, 0.33, 0.5, 0.75, 1.0}, and I have a model that takes the text as input and has a Sigmoid output.

If I train the model on this data, will the model keep generating labels that exactly equal one of the values in L? or it might generate, as an example, 0.4?

If not, is there a solution for that?

$\endgroup$
2
  • $\begingroup$ To clarify: You want the model to be able to generate a 0.4 if that would be the best estimate given the input data? $\endgroup$ Feb 3 at 19:32
  • $\begingroup$ yes @NeilSlater $\endgroup$
    – Minions
    Feb 3 at 19:40

2 Answers 2

2
$\begingroup$

As long as you train the model with a proper loss function for regression the model will learn to output any continuous values, not restricted to and most likely not exactly equal to the labels your providing during training, rather an approximation of them based on the level of generalization the model manage to learn.

The range of values the model can learn to output will also depend on the final activation function of your model. Using a sigmoid is indeed a good choice if your labels belong in the range 0-1.

If instead you don't want the model to learn continuous values then you need to frame the task as a classification problem, and convert your set of finite real values labels into a discrete representation, for example using one hot encoding and then train the model with a loss suited for classification.

$\endgroup$
2
$\begingroup$

Many machine learning models used for regression will interpolate their predictions as you seem to want, and can return target values not seen in the training set.

For example, basic linear regression will do this provided at least one of the input variables is also continuous. Also neural networks.

There are some cases where it may not happen:

  • Decision tree-based algorithms (such as random forests) approximate with combinations of step functions, so most will output a set of discrete values. However, in practice this set will include many values that are not in the training set, and it is usually not a real concern, if your goal is accurate predictions for unseen data.

  • If all the input features are discrete, then there will be a discrete set of output values, equal in number to all the possible combinations of input. Again, this may not be a practical concern in your case, and will very likely generate outputs that are not in the training set.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .