1
$\begingroup$

CONTEXT

I am trying to build a regression model that finds the optimal parameters for a given input. The data I am using are point clouds, with N points and 3 coordinates (x,y,z) each. Each point cloud is divided into neighborhoods of constant size and, during inference, a batch of these neighborhoods are fed into the model which outputs a set of parameters. The parameters represent a family of surfaces and the goal is to find parameters such that the surface fits the neighborhood of points as tightly as possible (in the least squares sense).

THE ISSUE

The problem is that each type of parameter must fall into a specific range, otherwise it has no meaning. For example the first two parameters must lie inside [0.1, 1.9], the next three must be strictly positive etc.. I have tried restraining the outputs by adding a scaled sigmoid activation or simply clamping the output to the range that I want. However, it seems that such hacks result in saturation, the model outputs negative values and all the outputs become 0 from clamping.

I can't imagine I'm the first one to encounter such a problem, but I haven't been able to find out a way to solve it. Is there a defacto way of dealing with this situation?

P.S. I am not including details of the model architecture to keep this question general interest, but I will include them upon request, if it helps.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Hi @VlassisFo, I think you could try making a custom loss function, which would add extra punishment when the parameters go over the limit - this might help to discourage your model from spitting out large/wrong values. $\endgroup$
    – mark mark
    Feb 24 at 17:13
  • $\begingroup$ @markmark That's an interesting idea! How would i go about making the loss as strict as possible? This constrain needs to be enforced in as little iterations as possible, as the rest of the training is based on the correctness of these parameters. $\endgroup$
    – VlassisFo
    Feb 24 at 17:24
  • $\begingroup$ For numbers within the acceptable range($[0.1, 1.9]$) your loss could work as normal. If a value, say, $5.9$, you could add normal loss, plus squared distance from the upper bound(in this case it would be $normal loss + (5.9-1.9)^2$). The further away the parameter is, the larger the loss will be. The neural net should be able to pick that up and learn quicker, in theory. $\endgroup$
    – mark mark
    Feb 24 at 18:22
  • $\begingroup$ Beware of the numbers that are below 1 ($0.5^2 = 0.25$), as they become smaller when squared. You could add 1 to all the distances to ensure that they always become larger. $\endgroup$
    – mark mark
    Feb 24 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.