I trained a neural network which makes a regression to a Poincarè Disk Model with radius $r = 1$.
I want to optimize using the hyperbolic distance
$$ \operatorname{arcosh} \left( 1 + \frac{2|pq|^2|r|^2}{(|r|^2 - |op|^2)(|r|^2 - |oq|^2)} \right) $$
where $|op|$ and $|oq|$ are the distances of $p$ and respectively $q$ to the centre of the disk, $|pq|$ the distance between $p$ and $q$, $|r|$ the radius of the boundary circle of the disk and $\operatorname{arcosh}$ is the inverse hyperbolic function of hyperbolic cosine.
But there is a problem
In the Poincarè Disk Model with $r = 1$, the distance is defined only for vectors which have norm less than $1$.
A neural network does not know this rule, so it can predict vectors with norm greater than $1$.
So, I tried to use the distance defined in a space with $r = 2$, and it works very well for the learning task, but I'm doubtful because the distance doesn't scale in a linear way.
Will there be unwanted effects, in your opinion?