Let's say we have several vector points. My goal is to distinguish the vectors, so I want to make them far from each other. Some of them are already far from each other, but some of them can be positioned very closely.

I want to get a certain mapping function that can separate such points that are close to each other, while still preserving the points that are already far away from each other.

I do not care what is the form of the mapping. Since the mapping will be employed as pre-processing, it does not have to be differentiable or even continuous.

I think this problem is somewhat similar to 'minimizing the maximum distance ratio between the points'. Maybe this problem can be understood as stretching the crushed graph to a sphere-like isotropic graph.

I googled it for an hour, but it seems that the people are usually interested in selecting the points that have such nice characteristics from a bunch of data, rather than mapping an existing vector points to a better one.

So, in conclusion, I could not find anything useful.

Maybe you can think 'the neural network will naturally learn it while solving classification problem'. But it failed. Because it is already struggling with too many burdens. So, this is why I want to help my network with pre-processing.

  • $\begingroup$ isn't it what kernels do? $\endgroup$ – Alireza May 20 '20 at 20:43
  • $\begingroup$ which kernel do you mean? $\endgroup$ – Jun May 21 '20 at 3:54
  • $\begingroup$ The general concept of kernel trick in kernel methods in machine learning. such as SVM. the kernels do what you're saying: transforming to a new hyperspace which a discriminative hyperplane can easier separate classes. $\endgroup$ – Alireza May 21 '20 at 7:58
  • $\begingroup$ this is interesting, but, why do you want to do this? $\endgroup$ – carlo Jun 19 '20 at 16:58
  • $\begingroup$ "vector points" means "set of points described as vectors" ? Is a 1D, 2D, 3D, ... problem ? The maximum distance will be reached send each point to the infinite (except one of them at origin). Do you have restrictions for the base space ? $\endgroup$ – pasaba por aqui Oct 17 '20 at 17:59

An interesting question. I would start by finding n nearest neighbors of each data point, then calculate their center of mass c and the point's distance d to its nth nearest neighbor. The smaller the d is the larger the density is around a given point. You could then iteratively step every point away from their c in an inverse proportion to the distance d with a suitable step size. This would spread out the clusters.

But this won't help you transform any new points in the dataset, maybe you can learn this arbitrary mapping R^n -> R^n by using an other neural network and apply it to new samples?

This is the first ad-hoc idea which came to my mind. It would be interesting to see a 2D animation of this.

A more rigorous approach might be a variational autoencoder, you can embed the data in a lower dimensional space with approximately normal distribution. But it doesn't guarantee that clusters would be as spread out as you'd like. An alternative loss function would help with that, for example every point's distance to their original nth closest neighbor should be as close to one as possible.

  • $\begingroup$ Thanks for your advise. I also considered about using k-nearest neighbor(kNN) based scheme, but as you know it includes max (or min) function in it. So I think it is very hard to use kNN related metric as a loss function to train VAE, even though the goal mapping itself do not have to be a differentiable one. But maybe some relaxed version of it, such as stochastic NN could be a naive replacement. $\endgroup$ – Jun May 21 '20 at 3:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.