# Is it possible to do K-nearest-neighbours before training DNN

The following X-shape alternated pattern can be separated quite well and super fast by K-nearest Neighbour algorithm (go to http://ml-playground.com to test it):

However, DNN seems to face great struggles to separate that X-shape alternated data. Is it possible to do K-nearest before DNN, ie. set the DNN weights somehow to simulate the result of K-nearest before doing DNN training?

Another place to test the X-shape alternated data: https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html

• If data already labeled clustering wouldn't add anything new. The only known way to set weights for DNN is training form random initialization. The convergence of training is based on randomness of weights (it's used in all existing proofs). It could be taht in future there will be non-random way to initialize weights, but those method necessarily will be very complex and putting additional constraint on them probably would be extremely hard. Anyway there is no such non-random methods for now. – mirror2image Sep 26 '19 at 5:56
• yeah, but if we can just initialise weights in some greedy manner – datdinhquoc Sep 26 '19 at 6:04

2. Generate $$N$$ new points uniformly at random from the input space.
3. Label the $$N$$ new points using the K-NN model.
4. Fit your DNN to the original dataset, plus the $$N$$ new points.