I am wondering if I can use neural networks to find features importances in similar manner as it can be done for random forests or decision trees and if so, how to do it?

I would like to use it on tabular time series data (not images). The reason why I want to find importances on neural networks not on decision trees is that NNs are more complicated algorithms so using NNs might point out some correlations that are not seen by simple algorithms and I need to know what features are found to be more useful with that complicated correlations.

I am not sure if I made it clear enough, please let me know if I have to explain something more.


This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to train on. I would us a rnn for with a softmax output layer that ranks features by performance.

You can find a good explanation of softmax here: https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax basically it will assign probability values for each output node with all of these values adding up to 1.0

  • $\begingroup$ Could you please explain the procedure in a bit more detailed way or give some reference link? $\endgroup$ – Makintosz Aug 13 '19 at 5:03
  • $\begingroup$ I added a bit of info on softmax, what are you using for optimization? knowing that i may be able to provide a bit more insight into how i would set this up $\endgroup$ – nickw Aug 13 '19 at 21:36
  • $\begingroup$ Thanks for more info! I am using tensorflow library, mostly with Adam optimizer (trying others as well) with custom learning rate. $\endgroup$ – Makintosz Aug 14 '19 at 7:22
  • $\begingroup$ at a ten thousand foot level this could be done by deciding the amount of features or the min probability as a cut of and then creating a net with softmax output layer with a node for each feature, you will likely want to use a reinforcement learning optimization technique as you will need to evaluate if how well the features the net picks perform in order to optimize, you could use neuroevolution with a fitness function for this or i imagine other rl algos would work too. $\endgroup$ – nickw Aug 20 '19 at 14:17

It's possible. I've used the olden() function from NeuralNetTools. Please take a look at this example found online: http://blogs2.datall-analyse.nl/2016/02/19/rcode_variable_importance_neural_network/

  • $\begingroup$ A bit more theory and proper explanation of potential solutions is warranted here. $\endgroup$ – hisairnessag3 Aug 12 '19 at 14:00
  • $\begingroup$ can you elaborate down your answer for the op $\endgroup$ – quintumnia Aug 12 '19 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.