Using an 'operation ID' as a neural network input

Question

Sorry if this is basic or covered elsewhere, I am just starting here and I wasn't able to find an answer, but I might have not been searching for the right thing. So:

I am training a neural network to predict current draw in a system. There are a number of obvious numerical inputs, like temperature, counting rate, voltage, etc.

The most predictive thing, however, is what operation the system is doing. So like, if it's doing a 'calibration' then the current profile is much different than if it's in 'standby'. I know that I can just use a different network for each operation, but in this case I have a couple hundred different macros defined and I don't want to have 200+ neural networks retrain all the time.

I also know that I can have a digital value as an input, but my understanding is that it has to be either 0/1. Also, the relationship to operation is not at all correlated - so operation 100 is not necessarily more current draw than 99 or less than 101.

So, is there a way to have an operation ID or something factor in, but not have it be in the linear combination mathematically? So, basically, tell the system to do a different training based on ID or something? I'll be using python and scikit-learn.

Thanks!

Answer 1

Great question.

The operational ID is actually a category, or categorical input. Each category requires its own input signal into the neural network. Each signal is actually an activation which is multiplied by a weight and added to all the other weights and inputs. This means that if you use the same input signal for all your operational IDs then 101 is more important than 50. As you have mentioned these inputs are very different, and therefore must have a binary input for each operation id. If you add more IDs then you have to retrain the neural network.

You also mentioned you will be using scikit-learn, which has built in functions for category data. Have a look at this link on the one hot encoder.