# How can I use a 2-dimensional feature matrix as the input to a neural network?

How can I use a 2-dimensional feature matrix, rather than a feature vector, as the input to a neural network?

For a WWII naval wargame, I have sorted out the features of interest to approximate the game state $$S$$ at a given time $$t$$.

• they are a maximum of 50 naval task forces and airbases on the map

• each one has a lot of features (hex, weather, number of ships, type, distance to other task forces, distance to objective, cargo, intelligence, combat value, speed, damage, etc.)

The output would be the probability of winning and the level of victory.

• I have seen that AlphaGo and AlphaGo Zero have been processing data as an image 19 x 19 x (17 or 48) features and then a CNN Dec 26 '18 at 12:14

The most obvious way to do this would be to simply "unroll" your matrix into a vector. Your example input matrix would get turned into the following input vector:

$$\left( \begin{array}{} a_1 & a_2 & \dots & a_t & b_1 & b_2 & \dots & b_t & c_1 & c_2 & \dots & c_t \end{array} \right)$$

I don't think there are any other clear ways to use an "input matrix" really. The only benefit I could see in using an input matrix rather than an unrolled vector (if it were possible to do so in whatever way) would be if doing so would somehow enable the learning algorithm to exploit the "domain knowledge" that certain input features are related to each other in special ways (i.e. features in the same row belong to the same unit, and features in the same column are the same "type" of feature, or other way around). Intuitively, I suspect something like this could be accomplished by restricting the number of connections you make to the next layer. For example, you could make a part of the next layer only be connected to all the $$a_i$$ features, a different part connected only to all the $$b_i$$ features, etc. Similarly, you could have a part that is connected only to the $$a_1, b_1, c_1, \dots$$ features, a different part only connected to the $$a_2, b_2, c_2, \dots$$ features, etc. I don't know for sure how well this would work though... just think that it could.

• Then it would make sense to remove the dependency of the units: for instance instead of keeping all distance between them, then add a feature representing the density of friendly units for the given unit. It would still mean 2000 features. Is it too much for feed forward neural network and single computer ? Dec 26 '18 at 12:17
• @CarrierBattles Hmmm... difficult for me to judge if it would be "too much". 2000 features is a lot though. I think it may be too much, definitely wouldn't be surprised if it were... but not 100% sure. Best way is to try and see. Your problem does sound like a large and complex problem in and of itself though (controlling many different units at the same time, like in Real-Time Strategy games, simply is a complex problem). When talking about such complex problems, it often becomes difficult to avoid solutions that are "too much" for a small amount of hardware. Dec 26 '18 at 12:49

(Split the answer from Dennis in two)

A very different approach would be to use a network architecture with recurrence, for example an LSTM. You could treat your input as a "sequence" rather than a matrix (just like a sentence in language processing would be a sequence of inputs), providing your feature vectors for different units as inputs one at a time. This would remove the need of having a giant input layer (with support for 50 units) in cases where only a small portion of them would be used (e.g., if you only have 5 units). There is not really a concept of "time" in your inputs though... ideally the output of your network would be invariant to the order in which you provide it with the different inputs for the different units, but that would not typically be the case with these kinds of architectures in practice.

• This seems a tweak of the usual way to use Recurrent Neural Networks but why not. I see the advantage of having only the right numbers of unit vectors instead a big block filled with zero for missing units. For the relationships between the units, I may strive to simplify the features using some friendly/enemy units density for each unit for instance. For each S(t), shall I always keep logical order (always the same unit first) or a Radom order ? Dec 26 '18 at 12:26
• By using this method and removing nice to have features, I may end up with less than 700 features in average What are the performance of such network compared to the classical perceptron ? Dec 26 '18 at 13:04