# CNN Input shape for DQN Q-calculating Network

Context: I want to build a DQN with as CNN for calculating its Q value on each step.

Enviroment's status can be described by the attributes of 3 machines (each one with own attributes). I'd also like to add some history of recent readings so DQN can understand what is happenening and not only a picture of current readings.

Let's say machine A, B and C have attributes {a1, a2, a3}, {b1, b2, b3} and {c1, c2}

I want to pass the last n reads. So CNN input should have something like:

[
[[a_11, a_12, a_13], [b_11, b_12, b_13], [c_11, c_12]],
[[a_21, a_22, a_23], [b_21, b_22, b_23], [c_21, c_22]],
...
[[a_n1, a_n2, a_n3], [b_n1, b_n2, b_n3], [c_n1, c_n2]]
]


Problem: I can't figure out how to represent this in a matrix because different machines have different amount of attributes

My thoughts:

• Each machine is a 1 dimension vector
• 3 machines can be interpreted in a 2D CNN. But not all machines are the same length...
• As 2D does not work, 3D CNN by adding time will not work as well

Suboptimal solution idea:

• Flatten all machine attributes to a single read as [a1, a2, a3, b1, ..., c2]
• Make it 2D with time
• Use 2D CNN with convolusional operations work in columns (not between them).

Questions:

1. Any better way to organize CNN input?
2. Maybe other kind of NN to estimate Q-Values is a better fit in this case?