I am working on graph optimisation problem using DQN - the graph is represented as an adjacency matrix and an agent moves through this matrix removing edges between nodes (add a 0) or adding edges between nodes (add a 1) - I am playing around with different state spaces and am now trying one with which I am a little confused on its representation.

The state principally is an embedding vector of the node the agent is at, a numpy array (of dimension (64,)) - this is the direct input into the neural network.

I am also playing around now with providing the agents position also along with the embedding vector to see if this improves learning. The agents position would simply be its coordinates on the adjacency matrix (i.e, row 5, col 17).

Question: Is it ok to provide a 66 dimensional vector (66,) with the agents coordinates as the first two values followed by the 64 dimension embedding vector? so it might look like:

([5, 17, 0.7, -0.1...*64])

Due to the size of the adjacency matrix I cannot one-hot encode the agents position, so in principle can the agent learn if the direct coordinates are also part of the state space information as above? - I know this is how cartpole works, but I know also that the state space is continuous, and adjacency matrix co-ordinates in my example are fixed.

thank you


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.