1
$\begingroup$

I have a game/simulation that takes a vector of encoded sequences of moves (up, down, left, right). Let's say that these are sequential step taken by an ant moving in a 2D space, starting from the origin. The moves are generated randomly.

I want to know for any game, if the ant gets farther than a certain distance y from the origin (although it might even be closer than y at the end of the game). I would like to classify games into "ant gets further away than y" with value of one, or zero for "ant does not get further away than y". I don't need an AI for this task, I have set this objective as a training goal for myself.

I am able to tell if the last position is past y or not, using a regular feed forward network, I believe it is easier because it is as easy as summing up all the moves, regardless of the order. But to tell if the ant got past y and then got back, that still needs to return one.

I thought I might be able to reach my objective through an RNN, encoding the moves as a sequence of one-hot encoded sequential directions to move towards. Currently, I am using one hidden layer (I tried with different sizes ranging from 10 to 100), backpropagating the loss only at the last step of a single training on a vector, but it seems like the RNN total loss doesn't decrease at all.

Is there any obivious flaw in my simulation, or in the neural network model? Is there a category of problems this could belong to?

$\endgroup$
  • 1
    $\begingroup$ I am having trouble understanding the problem. Or rather why you need any kind of AI here. If you know the move sequence and intend to process it all, then just simulating the moves will give you the distance at each time step, and it is trivial to get the distance if you have co-ordinates. I guess there is an important detail missing, such as not knowing the history of moves? Or it is a toy problem that you are interesting in solving with a neural network for some reason? Also please make clear what "can get further than a certain distance y" means - both the conditional and metric $\endgroup$ – Neil Slater Oct 15 at 21:03
  • $\begingroup$ I'm trying to categorize a sequence of moves into either "gets farther than y" or "doesn't get farther than y". "Can get further" means: at some point during the simulation, the steps taken have led the ant to be farther than y, though there are still some steps left. If this happens, then it should be labeled "gets farther than y", even if the remaining steps bring it closer than y again. Precisely: I don't need an AI, I have set this objective as a training goal to myself. Thank you very much for your time! $\endgroup$ – Genoma Oct 15 at 21:14
  • $\begingroup$ Thank you, I have tried to edit that information into the question. Feel free to adjust it further using edit in case I have changed the question to much. $\endgroup$ – Neil Slater Oct 16 at 6:22
0
$\begingroup$

This kind of problem does not really have a name other than "toy problem" since no-one needs to teach an AI to add up, multiply or divide* - there are already far more reliable and far faster ways to achieve that on any computer. What you are doing here is essentially vector addition, applying a distance metric then setting a true/false value based on a comparison. It would be 2 or 3 lines of code in most high-level programming languages.

Neural networks can learn any function though, so in theory what you want to do is possible. You should not expect results to be perfect, a statistical learner never actually learns the analytical form of a function or process, just the rough "shape" of it.

I have not done your experiment. However, your idea to use a RNN seems reasonable. With the details you have given, I can offer a few pieces of general advice:

  • Use a modern RNN gated architecture, either LSTM or GRU. That's because the point in the sequence of moves where you want to set a "distance exceeded" toggle could be many steps away from the end of a game. The simplest RNNs (with direct loop backs from output to input within a layer) can easily suffer from vanishing gradients in this situation, whilst LSTM and GRU architectures are designed to deal with it.

  • Generate a lot of training data. You will need many examples of both categories before any neural network will home in on what is causing one class or the other to occur. The learning is based on statistics, not reason.

  • Take a look at related LTSM examples that learn to add binary numbers. Repeat those even simpler experiments first, then move on to your own problem. This will avoid some beginner mistakes with poor choices of hyper-parameters, bad implementations etc.


* Humans of course do use intelligence, reasoning and logic to learn reliable procedures for addition, multiplication and division. No doubt someone could be interested in how an AI can replicate that, without starting from built-in capabilties of a CPU (which of course the humans designed and built those procedures into the system at a low level). However, that's at a higher level of AI research than we are dealing with here.

$\endgroup$
  • $\begingroup$ Thank you for the link, it will definitely be a starting point for me. Also thank you very much for your time, it always surprises me how much effort people put into answering me here, it is very important for me! $\endgroup$ – Genoma Oct 16 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.